* wip(go): return auth token with user on create org owner;
* wip(go): prevent use of default tenant slug ;
* wip(go): upsert settings for tenant in ctx;
* wip(go): use passed ctx when updating user; remove password validation from ValidateUser (already validated in UpdateUser);
* wip(go): qualify all tenant id usages with table name;
* wip(go): use passed ctx
* wip(go): return old enrollment key type on create;
* wip(go): prevent null networks return;
* wip(go): wire ctx in dns/nameserver functions; wire ctx in host registration functions;
* wip(go): wire ctx in node functions;
* wip(go): wire ctx in node functions;
* wip(go): wire ctx in extclient functions;
* wip(go): remove not needed field;
* wip(go): use passed ctx;
* wip(go): use unique index instead of primary key;
* wip(go): wire ctx in extclient apis;
* wip(go): wire ctx in extclient apis;
* wip(go): remove org id check from global scoped get tenant api;
* wip(go): add scope slug to discover response;
* wip(go): wire ctx in get users;
* wip(go): set tenant id in posture check violations;
* wip(go): upsert membership instead of user;
* wip(go): pass correct len;
* wip(go): optimize (single loop);
* wip(go): unify todos;
* wip(go): add todo;
* wip(go): wire context for network get;
* wip(go): wire context;
* wip(go): wire context;
* wip(go): wire ctx when getting tenant superadmin;
* wip(go): add network creator to network admin group;
* wip(go): check for user membership on sync;
* wip(go): add methods with strict membership check;
* wip(go): remove unwanted feature flags;
* wip(go): report per-tenant usage during license validation;
* wip(go): move email setting from tenant to org;
* wip(go): pass org scope to email init;
* wip(go): remove old tenant email settings usages; use org settings to get email settings
* wip(go): use get email settings from org settings;
* wip(go): remove email settings migration;
* wip(go): migrate email settings from server to org settings;
* wip(go): assign global network admin group membership only for super-admin and admins;
* wip(go): assign global network admin group membership only for super-admin and admins;
* wip(go): add membership alias to filter;
* wip(go): skip clearing local license validation response cache;
* wip(go): extend license validation response for multi-tenancy;
* wip(go): add functions to get sole tenant and org;
* wip(go): use tenant-specific feature flags;
* wip(go): add tenant status type;
* wip(go): remove old hook comment;
* wip(go): expect ctx during license validation;
* wip(go): always create default org and tenant for ce;
* wip(go): add tenant-specific check for license validation error;
* wip(go): cache license validation response in memory;
* wip(go): add todo to handle multiple-tenant for integration pull;
* wip(go): pass ctx during license validation;
* wip(go): add todo for making changes later; run hook separately for each tenant;
* wip(go): return specific errors when license validation response not in cache;
* wip(go): use retryable.httpclient for calling license validation api;
* wip(go): process license validation;
* wip(go): license validation function;
* wip(go): migration on license validation;
* wip(go): org settings not expected;
* wip(go): remove expiry from license validation response;
* wip(go): add limits and limits enforcement to license validation response;
* wip(go): remove expiry usages;
* wip(go): add arg to clear cache;
* wip(go): add arg to clear cache;
* wip(go): skip license validation if cached resp;
* wip(go): allow org_id and tenant_id query params;
* wip(go): upsert tenant membership on sync;
* wip(go): pass ctx to delete network roles;
* wip(go): create tenant membership on org owner creation on ce;
* wip(go): add tenant orchestrator;
* wip(go): use tenant orchestrator;
* wip(go): add default settings for tenant;
* wip(go): make org owner superadmin on all tenants;
* wip(go): simplify SyncOrgAndTenants;
* wip(go): retry license validation only 3 times
* wip(go): store invalid err;
* wip(go): add function to check if limits are to be enforced on a tenant;
* wip(go): pass tenant scope if possible when checking for license validation error;
* wip(go): remove merge errors;
* wip(go): add functions for limit checks;
* wip(go): wire host limit check;
* wip(go): wire ingress limit check;
* wip(go): wire egress limit check;
* wip(go): wire client limit check;
* wip(go): wire network limit check;
* wip(go): wire user limit check;
* wip(go): use correct errors;
* wip(go): skip telemetry for msps;
* wip(go): disable telemetry on msp tenants;
* wip(go): err check;
* wip(go): uncomment log event;
* wip(go): get sso state to get scope for non-headless oauth login;
* wip(go): inline delete pending user;
* wip(go): allow both org and tenant scope for pending users table;
* wip(go): add migration for pending user table during set tenant id and rekey org/tenant id;
* wip(go): update queries to use new scope fields in pending users table;
* wip(go): allow both org and tenant scope for user invites table;
* wip(go): update queries to use new scope fields;
* wip(go): add migration for rekey-ing org/tenant id and setting tenant id;
* wip(go): inline delete user invite;
* wip(go): set scope when creating invite;
* wip(go): get user count with membership is ctx is tenant scoped;
* wip(go): add methods to upsert and delete org memberships;
* wip(go): add global valid tenant and org roles map;
* wip(go): add scope query in user invite get by email;
* wip(go): add support for upsert and delete of org and tenant membership both;
* wip(go): infer scope for user invite apis;
* wip(go): wire ctx in get user invite by email;
* wip(go): allow no scope headers to be passed in infer scope apis;
* wip(go): add validation and checks for org-scoped invite api handlers;
* wip(go): wire ctx to is pending user check;
* wip(go): infer scope for pending user apis;
* wip(go): set default role based on ctx scope;
* wip(go): move mfa to tenant membership table; move account status to membership tables;
* wip(go): migrate account status and mfa settings;
* wip(go): update account status on sync;
* wip(go): use update mfa to update mfa setting;
* wip(go): set new membership settings on user creation;
* wip(go): use membership where required;
* wip(go): wire ctx on key unlink refs;
* wip(go): use r.Context directly without reinitializing db;
* wip(go): wire ctx in acls;
* wip(go): add tenant scoped primary key constructor;
* wip(go): use tenant scoped primary key for acls, dns, extclients and tags;
* wip(go): wire ctx for get groups in network;
* wip(go): wire ctx for get groups;
* wip(go): scope non-uuid group ids by tenant id;
* wip(go): migrate non-uuid group ids to scoped ids;
* wip(go): don't expect scope id in invite validation and signup apis;
* wip(go): wire ctx for get matching user; allow org scope when querying by external idp id;
* wip(go): add mfa to org;
* wip(go): scope preauth token;
* wip(go): add fallback for older auth tokens;
* wip(go): wire ctx for preauth token creation;
* wip(go): migrate mfa settings;
* wip(go): resolve mfa settings for inherited auth;
* wip(go): update mfa for org;
* wip(go): infer scope for totp apis;
* wip(go): use ctx when passed;
* wip(go): infer scope for list user apis;
* wip(go): ensure set primary keys don't cause errors during fetch;
* wip(go): make primary key for integrations uuid and store provider in separate key;
* wip(go): migrate integration ids;
* wip(go): use provider instead of id everywhere;
* wip(go): remove gorm column;
* wip(go): wire ctx for igw egress creation;
* wip(go): wire ctx for igw egress update;
* wip(go): wire ctx for host registration;
* wip(go): wire ctx for host registration;
* wip(go): merge errors;
* wip(go): merge errors;
* wip(go): wire ctx; remove default scope usages;
* wip(go): use get with membership;
* wip(go): org scope license validation;
* wip(go): remove debug log;
* wip(go): remove scoping on list roles;
* wip(go): ignore invite creation if already exists;
* wip(go): add is msp func to check if deployment is msp deployment;
* wip(go): ensure username is email for msp users;
* wip(go): add email validated check on user table;
* wip(go): mark email validated when oauth user;
* wip(go): handle gorm error;
* wip(go): add tenant id to siem grpc schema;
* wip(go): read tenant from scoped ctx;
* wip(go): pass scoped ctx to siem grpc client;
* wip(go): remove unused publisher;
* wip(go): publish all tenant exporter updates on pull;
* wip(go): fix wg usage;
* wip(go): tenant scope the ctx when getting server info;
* wip(go): add tenant id to enrollment key token;
* wip(go): make host register tenant scope;
* wip(go): add default settings;
* wip(go): use correct checks for is msp tenant;
* wip(go): add email validation extension;
* wip(go): add pages for validation check;
* wip(go): use extension for validating user;
* wip(go): add invite type;
* wip(go): create only one validation invite per user email;
* wip(go): add validate email handler;
* wip(go): add email builder;
* wip(go): add user invite type migration logic;
* wip(go): only log email send failures;
* wip(go): add tenant id to flows table;
* wip(go): add func to check if connected to clickhouse;
* wip(go): migrate clickhouse;
* wip(go): add tenant id to mdm and edr state;
* wip(go): add delete methods for tables;
* wip(go): add tenant teardown method;
* wip(go): handle tenant deletion;
* wip(go): clean up flow logs on tenant deletion;
* wip(go): add update org membership role id method;
* wip(go): add transfer org ownership api;
* wip(go): move audit logs retention period to org settings;
* wip(go):get retention period from org settings;
* wip(go): remove old audit logs usage;
* wip(go): migrate audit logs retention period;
* wip(go): unmarshal decodes nums as float64;
* wip(go): run flow logs cleanup for all tenants in one go;
* wip(go): add audit logs retention period and smtp settings to tenant settings table for non-msp tenants;
* wip(go): resolve retention period and smtp settings correctly;
* wip(go): handle retention period and smtp settings updates and fetches correctly;
* wip(go): overlay retention period and smtp settings on server settings;;
* wip(go): upsert overlayed settings;
* wip(go): add azure tenant id to org settings;
* wip(go): tenant scope list roles;
* wip(go): add noop creds;
* wip(go): use first instead of find;
* wip(go): use correct roles when transfering ownership;
* wip(go): delete access tokens only if tenant scoped;
* wip(go): make delete user work for both org and tenant scope;
* wip(go): delete extclients only if tenant scoped;
* wip(go): check if caller is allowed to delete user;
* wip(go): infer scope for update user;
* wip(go): prevent org-owner elevation;
* wip(go): permissions check for update;
* wip(go): delete user access tokens only if tenant scope;
* wip(go): delete extclients only if tenant scope;
* wip(go): move upsert membership in upsert user;
* wip(go): set api host when constructing server config;
* wip(go): fix static errors;
* wip(go): fix test cases;
* wip(go): use correct get ns order;
* wip(go): use correct get pending host order;
* wip(go): use correct get access token order;
* wip(go): use correct get event order;
* wip(go): return error response on emqx user creation error;
* wip(go): store correct value in cache;
* wip(go): return better error response;
* wip(go): return better error response;
* wip(go): use cache when listing egress acls;
* wip(go): wire ctx in UserHasNetworkGroupAccess;
* wip(go): remove double scope wrapping;
* wip(go): add security check for list and get org;
* wip(go): ratelimit discover api;
* wip(go): expect sole org id in get and create org owner;
* wip(go): wrap create org owner in tx;
* feat(go): add user schema;
* feat(go): migrate to user schema;
* feat(go): add audit fields;
* feat(go): remove unused fields from the network model;
* feat(go): add network schema;
* feat(go): migrate to network schema;
* refactor(go): add comment to clarify migration logic;
* fix(go): test failures;
* fix(go): test failures;
* feat(go): change membership table to store memberships at all scopes;
* feat(go): add schema for access grants;
* feat(go): remove nameservers from new networks table; ensure db passed for schema functions;
* feat(go): set max conns for sqlite to 1;
* fix(go): issues updating user account status;
* NM-236: streamline operations in HA mode
* NM-236: only master pod should subscribe to updates from clients
* refactor(go): remove converters and access grants;
* refactor(go): add json tags in schema models;
* refactor(go): rename file to migrate_v1_6_0.go;
* refactor(go): add user groups and user roles tables; use schema tables;
* refactor(go): inline get and list from schema package;
* refactor(go): inline get network and list users from schema package;
* fix(go): staticcheck issues;
* fix(go): remove test not in use; fix test case;
* fix(go): validate network;
* fix(go): resolve static checks;
* fix(go): new models errors;
* fix(go): test errors;
* fix(go): handle no records;
* fix(go): add validations for user object;
* fix(go): set correct extclient status;
* fix(go): test error;
* feat(go): make schema the base package;
* feat(go): add host schema;
* feat(go): use schema host everywhere;
* feat(go): inline get host, list hosts and delete host;
* feat(go): use non-ptr value;
* feat(go): use save to upsert all fields;
* feat(go): use save to upsert all fields;
* feat(go): save turn endpoint as string;
* feat(go): check for gorm error record not found;
* fix(go): test failures;
* fix(go): update all network fields;
* fix(go): update all network fields;
* feat(go): add paginated list networks api;
* feat(go): add paginated list users api;
* feat(go): add paginated list hosts api;
* feat(go): add pagination to list groups api;
* fix(go): comment;
* fix(go): implement marshal and unmarshal text for custom types;
* fix(go): implement marshal and unmarshal json for custom types;
* fix(go): just use the old model for unmarshalling;
* fix(go): implement marshal and unmarshal json for custom types;
* NM-271:Import swap: compress/gzip replaced with github.com/klauspost/compress/gzip (2-4x faster, wire-compatible output). Added sync import.
Two sync.Pool variables (gzipWriterPool, bufferPool): reuse gzip.Writer and bytes.Buffer across calls instead of allocating fresh ones per publish.
compressPayload rewritten: pulls writer + buffer from pools, resets them, compresses at gzip.BestSpeed (level 1), copies the result out of the pooled buffer, and returns both objects to the pools.
* feat(go): remove paginated list networks api;
* feat(go): use custom paginated response object;
* NM-271: Improve server scalability under high host count
- Replace stdlib compress/gzip with klauspost/compress at BestSpeed and
pool gzip writers and buffers via sync.Pool to eliminate compression
as the dominant CPU hotspot.
- Debounce peer update broadcasts with a 500ms resettable window capped
at 3s max-wait, coalescing rapid-fire PublishPeerUpdate calls into a
single broadcast cycle.
- Cache HostPeerInfo (batch-refreshed by debounce worker) and
HostPeerUpdate (stored as side-effect of each publish) so the pull API
and peer_info API serve from pre-computed maps instead of triggering
expensive per-host computations under thundering herd conditions.
- Warm both caches synchronously at startup before the first publish
cycle so early pull requests are served instantly.
- Bound concurrent MQTT publishes to 5 via semaphore to prevent
broker TCP buffer overflows that caused broken pipe disconnects.
- Remove manual Disconnect+SetupMQTT from ConnectionLostHandler and
rely on the paho client's built-in AutoReconnect; add a 5s retry
wait in publish() to ride out brief reconnection windows.
* NM-271: Reduce server CPU contention under high concurrent load
- Cache ServerSettings with atomic.Value to eliminate repeated DB reads
on every pull request (was 32+ goroutines blocked on read lock)
- Batch UpdateNodeCheckin writes in memory, flush every 30s to reduce
per-checkin write lock contention (was 88+ goroutines blocked)
- Enable SQLite WAL mode + busy_timeout and remove global dbMutex;
let SQLite handle concurrency natively (reads no longer block writes)
- Move ResetFailedOverPeer/ResetAutoRelayedPeer to async in pull()
handler since results don't affect the cached response
- Skip no-op UpsertNode writes in failover/relay reset functions
(early return when node has no failover/relay state)
- Remove CheckHostPorts from hostUpdateFallback hot path
- Switch to pure-Go SQLite driver (glebarez/sqlite), set CGO_ENABLED=0
* fix(go): ensure default values for page and per_page are used when not passed;
* fix(go): rename v1.6.0 to v1.5.1;
* fix(go): check for gorm.ErrRecordNotFound instead of database.IsEmptyRecord;
* fix(go): use host id, not pending host id;
* NM-271: Revert pure-Go SQLite and FIPS disable to verify impact
Revert to CGO-based mattn/go-sqlite3 driver and re-enable FIPS to
isolate whether these changes are still needed now that the global
dbMutex has been removed and WAL mode is enabled. Keep WAL mode
pragma with mattn-compatible DSN format.
* feat(go): add filters to paginated apis;
* feat(go): add filters to paginated apis;
* feat(go): remove check for max username length;
* feat(go): add filters to count as well;
* feat(go): use library to check email address validity;
* feat(go): ignore pagination if params not passed;
* fix(go): pagination issues;
* fix(go): check exists before using;
* fix(go): remove debug log;
* NM-271: rm debug logs
* NM-271: check if caching is enabled
* NM-271: add server sync mq topic for HA mode
* NM-271: fix build
* NM-271: push metrics in batch to exproter over api
* NM-271: use basic auth for exporter metrics api
* fix(go): use gorm err record not found;
* NM-271: Add monitoring stack on demand
* NM-271: -m arg for install script should only add monitoring stack
* fix(go): use gorm err record not found;
* NM-271: update docker compose file for prometheus
* NM-271: update docker compose file for prometheus
* fix(go): use user principal name when creating pending user;
* fix(go): use schema package for consts;
* NM-236: rm duplicate network hook
* NM-271: add server topic to reset idp hooks on master node
* fix(go): prevent disabling superadmin user;
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* fix(go): swap is admin and is superadmin;
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* fix(go): remove dead code block;
https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837937
* fix(go): incorrect message when trying to disable self;
https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837934
* NM-271: fix stale peers on reset_failovered pull and add HTTP timeout to metrics exporter
Run the failover/relay reset synchronously in the pull handler so the
response reflects post-reset topology instead of serving stale cached
peers. Add a 30s timeout to the metrics exporter HTTP client to prevent
PushAllMetricsToExporter from blocking the Keepalive loop.
* NM-271: fix gzip pool corruption, MQTT topic mismatch, stale settings cache, and reduce redundant DB fetches
- Only return gzip.Writer to pool after successful Close to prevent
silently malformed MQTT payloads from a previously errored writer.
- Fix serversync subscription to exact topic match since syncType is
now in the message payload, not the topic path.
- Prevent zero-value ServerSettings from being cached indefinitely
when the DB record is missing or unmarshal fails on startup.
- Return fetched hosts/nodes from RefreshHostPeerInfoCache so
warmPeerCaches reuses them instead of querying the DB twice.
- Compute fresh HostPeerUpdate on reset_failovered pull instead of
serving stale cache, and store result back for subsequent requests.
* NM-271: fix gzip writer pool leak, log checkin flush errors, and fix master pod ordinal parsing
- Reset gzip.Writer to io.Discard before returning to pool so errored
writers are never leaked or silently reused with corrupt state.
- Track and log failed DB inserts in FlushNodeCheckins so operators
have visibility when check-in timestamps are lost.
- Parse StatefulSet pod ordinal as integer instead of using HasSuffix
to prevent netmaker-10 from being misidentified as master pod.
* NM-271: simplify masterpod logic
* fix(go): use correct header;
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* fix(go): return after error response;
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* fix(go): use correct order of params;
https://github.com/gravitl/netmaker/pull/3910#discussion_r2929593036
* fix(go): set default values for page and page size; use v2 instead of /list;
* NM-271: use host name
* Update mq/serversync.go
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* NM-271: fix duplicate serversynce case
* NM-271: streamline gw updates
* Update logic/auth.go
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* Update schema/user_roles.go
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* fix(go): syntax error;
* fix(go): set default values when page and per_page are not passed or 0;
* fix(go): use uuid.parse instead of uuid.must parse;
* fix(go): review errors;
* fix(go): review errors;
* Update controllers/user.go
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* Update controllers/user.go
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* NM-163: fix errors:
* Update db/types/options.go
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* fix(go): persist return user in event;
* Update db/types/options.go
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* NM-271: signal pull on ip changes
* NM-163: duplicate lines of code
* NM-163: fix(go): fix missing return and filter parsing in user controller
- Add missing return after error response in updateUserAccountStatus
to prevent double-response and spurious ext-client side-effects
- Use switch statements in listUsers to skip unrecognized
account_status and mfa_status filter values
* NM-271: signal pull req on node ip change
* fix(go): check for both min and max page size;
* NM-271: refresh node object before update
* fix(go): enclose transfer superadmin in transaction;
* fix(go): review errors;
* fix(go): remove free tier checks;
* fix(go): review fixes;
* NM-271: streamline ip pool ops
* NM-271: fix tests, set max idle conns
* NM-271: fix(go): fix data races in settings cache and peer update worker
- Use pointer type in atomic.Value for serverSettingsCache to avoid
replacing the variable non-atomically in InvalidateServerSettingsCache
- Swap peerUpdateReplace flag before draining the channel to prevent
a concurrent replacePeers=true from being consumed by the wrong cycle
---------
Co-authored-by: VishalDalwadi <[email protected]>
Co-authored-by: Vishal Dalwadi <[email protected]>
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* enable egress routing peers with tags
* remove tag from egress when deleted
* fix egress tag functionality
* filter duplicate egress ips
* set default stun server if unset
* add version to status api
* sync deleted node udpate host deletion
* feat(go): allow different session durations for client apps;
* feat(go): assume call is from netdesk app if header absent;
* feat(go): allow header;
* feat(go): set client jwt validity duration on migration.
* revert inet gws from acl policies
* add egress range with metric for inet gw
* link pro inet funcs
* add timestamp params to activity apis
* Add default dns domain
* move relevant acl and tag code to CE and Pro pkgs
* intialise pro acl funcs
* list gateways by user access
* check user gw access by policies
* filter out user policies on CE
* filter out tagged policies on CE
* fix ce acl comms
* allow gateways tag
* allow gateway tag on CE, remove failover and gw check on acl policy
* add gw rules func to pro
* add inet gw support on CE
* add egress acl API
* add egress acl API
* fix(go): set is_gw when converting api node to server node;
* fix(go): set is_gw when converting api node to server node;
* fix policy validity checker for inet gws
* move dns option to host model
* fix node removal from egress policy on delete
* add migration logic for ManageDNS
* fix dns json field
* fix nil error on node tags
* add egress info to relayed nodes
---------
Co-authored-by: Vishal Dalwadi <[email protected]>
* check host ports on join
* if 443 not available fallback to 51821
* if 443 not available fallback to 51821
* add config for auto delete of offline nodes
* autocleanup offline nodes
* delete offline nodes on startup
* fix relay via join token
* feat: api access tokens
* revoke all user tokens
* redefine access token api routes, add auto egress option to enrollment keys
* add server settings apis, add db table for settigs
* handle server settings updates
* switch to using settings from DB
* fix sever settings migration
* revet force migration for settings
* fix server settings database write
* fix revoked tokens to be unauthorized
* remove unused functions
* convert access token to sql schema
* switch access token to sql schema
* fix merge conflicts
* fix server settings types
* bypass basic auth setting for super admin
* add TODO comment
* publish peer update on settings update
* chore(go): import style changes from migration branch;
1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.
* remove nat check on egress gateway request
* Revert "remove nat check on egress gateway request"
This reverts commit 0aff12a189.
* feat(go): add db middleware;
* feat(go): restore method;
* feat(go): add user access token schema;
* fix user auth api:
* re initalise oauth and email config
* set verbosity
* sync auto update settings with hosts
* sync auto update settings with hosts
* mask secret and convert jwt duration to minutes
* convert jwt duration to minutes
* notify peers after settings update
* compare with curr settings before updating
* send host update to devices on auto update
---------
Co-authored-by: Vishal Dalwadi <[email protected]>
* feat: api access tokens
* revoke all user tokens
* redefine access token api routes, add auto egress option to enrollment keys
* fix revoked tokens to be unauthorized
* remove unused functions
* convert access token to sql schema
* switch access token to sql schema
* revoke token generated by an user
* add user token creation restriction by user role
* add forbidden check for access token creation
* revoke user token when group or role is changed
* add default group to admin users on update
* fix token removal on user update
* fix token removal on user update
* set default metrics port 8889
* set default metrics port 51821
* add metrics port to server config
* bind caddy only on tcp
* add var for pulling files
* add new line
* update peer update model
* check if port is not zero
* set replace peer to false on pull
* do not replace peers on failover sync
* remove debug log
* add old peer update fields for backwards compatibility
* add old json tag
* add debug log in caller trace func
* add live status of node
* handle static node status
* add public IP field to server configuration
* get public Ip from config
* improve node status logic
* improvise status check
* use only checkin status on old nodes
---------
Co-authored-by: the_aceix <[email protected]>
* add switch for manage dns
* manage DNS sync publish
* add dns sync api
* add manageDNS field in peerUpdate
* add default dns for extClent if manage dns enabled
* add DEFAULT_DOMAIN for internal DNS lookup
* move DNSSync to peerUpdate
* fix empty host in network issue
* sync up dns when custom dns add/delete
* fix custom DNS ip4/ipv6 validator issue
* generalise smtp config
* copy over smtp vars
* env new line
* fix master key api access
* comment user tests
* fix network and user invite for master key access
* remove email sender type
* user mgmt commands
* check user role on CE
* user role nmtcl cmds
* user groups commands
* fix role and groups command
* fix user create cmd
* add usage info
* rm user role check
* fix user update cmd
* fix static check
* add backwards comptability support for extclient api for mobile
* rm debug logs
* set frontend url from base domain if empty
* generalise smtp config
* copy over smtp vars
* env new line
* fix master key api access
* comment user tests
* fix network and user invite for master key access
* remove email sender type
* user mgmt models
* define user roles
* define models for new user mgmt and groups
* oauth debug log
* initialize user role after db conn
* print oauth token in debug log
* user roles CRUD apis
* user groups CRUD Apis
* additional api checks
* add additional scopes
* add additional scopes url
* add additional scopes url
* rm additional scopes url
* setup middlleware permission checks
* integrate permission check into middleware
* integrate permission check into middleware
* check for headers for subjects
* refactor user role models
* refactor user groups models
* add new user to pending user via RAC login
* untracked
* allow multiple groups for an user
* change json tag
* add debug headers
* refer network controls form roles, add debug headers
* refer network controls form roles, add debug headers
* replace auth checks, add network id to role model
* nodes handler
* migration funcs
* invoke sync users migration func
* add debug logs
* comment middleware
* fix get all nodes api
* add debug logs
* fix middleware error nil check
* add new func to get username from jwt
* fix jwt parsing
* abort on error
* allow multiple network roles
* allow multiple network roles
* add migration func
* return err if jwt parsing fails
* set global check to true when accessing user apis
* set netid for acls api calls
* set netid for acls api calls
* update role and groups routes
* add validation checks
* add invite flow apis and magic links
* add invited user via oauth signup automatically
* create invited user on oauth signup, with groups in the invite
* add group validation for user invite
* update create user handler with new role mgmt
* add validation checks
* create user invites tables
* add error logging for email invite
* fix invite singup url
* debug log
* get query params from url
* get query params from url
* add query escape
* debug log
* debug log
* fix user signup via invite api
* set admin field for backward compatbility
* use new role id for user apis
* deprecate use of old admin fields
* deprecate usage of old user fields
* add user role as service user if empty
* setup email sender
* delete invite after user singup
* add plaform user role
* redirect on invite verification link
* fix invite redirect
* temporary redirect
* fix invite redirect
* point invite link to frontend
* fix query params lookup
* add resend support, configure email interface types
* fix groups and user creation
* validate user groups, add check for metrics api in middleware
* add invite url to invite model
* migrate rac apis to new user mgmt
* handle network nodes
* add platform user to default role
* fix user role migration
* add default on rag creation and cleanup after deletion
* fix rac apis
* change to invite code param
* filter nodes and hosts based on user network access
* extend create user group req to accomodate users
* filter network based on user access
* format oauth error
* move user roles and groups
* fix get user v1 api
* move user mgmt func to pro
* add user auth type to user model
* fix roles init
* remove platform role from group object
* list only platform roles
* add network roles to invite req
* create default groups and roles
* fix middleware for global access
* create default role
* fix nodes filter with global network roles
* block selfupdate of groups and network roles
* delete netID if net roles are empty
* validate user roles nd groups on update
* set extclient permission scope when rag vpn access is set
* allow deletion of roles and groups
* replace _ with - in role naming convention
* fix failover middleware mgmt
* format oauth templates
* fetch route temaplate
* return err if user wrong login type
* check user groups on rac apis
* fix rac apis
* fix resp msg
* add validation checks for admin invite
* return oauth type
* format group err msg
* fix html tag
* clean up default groups
* create default rag role
* add UI name to roles
* remove default net group from user when deleted
* reorder migration funcs
* fix duplicacy of hosts
* check old field for migration
* from pro to ce make all secondary users admins
* from pro to ce make all secondary users admins
* revert: from pro to ce make all secondary users admins
* make sure downgrades work
* fix pending users approval
* fix duplicate hosts
* fix duplicate hosts entries
* fix cache reference issue
* feat: configure FRONTEND_URL during installation
* disable user vpn access when network roles are modified
* rm vpn acces when roles or groups are deleted
* add http to frontend url
* revert crypto version
* downgrade crytpo version
* add platform id check on user invites
---------
Co-authored-by: the_aceix <[email protected]>
* NET-1440 scale test changes
* fix UT error and add error info
* load metric data into cacha in startup
* remove debug info for metric
* add server telemetry and hasSuperAdmin to cache
* fix user UT case
* update sqlite connection string for performance
* update check-in TS in cache only if cache enabled
* update metric data in cache only if cache enabled and write to DB once in stop
* update server status in mq topic
* add failover existed to server status update
* only send mq messsage when there is server status change
* batch peerUpdate
* code changes for scale for review
* update UT case
* update mq client check
* mq connection code change
* revert server status update changes
* revert batch peerUpdate
* remove server status update info
* batch peerUpdate
* code changes based on review and setupmqtt in keepalive
* set the mq message order to false for PIN
* remove setupmqtt in keepalive
* add peerUpdate batch size to config
* update batch peerUpdate
* recycle ip in node deletion
* update ip allocation logic
* remove ip addr cap
* remove ippool file
* update get extClient func
* remove ip from cache map when extClient is removed
* add batch peerUpdate switch
* set batch peerUpdate to true by default
---------
Co-authored-by: Max Ma <[email protected]>
* add api to check if failover node existed
* remove 5 minute peerUpdate
* update peerUpdate to trigger pull
* update Action name to SignalPull
* revert the peerUpdate from SignalPull
* fix getfailover error issue
* rm acls creation for on-prem emqx
* remove use of acls
* add additional broker status field on status api
* NET-1165: Remove creation of acls on emqx (#2996)
* rm acls creation for on-prem emqx
* remove use of acls
* add additional broker status field on status api
* comment out mq reconnect logic
* configure mq conn params
* add metric_interval in ENV for publishing metrics
* add metric_interval in ENV for publishing metrics
* update PUBLISH_METRIC_INTERVAL env name
* revert the mq setttings back
* fix error nil issue
---------
Co-authored-by: abhishek9686 <[email protected]>
Co-authored-by: Abhishek K <[email protected]>
* add pending users api
* insert user to pending users on first time oauth login
* add pending user check on headless login
* fix conflicting apis
* no records error
* add allowed emails domains for oauth singup to config
* check if user is allowed to signup
* NET-655
* Updated HostPull structure to include EgressRoutes and FirewallUpdate models.
* added ServerVersion structure to hostpull model
* added ServerVersion structure to hostpull model
* removed ServerVersion structure
* removed ServerVersion structure
* added egressroute and fwupdate to hostpull handler
* add host update fallback handler
* set broker type on server cfg
* use actual host password to create emqx user
---------
Co-authored-by: Christopher Blaha <[email protected]>
Co-authored-by: Abhishek Kondur <[email protected]>
* cache enabled option, cache hosts data if only enabled
* cache nodes only when enabled
* cache extclients only when enabled
* cache acls only when enabled
Vishal Dalwadi