210 Commits
Author SHA1 Message Date
abhishek9686 881c9d120d NM-295: publish WSS uplink endpoints for selfsigned and external termination
Add TLS mode, listen addr, optional public hostname, and proxy-mode client
port (443 / TCP_PROXY_PUBLIC_PORT) when building tcp_proxy_endpoint.
2026-08-27 23:50:50 +05:30
340ae1521f NM-295: publish TCP uplink flags and peer tcp_proxy_endpoint (#4106)
* wip(go): use default jwt for org tokens;

* wip(go): detach request context from goroutine context;

* wip(go): allow single tenant creation;

* wip(go): remove debug logs;

* wip(go): derive host tenant id from enrollment key tenant id;

* wip(go): skip server settings migration for new deployment;

* wip(go): return auth token with user on create org owner;

* wip(go): prevent use of default tenant slug ;

* wip(go): upsert settings for tenant in ctx;

* wip(go): use passed ctx when updating user; remove password validation from ValidateUser (already validated in UpdateUser);

* wip(go): qualify all tenant id usages with table name;

* wip(go): use passed ctx

* wip(go): return old enrollment key type on create;

* wip(go): prevent null networks return;

* wip(go): wire ctx in dns/nameserver functions; wire ctx in host registration functions;

* wip(go): wire ctx in node functions;

* wip(go): wire ctx in node functions;

* wip(go): wire ctx in extclient functions;

* wip(go): remove not needed field;

* wip(go): use passed ctx;

* wip(go): use unique index instead of primary key;

* wip(go): wire ctx in extclient apis;

* wip(go): wire ctx in extclient apis;

* wip(go): remove org id check from global scoped get tenant api;

* wip(go): add scope slug to discover response;

* wip(go): wire ctx in get users;

* wip(go): set tenant id in posture check violations;

* wip(go): upsert membership instead of user;

* wip(go): pass correct len;

* wip(go): optimize (single loop);

* wip(go): unify todos;

* wip(go): add todo;

* wip(go): wire context for network get;

* wip(go): wire context;

* wip(go): wire context;

* wip(go): wire ctx when getting tenant superadmin;

* wip(go): add network creator to network admin group;

* wip(go): check for user membership on sync;

* wip(go): add methods with strict membership check;

* wip(go): remove unwanted feature flags;

* wip(go): report per-tenant usage during license validation;

* wip(go): move email setting from tenant to org;

* wip(go): pass org scope to email init;

* wip(go): remove old tenant email settings usages; use org settings to get email settings

* wip(go): use get email settings from org settings;

* wip(go): remove email settings migration;

* wip(go): migrate email settings from server to org settings;

* wip(go): assign global network admin group membership only for super-admin and admins;

* wip(go): assign global network admin group membership only for super-admin and admins;

* wip(go): add membership alias to filter;

* wip(go): skip clearing local license validation response cache;

* wip(go): extend license validation response for multi-tenancy;

* wip(go): add functions to get sole tenant and org;

* wip(go): use tenant-specific feature flags;

* wip(go): add tenant status type;

* wip(go): remove old hook comment;

* wip(go): expect ctx during license validation;

* wip(go): always create default org and tenant for ce;

* wip(go): add tenant-specific check for license validation error;

* wip(go): cache license validation response in memory;

* wip(go): add todo to handle multiple-tenant for integration pull;

* wip(go): pass ctx during license validation;

* wip(go): add todo for making changes later; run hook separately for each tenant;

* wip(go): return specific errors when license validation response not in cache;

* wip(go): use retryable.httpclient for calling license validation api;

* wip(go): process license validation;

* wip(go): license validation function;

* wip(go): migration on license validation;

* wip(go): org settings not expected;

* wip(go): remove expiry from license validation response;

* wip(go): add limits and limits enforcement to license validation response;

* wip(go): remove expiry usages;

* wip(go): add arg to clear cache;

* wip(go): add arg to clear cache;

* wip(go): skip license validation if cached resp;

* wip(go): allow org_id and tenant_id query params;

* wip(go): upsert tenant membership on sync;

* wip(go): pass ctx to delete network roles;

* wip(go): create tenant membership on org owner creation on ce;

* wip(go): add tenant orchestrator;

* wip(go): use tenant orchestrator;

* wip(go): add default settings for tenant;

* wip(go): make org owner superadmin on all tenants;

* wip(go): simplify SyncOrgAndTenants;

* wip(go): retry license validation only 3 times

* wip(go): store invalid err;

* wip(go): add function to check if limits are to be enforced on a tenant;

* wip(go): pass tenant scope if possible when checking for license validation error;

* wip(go): remove merge errors;

* wip(go): add functions for limit checks;

* wip(go): wire host limit check;

* wip(go): wire ingress limit check;

* wip(go): wire egress limit check;

* wip(go): wire client limit check;

* wip(go): wire network limit check;

* wip(go): wire user limit check;

* wip(go): use correct errors;

* wip(go): skip telemetry for msps;

* wip(go): disable telemetry on msp tenants;

* wip(go): err check;

* wip(go): uncomment log event;

* wip(go): get sso state to get scope for non-headless oauth login;

* wip(go): inline delete pending user;

* wip(go): allow both org and tenant scope for pending users table;

* wip(go): add migration for pending user table during set tenant id and rekey org/tenant id;

* wip(go): update queries to use new scope fields in pending users table;

* wip(go): allow both org and tenant scope for user invites table;

* wip(go): update queries to use new scope fields;

* wip(go): add migration for rekey-ing org/tenant id and setting tenant id;

* wip(go): inline delete user invite;

* wip(go): set scope when creating invite;

* wip(go): get user count with membership is ctx is tenant scoped;

* wip(go): add methods to upsert and delete org memberships;

* wip(go): add global valid tenant and org roles map;

* wip(go): add scope query in user invite get by email;

* wip(go): add support for upsert and delete of org and tenant membership both;

* wip(go): infer scope for user invite apis;

* wip(go): wire ctx in get user invite by email;

* wip(go): allow no scope headers to be passed in infer scope apis;

* wip(go): add validation and checks for org-scoped invite api handlers;

* wip(go): wire ctx to is pending user check;

* wip(go): infer scope for pending user apis;

* wip(go): set default role based on ctx scope;

* wip(go): move mfa to tenant membership table; move account status to membership tables;

* wip(go): migrate account status and mfa settings;

* wip(go): update account status on sync;

* wip(go): use update mfa to update mfa setting;

* wip(go): set new membership settings on user creation;

* wip(go): use membership where required;

* wip(go): wire ctx on key unlink refs;

* wip(go): use r.Context directly without reinitializing db;

* wip(go): wire ctx in acls;

* wip(go): add tenant scoped primary key constructor;

* wip(go): use tenant scoped primary key for acls, dns, extclients and tags;

* wip(go): wire ctx for get groups in network;

* wip(go): wire ctx for get groups;

* wip(go): scope non-uuid group ids by tenant id;

* wip(go): migrate non-uuid group ids to scoped ids;

* wip(go): don't expect scope id in invite validation and signup apis;

* wip(go): wire ctx for get matching user; allow org scope when querying by external idp id;

* wip(go): add mfa to org;

* wip(go): scope preauth token;

* wip(go): add fallback for older auth tokens;

* wip(go): wire ctx for preauth token creation;

* wip(go): migrate mfa settings;

* wip(go): resolve mfa settings for inherited auth;

* wip(go): update mfa for org;

* wip(go): infer scope for totp apis;

* wip(go): use ctx when passed;

* wip(go): infer scope for list user apis;

* wip(go): ensure set primary keys don't cause errors during fetch;

* wip(go): make primary key for integrations uuid and store provider in separate key;

* wip(go): migrate integration ids;

* wip(go): use provider instead of id everywhere;

* wip(go): remove gorm column;

* wip(go): wire ctx for igw egress creation;

* wip(go): wire ctx for igw egress update;

* wip(go): wire ctx for host registration;

* wip(go): wire ctx for host registration;

* wip(go): merge errors;

* wip(go): merge errors;

* wip(go): wire ctx; remove default scope usages;

* wip(go): use get with membership;

* wip(go): org scope license validation;

* wip(go): remove debug log;

* wip(go): remove scoping on list roles;

* wip(go): ignore invite creation if already exists;

* wip(go): add is msp func to check if deployment is msp deployment;

* wip(go): ensure username is email for msp users;

* wip(go): add email validated check on user table;

* wip(go): mark email validated when oauth user;

* wip(go): handle gorm error;

* wip(go): add tenant id to siem grpc schema;

* wip(go): read tenant from scoped ctx;

* wip(go): pass scoped ctx to siem grpc client;

* wip(go): remove unused publisher;

* wip(go): publish all tenant exporter updates on pull;

* wip(go): fix wg usage;

* wip(go): tenant scope the ctx when getting server info;

* NM-295: publish TCP uplink flags and peer tcp_proxy_endpoint

Add gateway tcp_proxy_enabled/listen port and client use_tcp_uplink,
and populate PeerIDs on host-level pulls so clients receive dial targets.

* wip(go): add tenant id to enrollment key token;

* wip(go): make host register tenant scope;

* wip(go): add default settings;

* wip(go): use correct checks for is msp tenant;

* wip(go): add email validation extension;

* wip(go): add pages for validation check;

* wip(go): use extension for validating user;

* wip(go): add invite type;

* wip(go): create only one validation invite per user email;

* wip(go): add validate email handler;

* wip(go): add email builder;

* wip(go): add user invite type migration logic;

* wip(go): only log email send failures;

* wip(go): add tenant id to flows table;

* wip(go): add func to check if connected to clickhouse;

* wip(go): migrate clickhouse;

* wip(go): add tenant id to mdm and edr state;

* wip(go): add delete methods for tables;

* wip(go): add tenant teardown method;

* wip(go): handle tenant deletion;

* wip(go): clean up flow logs on tenant deletion;

* wip(go): add update org membership role id method;

* wip(go): add transfer org ownership api;

* wip(go): move audit logs retention period to org settings;

* wip(go):get retention period from org settings;

* wip(go): remove old audit logs usage;

* wip(go): migrate audit logs retention period;

* wip(go): unmarshal decodes nums as float64;

* wip(go): run flow logs cleanup for all tenants in one go;

* wip(go): add audit logs retention period and smtp settings to tenant settings table for non-msp tenants;

* wip(go): resolve retention period and smtp settings correctly;

* wip(go): handle retention period and smtp settings updates and fetches correctly;

* wip(go): overlay retention period and smtp settings on server settings;;

* wip(go): upsert overlayed settings;

* NM-295: resolve pr comments

* wip(go): add azure tenant id to org settings;

* wip(go): tenant scope list roles;

* wip(go): add noop creds;

* wip(go): use first instead of find;

* NM-295: check network hook cleanup time

* wip(go): use correct roles when transfering ownership;

* wip(go): delete access tokens only if tenant scoped;

* wip(go): make delete user work for both org and tenant scope;

* wip(go): delete extclients only if tenant scoped;

* wip(go): check if caller is allowed to delete user;

* wip(go): infer scope for update user;

* wip(go): prevent org-owner elevation;

* wip(go): permissions check for update;

* wip(go): delete user access tokens only if tenant scope;

* wip(go): delete extclients only if tenant scope;

* wip(go): move upsert membership in upsert user;

* wip(go): set api host when constructing server config;

* NM-295: mv proxy settings to host

* NM-295: preserve tcp uplink setting on disconnect/connect

* NM-295: show tcpuplink option on exit node gw if enabled

* NM-295: fix orphan gateway relayed client nodes

---------

Co-authored-by: VishalDalwadi <[email protected]>
Co-authored-by: Vishal Dalwadi <[email protected]>
2026-08-18 16:17:30 +05:30
Vishal Dalwadi 132aee0bb3 NM-341-v6: AMB Multi-tenancy Management (#4101)
* wip(go): return auth token with user on create org owner;

* wip(go): prevent use of default tenant slug ;

* wip(go): upsert settings for tenant in ctx;

* wip(go): use passed ctx when updating user; remove password validation from ValidateUser (already validated in UpdateUser);

* wip(go): qualify all tenant id usages with table name;

* wip(go): use passed ctx

* wip(go): return old enrollment key type on create;

* wip(go): prevent null networks return;

* wip(go): wire ctx in dns/nameserver functions; wire ctx in host registration functions;

* wip(go): wire ctx in node functions;

* wip(go): wire ctx in node functions;

* wip(go): wire ctx in extclient functions;

* wip(go): remove not needed field;

* wip(go): use passed ctx;

* wip(go): use unique index instead of primary key;

* wip(go): wire ctx in extclient apis;

* wip(go): wire ctx in extclient apis;

* wip(go): remove org id check from global scoped get tenant api;

* wip(go): add scope slug to discover response;

* wip(go): wire ctx in get users;

* wip(go): set tenant id in posture check violations;

* wip(go): upsert membership instead of user;

* wip(go): pass correct len;

* wip(go): optimize (single loop);

* wip(go): unify todos;

* wip(go): add todo;

* wip(go): wire context for network get;

* wip(go): wire context;

* wip(go): wire context;

* wip(go): wire ctx when getting tenant superadmin;

* wip(go): add network creator to network admin group;

* wip(go): check for user membership on sync;

* wip(go): add methods with strict membership check;

* wip(go): remove unwanted feature flags;

* wip(go): report per-tenant usage during license validation;

* wip(go): move email setting from tenant to org;

* wip(go): pass org scope to email init;

* wip(go): remove old tenant email settings usages; use org settings to get email settings

* wip(go): use get email settings from org settings;

* wip(go): remove email settings migration;

* wip(go): migrate email settings from server to org settings;

* wip(go): assign global network admin group membership only for super-admin and admins;

* wip(go): assign global network admin group membership only for super-admin and admins;

* wip(go): add membership alias to filter;

* wip(go): skip clearing local license validation response cache;

* wip(go): extend license validation response for multi-tenancy;

* wip(go): add functions to get sole tenant and org;

* wip(go): use tenant-specific feature flags;

* wip(go): add tenant status type;

* wip(go): remove old hook comment;

* wip(go): expect ctx during license validation;

* wip(go): always create default org and tenant for ce;

* wip(go): add tenant-specific check for license validation error;

* wip(go): cache license validation response in memory;

* wip(go): add todo to handle multiple-tenant for integration pull;

* wip(go): pass ctx during license validation;

* wip(go): add todo for making changes later; run hook separately for each tenant;

* wip(go): return specific errors when license validation response not in cache;

* wip(go): use retryable.httpclient for calling license validation api;

* wip(go): process license validation;

* wip(go): license validation function;

* wip(go): migration on license validation;

* wip(go): org settings not expected;

* wip(go): remove expiry from license validation response;

* wip(go): add limits and limits enforcement to license validation response;

* wip(go): remove expiry usages;

* wip(go): add arg to clear cache;

* wip(go): add arg to clear cache;

* wip(go): skip license validation if cached resp;

* wip(go): allow org_id and tenant_id query params;

* wip(go): upsert tenant membership on sync;

* wip(go): pass ctx to delete network roles;

* wip(go): create tenant membership on org owner creation on ce;

* wip(go): add tenant orchestrator;

* wip(go): use tenant orchestrator;

* wip(go): add default settings for tenant;

* wip(go): make org owner superadmin on all tenants;

* wip(go): simplify SyncOrgAndTenants;

* wip(go): retry license validation only 3 times

* wip(go): store invalid err;

* wip(go): add function to check if limits are to be enforced on a tenant;

* wip(go): pass tenant scope if possible when checking for license validation error;

* wip(go): remove merge errors;

* wip(go): add functions for limit checks;

* wip(go): wire host limit check;

* wip(go): wire ingress limit check;

* wip(go): wire egress limit check;

* wip(go): wire client limit check;

* wip(go): wire network limit check;

* wip(go): wire user limit check;

* wip(go): use correct errors;

* wip(go): skip telemetry for msps;

* wip(go): disable telemetry on msp tenants;

* wip(go): err check;

* wip(go): uncomment log event;

* wip(go): get sso state to get scope for non-headless oauth login;

* wip(go): inline delete pending user;

* wip(go): allow both org and tenant scope for pending users table;

* wip(go): add migration for pending user table during set tenant id and rekey org/tenant id;

* wip(go): update queries to use new scope fields in pending users table;

* wip(go): allow both org and tenant scope for user invites table;

* wip(go): update queries to use new scope fields;

* wip(go): add migration for rekey-ing org/tenant id and setting tenant id;

* wip(go): inline delete user invite;

* wip(go): set scope when creating invite;

* wip(go): get user count with membership is ctx is tenant scoped;

* wip(go): add methods to upsert and delete org memberships;

* wip(go): add global valid tenant and org roles map;

* wip(go): add scope query in user invite get by email;

* wip(go): add support for upsert and delete of org and tenant membership both;

* wip(go): infer scope for user invite apis;

* wip(go): wire ctx in get user invite by email;

* wip(go): allow no scope headers to be passed in infer scope apis;

* wip(go): add validation and checks for org-scoped invite api handlers;

* wip(go): wire ctx to is pending user check;

* wip(go): infer scope for pending user apis;

* wip(go): set default role based on ctx scope;

* wip(go): move mfa to tenant membership table; move account status to membership tables;

* wip(go): migrate account status and mfa settings;

* wip(go): update account status on sync;

* wip(go): use update mfa to update mfa setting;

* wip(go): set new membership settings on user creation;

* wip(go): use membership where required;

* wip(go): wire ctx on key unlink refs;

* wip(go): use r.Context directly without reinitializing db;

* wip(go): wire ctx in acls;

* wip(go): add tenant scoped primary key constructor;

* wip(go): use tenant scoped primary key for acls, dns, extclients and tags;

* wip(go): wire ctx for get groups in network;

* wip(go): wire ctx for get groups;

* wip(go): scope non-uuid group ids by tenant id;

* wip(go): migrate non-uuid group ids to scoped ids;

* wip(go): don't expect scope id in invite validation and signup apis;

* wip(go): wire ctx for get matching user; allow org scope when querying by external idp id;

* wip(go): add mfa to org;

* wip(go): scope preauth token;

* wip(go): add fallback for older auth tokens;

* wip(go): wire ctx for preauth token creation;

* wip(go): migrate mfa settings;

* wip(go): resolve mfa settings for inherited auth;

* wip(go): update mfa for org;

* wip(go): infer scope for totp apis;

* wip(go): use ctx when passed;

* wip(go): infer scope for list user apis;

* wip(go): ensure set primary keys don't cause errors during fetch;

* wip(go): make primary key for integrations uuid and store provider in separate key;

* wip(go): migrate integration ids;

* wip(go): use provider instead of id everywhere;

* wip(go): remove gorm column;

* wip(go): wire ctx for igw egress creation;

* wip(go): wire ctx for igw egress update;

* wip(go): wire ctx for host registration;

* wip(go): wire ctx for host registration;

* wip(go): merge errors;

* wip(go): merge errors;

* wip(go): wire ctx; remove default scope usages;

* wip(go): use get with membership;

* wip(go): org scope license validation;

* wip(go): remove debug log;

* wip(go): remove scoping on list roles;

* wip(go): ignore invite creation if already exists;

* wip(go): add is msp func to check if deployment is msp deployment;

* wip(go): ensure username is email for msp users;

* wip(go): add email validated check on user table;

* wip(go): mark email validated when oauth user;

* wip(go): handle gorm error;

* wip(go): add tenant id to siem grpc schema;

* wip(go): read tenant from scoped ctx;

* wip(go): pass scoped ctx to siem grpc client;

* wip(go): remove unused publisher;

* wip(go): publish all tenant exporter updates on pull;

* wip(go): fix wg usage;

* wip(go): tenant scope the ctx when getting server info;

* wip(go): add tenant id to enrollment key token;

* wip(go): make host register tenant scope;

* wip(go): add default settings;

* wip(go): use correct checks for is msp tenant;

* wip(go): add email validation extension;

* wip(go): add pages for validation check;

* wip(go): use extension for validating user;

* wip(go): add invite type;

* wip(go): create only one validation invite per user email;

* wip(go): add validate email handler;

* wip(go): add email builder;

* wip(go): add user invite type migration logic;

* wip(go): only log email send failures;

* wip(go): add tenant id to flows table;

* wip(go): add func to check if connected to clickhouse;

* wip(go): migrate clickhouse;

* wip(go): add tenant id to mdm and edr state;

* wip(go): add delete methods for tables;

* wip(go): add tenant teardown method;

* wip(go): handle tenant deletion;

* wip(go): clean up flow logs on tenant deletion;

* wip(go): add update org membership role id method;

* wip(go): add transfer org ownership api;

* wip(go): move audit logs retention period to org settings;

* wip(go):get retention period from org settings;

* wip(go): remove old audit logs usage;

* wip(go): migrate audit logs retention period;

* wip(go): unmarshal decodes nums as float64;

* wip(go): run flow logs cleanup for all tenants in one go;

* wip(go): add audit logs retention period and smtp settings to tenant settings table for non-msp tenants;

* wip(go): resolve retention period and smtp settings correctly;

* wip(go): handle retention period and smtp settings updates and fetches correctly;

* wip(go): overlay retention period and smtp settings on server settings;;

* wip(go): upsert overlayed settings;

* wip(go): add azure tenant id to org settings;

* wip(go): tenant scope list roles;

* wip(go): add noop creds;

* wip(go): use first instead of find;

* wip(go): use correct roles when transfering ownership;

* wip(go): delete access tokens only if tenant scoped;

* wip(go): make delete user work for both org and tenant scope;

* wip(go): delete extclients only if tenant scoped;

* wip(go): check if caller is allowed to delete user;

* wip(go): infer scope for update user;

* wip(go): prevent org-owner elevation;

* wip(go): permissions check for update;

* wip(go): delete user access tokens only if tenant scope;

* wip(go): delete extclients only if tenant scope;

* wip(go): move upsert membership in upsert user;

* wip(go): set api host when constructing server config;

* wip(go): fix static errors;

* wip(go): fix test cases;

* wip(go): use correct get ns order;

* wip(go): use correct get pending host order;

* wip(go): use correct get access token order;

* wip(go): use correct get event order;

* wip(go): return error response on emqx user creation error;

* wip(go): store correct value in cache;

* wip(go): return better error response;

* wip(go): return better error response;

* wip(go): use cache when listing egress acls;

* wip(go): wire ctx in UserHasNetworkGroupAccess;

* wip(go): remove double scope wrapping;

* wip(go): add security check for list and get org;

* wip(go): ratelimit discover api;

* wip(go): expect sole org id in get and create org owner;

* wip(go): wrap create org owner in tx;
2026-08-05 21:25:02 +08:00
abhishek9686 6d841de0f6 NM-366: Move inet gws functionality into egress 2026-07-08 15:35:51 +05:30
Vishal DalwadiandAbhishek Kondur f9bbbd7efa NM-341: Schema Migration for Serverconf, Generated and Enrollment Keys Table (#4065)
* feat(go): remove enrollment key cache;

* feat(go): add internal schema;

* feat(go): use the new internal table to fetch serverconf records;

* feat(go): move internal key constants to schema/internal.go;

* feat(go): add server conf table migration;

* feat(go): use the new internal table for generated records;

* feat(go): add generated table migration;

* feat(go): use the new internal table for serveruuid records;

* feat(go): add server uuid table migration;

* chore(go): remove empty file;

* feat(go): store license validation cached response in the new internal table;

* chore(go): move table name constants;

* feat(go): use migration job name v1.7.0;

* wip(go): add schema definition for enrollment keys;

* wip(go): use schema.EnrollmentKey in orchestrator pkg;

* wip(go): use schema.EnrollmentKey in host registration and pending host approval;

* wip(go): use schema.EnrollmentKey in gateway/node cleanup;

* wip(go): use schema.EnrollmentKey in network cleanup;

* wip(go): use schema.EnrollmentKey in creating default network enrollment key;

* wip(go): use schema.EnrollmentKey in enrollment key logic/controller;

* wip(go): add schema migration for enrollment keys;

* fix(go): remove ref to non-existent constant;

* fix(go): use correct package+type for orchestrator test;

* feat(go): add internal table to list models function;

* fix(go): use int for enrollment key type;

* fix(go): remove tags from networks;

* fix(go): ensure same response for get enrollment keys as before;

* fix(go): enrollment key migration;

* wip(go): remove sqlite, rqlite, postgres and utils from database package;

* wip(go): remove unused database pkg method DeleteAllRecords;

* wip(go): remove db connection management code and accompanying consts from database/ pkg;

* wip(go): add schema definitions for org, tenant and org settings tables;

* wip(go): add tenant id to add tables in schema/ pkg; add tenant-level unique constraints for network, user invite and pending users;

* wip(go): remove usages of host actions table;

* wip(go): create default org and tenant on migration;

* wip(go): manage kv tables with gorm;

* wip(go): bring network id out of value for acls, dns, extclients, metrics and tags tables;

* wip(go): store user settings in user table;

* wip(go): add server settings migration logic;

* wip(go): replace database/ pkg functions with methods from the schema/ pkg;

* wip(go): add schema definitions for tenant settings; replace database/ pkg functions with methods from the schema/ pkg;

* wip(go): add index on tenant id;

* wip(go): remove tenant id column from user role table;

* wip(go): set tenant_id to default tenant's id on all records in all tables on migration;

* wip(go): extract out network_id on acls, dns, extclients, metrics and tags tables on migration;

* wip(go): add scope pkg with middleware to extract scope from headers or set default scope;

* wip(go): scope (most) apis to tenant;

* wip(go): remove users and user roles table from set tenant id migration function;

* wip(go): handle test case errors;

* wip(go): cache default tenant and org ids;

* feat(go): add utils to get server settings during migration;

* feat(go): check nil before conversion;

* feat(go): use correct arg order;

* wip(go): pass db context;

* wip(go): correct api handler router tag;

* wip(go): add schema definitions for org and tenant membership tables;

* wip(go): use middleware from middleware/ package;

* wip(go): add org role constants;

* wip(go): read user tenant role and groups from tenant membership table;

* wip(go): add migration logic for memberships;

* wip(go): undo inadvertent paste;

* wip(go): add migration logic for migrating user's role and groups to tenant membership table and setting org owner;

* wip(go): use tenant membership table to set user role and groups;

* wip(go): test cases;

* feat(go): check for nil;

* feat(go): move auth type, password and external idp id to tenant membership table;

* wip(go): use middleware from middleware/ package;

* wip(go): add schema definitions for org and tenant membership tables;

* wip(go): add temp default function to use default tenant scoped contexts everywhere;

* wip(go): use scope.Default to set tenant id before db ops;

* wip(go): create default org and tenant for tests as well;

* wip(go): move Default to logic to remove import cycle;

* wip(go): scope auth request to default context;

* wip(go): migration fixes;

* wip(go): use correct column; migrate membership on user migrate;

* fix(go): add smtp tls verify skip setting;

* fix(go): add smtp tls verify skip setting;

---------

Co-authored-by: Abhishek Kondur <[email protected]>
2026-07-06 23:00:21 +05:30
VishalDalwadi ac71c777bf fix(go): prevent posture checks field update from nmui; 2026-06-23 23:12:19 +05:30
Vishal Dalwadi 2a65132ae5 Fixes/v1.5.2 (#4043)
* feat(go): allow setting enrollment key as default on update;

* feat(go): clear default enrollment keys on update;

* feat(go): remove datadog client dependency;

* feat(go): remove ipaddr dependency;

* feat(go): remove okta sdk dependency;

* chore(go): make schema filename plural;

* fix(go): prevent resetting node expiration time on update;

* fix(go): set expiration datetime only if non-zero;

* fix(go): upsert violations only on violation check;

* fix(go): copy status on conversion;

* fix(go): check posture check violations on node create;

* feat(go): separate out last evaluation cycle id and last evaluated at;

* feat(go): add reserved set to store ips that are pending commit; avoid double allocation;

* fix(go): copy auto relayed peers on conversion;

* fix(go): remove posture checks on delete ingress gateway and update node;

* fix(go): use column update functions to update status and violations;

* fix(go): add type to dns entry model;

* fix(go): drop groups on role change;

* fix(go): drop groups on role upgrade to admin/superadmin;

* fix(go): run posture checks on host update;

* fix(go): read from correct variable;

* fix(go): remove groups membership changes on role change;
2026-06-05 13:07:04 +05:30
fdcf11e0db NM-293: Nodes Schema Migration (#3982)
* feat(go): add schema for pending users table;

1. Schema Definition for Pending Users table.
2. Use the newer table everywhere.
3. Migration stubs for v1.5.2.
4. Migration code for Pending Users table;

* feat(go): add schema for user invites table;

1. Schema Definition for User Invites table.
2. Use the newer table everywhere.
3. Migration code for User Invites table;

* feat(go): add schema definitions for nodes, gateways and posture check violations table;

* feat(go): remove legacy node usages;

* fix(go): address code review comments;

1. remove column names in primary key for PostureCheckViolation struct.
2. use correct column name for network_id.
3. handle rollback error.

* feat(go): check ip uniqueness using db query;

* feat(go): remove nodes and network nodes cache; remove unused functions and consts;

* feat(go): delete unused function GetExtClientHAConf;

* feat(go): optimize DoesHostExistInTheNetworkAlready;

* feat(go): remove legacy node and migration;

* feat(go): implement network orchestrator;

1. Implements IP Allocation for Nodes and Extclients;

* feat(go): implement gateway orchestrator and extensions;

1. Implements CreateGateway;
2. Pro and CE Gateway Extensions to ConfigureAutoRelay;

* feat(go): implement node orchestrator and extensions;

1. Implements CreateNode;
2. Pro and CE Node Extensions to ConfigureAutoAssignGateway;

* feat(go): implement orchestrator repository and extensions factory;

* feat(go): add repository initialization with correct factory;

* feat(go): remove persistent keep alive and mtu from gateway;

* feat(go): add auto relayed peers to node;

* feat(go): rename relay gateway to relaying node; use null string instead of string pointer;

* fix(go): merge errors;

* feat(go): add update node last checkin function;

* feat(go): wire orchestrator create node;

1. Enrollment Key Join.
2. API: Add Host to Network.
3. API: Approve Pending Host.
4. API: Update Host (Make Host Default).
5. API: Create Network (Add Default Hosts to Network).

* feat(go): inline create extclient;

Remove cyclic dependency between logic and orchestrator;

* feat(go): remove old create node functions;

* feat(go): optimize bulk update node connected status;

* feat(go): mark pending delete directly; move constants;

* feat(go): remove unused validations;

* feat(go): remove allocated ip map cache;

* feat(go): use host persistent keep alive and mtu for extclient config;

* feat(go): remove (almost) all failover "unused" code;

* feat(go): guard address allocation with mtx;

* feat(go): validate node ip uniqueness with network orchestrator;

* feat(go): convert schema node to models node for host update;

* feat(go): remove nodes cache initialization;

* feat(go): use mark for deletion for node soft delete and delete for hard delete; add preload options for node get;

* feat(go): removed unused code;

1. No calls for GetRelays and commented GetRelayedNodes.
2. GetNodeEgressInfo populates egress details for node but they aren't validated here.

* feat(go): convert duration to seconds;

* feat(go): remove gateway schema, extensions and orchestrator; move gateway management to nodes extension and orchestrator;

* feat(go): remove self-referential foreign key;

* feat(go): add converters for schema node; use net tables for getting a node and listing all nodes;

* feat(go): remove unused code;

* feat(go): use models.Node for delete channel;

* feat(go): optimize node telemetry gen;

* feat(go): pass schema.Node to check zombies;

* feat(go): add model.Node to schema.Node conversion;

* feat(go): upsert violations on node upsert;

* feat(go): remove all SetDNS usage;

* feat(go): use fetchAll instead of database.FetchRecords;

* feat(go): add nodes migration;

* feat(go): remove unused node methods;

* refactor(go): improve field names;

* feat(go): remove unused network methods;

* feat(go): move type NodeStatus and status constants to scheme package;

* feat(go): add list network nodes paginated api;

* feat(go): add filters to list nodes paginated api;

* feat(go): improve logs; manage tx outside of schema pkg;

* feat(go): ensure address empty for nil ips;

* feat(go): add json tags;

* feat(go): add json tags;

* feat(go): use string ptr instead of NullString;

* feat(go): use api host in list response;

* feat(go): optimize list posture check violations api;

* feat(go): list only current violations;

* feat(go): scope table field on join;

* NM-293: fix static checks

* NM-293: fix static checks

* NM-293: fix static checks

* Update orchestrator/network.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-293: fix invalid gorm struct tags

* NM-293: Only set RelayingNodeID when the node is actually relayed.

* NM-293: use RelayedByNodeID for clarity

* feat(go): add pkg for json expressions; add node methods to do json operations in db;

* feat(go): make options common for all orchestrators; use orchestrator create gateway in api;

* feat(go): add delete all methods; add set internet gateway in db;

* feat(go): set is ingress gateway to is gateway;

* feat(go): restore igws handlers;

* feat(go): use network name in tag;

* feat(go): add skip mq host and node update options;

* feat(go): add node orchestrator test suite;

* feat(go): set network addr range on conversion;

* feat(go): upsert node instead of update node;

* feat(go): remove old todos;

* feat(go): use only is_gw to determine whether gateway;

* feat(go): add create node with default host test case;

* feat(go): add create node with enrollment key test case;

* feat(go): add pro node orchestrator test suite;

* feat(go): update node and client status on update metrics;

* feat(go): add node status filters;

* feat(go): update node status on bulk update as well;

* feat(go): prevent reset of host password on upsert;

* feat(go): disable platform/service user device join;

* feat(go): disable platform/service user device join;

* feat(go): add search across fields for list network nodes api;

* feat(go): add field for additional gateway endpoints;

* feat(go): configure tags only if they belong to the node network;

* feat(go): optimize tag assignments and unassignments;

* feat(go): add posture check violations check before create node;

Check not added for default host on network create;

* feat(go): copy key before editing key networks to prevent changes in the cache;

* feat(go): construct another network slice instead of in-place delete;

* feat(go): error only when no networks left to join;

* feat(go): move set defaults to schema migration;

* feat(go): load server settings in cache before migration;

* feat(go): migrate gateway nameserver and egress along with nodes;

* feat(go): disable netclient auto update if disabled globally;

* refactor(go): move network nameserver migration to separate function;

* feat(go): calculate node status on migrate (skipping metrics check);

Previously, we weren't setting the node status in db, just recalculating on the fly. With this version, we expect the node status to be in the db and recalculated when metrics are pushed from netclient. So we have to recalculate the node status to initialize the status value.

* feat(go): update only if nodeIDs is non-empty;

WithFilter adds the where clause only if atleast one value is passed. To prevent updating status of all nodes, add a check for nodeIDs.

* fix(go): remove tag;

Execute assign tag only when at least one node has to be tagged;

* feat(go): add egress nat mode migration;

* fix(go): use correct network id;

* fix(go): make is_auto_relay string;

* fix(go): create tag before use to pass test;

* fix(go): move delete to testutils package;

---------

Co-authored-by: abhishek9686 <[email protected]>
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-05-15 20:35:53 +05:30
Abhishek Kondurandtenki-reviewer[bot] c3c3ed1fb8 NM-254: add bulk delete apis, rm old acl code (#3937)
* NM-254: add bulk delete apis for users, hosts, nodes and optimise postgres connection settings

* NM-254: rm debug logs

* NM-254: add bulk delete apis, remove old acl code

* NM-254: rm unused flag

* NM-254: fix bulk delete bugs, add security and performance improvements

- Fix host delete notifying peers before confirming deletion from DB
- Fix self-delete vulnerability in bulk user delete
- Fix DissasociateNodeFromHost failing when host.Nodes is empty
- Fix AssociateNodeToHost/DissasociateNodeFromHost stale read race
- Hoist GetAllExtClients outside loop in bulk user delete/status
- Move initializeUUID outside master-pod guard for HA correctness

* NM-254: return 202 Accepted for async bulk APIs, fix relay allowedIPs and host association error handling

- Change all bulk endpoints (hosts, nodes, users, ext clients) from
  200 OK to 202 Accepted to correctly signal async processing
- Add ReturnAcceptedResponse helper in logic/errors.go
- Fix GetAllowedIpsForRelayed returning empty allowedIPs slice,
  restoring relay connectivity
- Make AssociateNodeToHost and DissasociateNodeFromHost return an
  error when the host DB re-fetch fails instead of silently using
  stale data
- Add bulk-apis.md documenting all five bulk endpoints

* NM-254: rm coredns container

* NM-254: add bulk apis for node,extclient status, add activity logs to bulk apis

* NM-254: add bulk api for connection toggle

* NM-254: add network check

* Update controllers/hosts.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-254: optimise bulk extclient deletion

---------

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-03-26 10:15:07 +05:30
edda2868fc NM-163: Users, Groups, Roles, Networks and Hosts Table Migration (#3910)
* feat(go): add user schema;

* feat(go): migrate to user schema;

* feat(go): add audit fields;

* feat(go): remove unused fields from the network model;

* feat(go): add network schema;

* feat(go): migrate to network schema;

* refactor(go): add comment to clarify migration logic;

* fix(go): test failures;

* fix(go): test failures;

* feat(go): change membership table to store memberships at all scopes;

* feat(go): add schema for access grants;

* feat(go): remove nameservers from new networks table; ensure db passed for schema functions;

* feat(go): set max conns for sqlite to 1;

* fix(go): issues updating user account status;

* refactor(go): remove converters and access grants;

* refactor(go): add json tags in schema models;

* refactor(go): rename file to migrate_v1_6_0.go;

* refactor(go): add user groups and user roles tables; use schema tables;

* refactor(go): inline get and list from schema package;

* refactor(go): inline get network and list users from schema package;

* fix(go): staticcheck issues;

* fix(go): remove test not in use; fix test case;

* fix(go): validate network;

* fix(go): resolve static checks;

* fix(go): new models errors;

* fix(go): test errors;

* fix(go): handle no records;

* fix(go): add validations for user object;

* fix(go): set correct extclient status;

* fix(go): test error;

* feat(go): make schema the base package;

* feat(go): add host schema;

* feat(go): use schema host everywhere;

* feat(go): inline get host, list hosts and delete host;

* feat(go): use non-ptr value;

* feat(go): use save to upsert all fields;

* feat(go): use save to upsert all fields;

* feat(go): save turn endpoint as string;

* feat(go): check for gorm error record not found;

* fix(go): test failures;

* fix(go): update all network fields;

* fix(go): update all network fields;

* feat(go): add paginated list networks api;

* feat(go): add paginated list users api;

* feat(go): add paginated list hosts api;

* feat(go): add pagination to list groups api;

* fix(go): comment;

* fix(go): implement marshal and unmarshal text for custom types;

* fix(go): implement marshal and unmarshal json for custom types;

* fix(go): just use the old model for unmarshalling;

* fix(go): implement marshal and unmarshal json for custom types;

* feat(go): remove paginated list networks api;

* feat(go): use custom paginated response object;

* fix(go): ensure default values for page and per_page are used when not passed;

* fix(go): rename v1.6.0 to v1.5.1;

* fix(go): check for gorm.ErrRecordNotFound instead of database.IsEmptyRecord;

* fix(go): use host id, not pending host id;

* feat(go): add filters to paginated apis;

* feat(go): add filters to paginated apis;

* feat(go): remove check for max username length;

* feat(go): add filters to count as well;

* feat(go): use library to check email address validity;

* feat(go): ignore pagination if params not passed;

* fix(go): pagination issues;

* fix(go): check exists before using;

* fix(go): remove debug log;

* fix(go): use gorm err record not found;

* fix(go): use gorm err record not found;

* fix(go): use user principal name when creating pending user;

* fix(go): use schema package for consts;

* fix(go): prevent disabling superadmin user;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): swap is admin and is superadmin;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): remove dead code block;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837937

* fix(go): incorrect message when trying to disable self;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2928837934

* fix(go): use correct header;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): return after error response;

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): use correct order of params;

https://github.com/gravitl/netmaker/pull/3910#discussion_r2929593036

* fix(go): set default values for page and page size; use v2 instead of /list;

* Update logic/auth.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* Update schema/user_roles.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): syntax error;

* fix(go): set default values when page and per_page are not passed or 0;

* fix(go): use uuid.parse instead of uuid.must parse;

* fix(go): review errors;

* fix(go): review errors;

* Update controllers/user.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* Update controllers/user.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-163: fix errors:

* Update db/types/options.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* fix(go): persist return user in event;

* Update db/types/options.go

Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>

* NM-163: duplicate lines of code

* NM-163: fix(go): fix missing return and filter parsing in user controller

- Add missing return after error response in updateUserAccountStatus
  to prevent double-response and spurious ext-client side-effects
- Use switch statements in listUsers to skip unrecognized
  account_status and mfa_status filter values

* fix(go): check for both min and max page size;

* fix(go): enclose transfer superadmin in transaction;

* fix(go): review errors;

* fix(go): remove free tier checks;

* fix(go): review fixes;

---------

Co-authored-by: VishalDalwadi <[email protected]>
Co-authored-by: Vishal Dalwadi <[email protected]>
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
2026-03-17 19:36:52 +05:30
Abhishek Kondur eed32cd2d6 Merge pull request #3735 from gravitl/NM-166
NM-166: Device Posture Checks
2025-12-05 10:33:11 +04:00
Abhishek K c643a50b67 NM-159: simplify auto assign gw logic (#3719)
* force update host dns field if node is acting as inet gw

* add old acl support checks

* move auto relay migration to pro pkg

* add check to avoid unsetting relayed node

* simplify auto assign gw logic

* send auto assign update on un relay

* set checking time to latest on updates

* fix HA auto Relay logic

* add relay node metrics to peer signal

* move auto relay peer check

* publish host peer update

* check and unset unrelayed auto peers

* use relay node mutex to avoid rac condition

* reset autorelayed peers on auto assign gw
2025-11-06 14:54:09 +04:00
Abhishek K 74fef9fbc6 NM-122: Auto Relay, auto assignment of Gw (#3697)
* add auto realy handlers and logic funcs

* add pro func connectors

* Add auto relayed peer ips on peer update, set auto relay on gw creation

* add network id to signal, add autorelay nodes to peerudpate

* add autorelay peer update logic

* add nodes to peer update

* revert node model change

* reset auto relayed peers on the relay node on reset, add auto relay nodes to pull

* add logic api to update auto relay node

* add autoassigngw field to node, add logic to swith relay node in relayme udpate api

* add gw nodes to pull

* intilaise gw map

* HA relay functionality

* add autoassign gw option to enrollment key

* publish intant action to auto assign gw

* fix static checks

* unset relay if auto assign removed

* add host node model to auto relay info

* add host node model to auto relay info

* only use hostNode model for gws info

* handle autoassigned gw peer in the update

* handle autoassigned gw peer in the update

* handle peer updates for autoassigned gw peer

* unset auto assigned peer if relayed or failedovered
2025-10-28 09:53:31 +04:00
Abhishek K 49e28e3385 NM-137: Add addtional mq actions to host api (#3671)
* add host node update action

* add peer signal action to fallback api

* add replace peers to host pull

* add delete host action to fallback api

* update base go builder image

* update go builder tag

* check host port to avoid conflicts behind NAT

* fix connect/disconnect on api

* send pull signal on disconnect from UI

* fix panic on host join via user auth

* reset failover on disconnect
2025-10-07 13:16:31 +04:00
Abhishek K 0f884d4f36 NET-2061: revert Inet gws, fix extclient comms with user policies (#3482)
* revert inet gws from acl policies

* add egress range with metric for inet gw

* link pro inet funcs

* fix extclient comms with users

* remove TODO comments
2025-06-05 22:42:16 +05:30
Abhishek KandVishal Dalwadi 5849e0e2e1 NET-1932: sort out acls into CE and PRO (#3460)
* move relevant acl and tag code to CE and Pro pkgs

* intialise pro acl funcs

* list gateways by user access

* check user gw access by policies

* filter out user policies on CE

* filter out tagged policies on CE

* fix ce acl comms

* allow gateways tag

* allow gateway tag  on CE, remove failover and gw check on acl policy

* add gw rules func to pro

* add inet gw support on CE

* add egress acl API

* add egress acl API

* fix(go): set is_gw when converting api node to server node;

* fix(go): set is_gw when converting api node to server node;

* fix policy validity checker for inet gws

* move dns option to host model

* fix node removal from egress policy on delete

* add migration logic for ManageDNS

* fix dns json field

* fix nil error on node tags

* add egress info to relayed nodes

---------

Co-authored-by: Vishal Dalwadi <[email protected]>
2025-06-02 13:05:10 +05:30
Abhishek KandVishal Dalwadi 307a3d1e4b NET-1932: Merge egress and internet gateways (#3436)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* egress model

* fix revoked tokens to be unauthorized

* update egress model

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* setup api handlers for egress revamp

* use single DB, fix update nat boolean field

* extend validaiton checks for egress ranges

* add migration to convert to new egress model

* fix panic interface conversion

* publish peer update on settings update

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* remove nat check on egress gateway request

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* add inet gw status to egress model

* fetch node ids in the tag, add inet gw info clients

* add inet gw info to node from egress list

* add migration logic internet gws

* create default acl policies

* add egress info

* add egress TODO

* add egress TODO

* fix user auth api:

* add reference id to acl policy

* add egress response from DB

* publish peer update on egress changes

* re initalise oauth and email config

* set verbosity

* normalise cidr on egress req

* add egress id to acl group

* change acls to use egress id

* resolve merge conflicts

* fix egress reference errors

* move egress model to schema

* add api context to DB

* sync auto update settings with hosts

* sync auto update settings with hosts

* check acl for egress node

* check for egress policy in the acl dst groups

* fix acl rules for egress policies with new models

* add status to egress model

* fix inet node func

* mask secret and convert jwt duration to minutes

* enable egress policies on creation

* convert jwt duration to minutes

* add relevant ranges to inet egress

* skip non active egress routes

* resolve merge conflicts

* fix static check

* update gorm tag for primary key on egress model

* create user policies for egress resources

* resolve merge conflicts

* get egress info on failover apis, add egress src validation for inet gws

* add additional validation checks on egress req

* add additional validation checks on egress req

* skip all resources for inet policy

* delete associated egress acl policies

* fix failover of inetclient

* avoid setting inet client asd inet gw

* fix all resource egress policy

* fix inet gw egress rule

* check for node egress on relay req

* fix egress acl rules comms

* add new field for egress info on node

* check acl policy in failover ctx

* avoid default host to be set as inet client

* fix relayed egress node

* add valid error messaging for egress validate func

* return if inet default host

* jump port detection to 51821

* check host ports on pull

* check user access gws via acls

* add validation check for default host and failover for inet clients

* add error messaging for acl policy check

* fix inet gw status

* ignore failover req for peer using inet gw

* check for allowed egress ranges for a peer

* add egress routes to static nodes by access

* avoid setting failvoer as inet client

* fix egress error messaging

* fix extclients egress comms

* fix inet gw acting as inet client

* return formatted error on update acl validation

* add default route for static nodes on inetclient

* check relay node acting as inetclient

* move inet node info to separate field, fix all resouces policy

* remove debug logs

---------

Co-authored-by: Vishal Dalwadi <[email protected]>
2025-05-21 12:50:21 +05:30
abhishek9686 225bf37e02 resolve merge conflicts 2025-02-28 12:18:38 +04:00
abhishek9686 dd8cd5856e change mutex to write lock 2025-02-24 13:38:34 +03:00
abhishek9686 92698363cd add node mutex to model 2025-02-19 22:17:40 +04:00
Abhishek K 8297642b90 NET-1914: add gw apis, move relays into CE (#3309)
* add gw apis, move relays into CE

* set gw field on relay and ingress creation

* add gw handlers to relay and ingress apis

* if node is inetgw and gw add dns

* remove pro check on relays

* fetch node before updating
2025-01-28 11:28:31 +05:30
Abhishek K c02ec193db NET-1920: Add disconnected node status (#3300)
* create peer ack table

* add restricted status

* add disconnected status
2025-01-28 11:26:12 +05:30
Yabin Ma 1d9e0f71df add format int64 for swagger doc (#3247)
* add format int64 for swagger doc

* add format int64 for swagger doc

* add global override

* swagger doc update

* add new line for .swaggo file
2024-12-12 09:52:52 +04:00
Abhishek K f124b10c35 NET-1784: Adv Acl Rules (#3239)
* define direction on acl req

* define protocol types and rule model

* get rules for node

* fetch acl rule for a node

* redine acl firewall model

* add json tags

* update port,protocol, and direction

* add json tags to acl options

* convert protocol to string

* simplify acl map

* add json tags to acl rules

* add networks to fw update

* add acls rules

* NET-1784: add allow all field

* add allow all field on fw udpate

* remove debug logs

* fix port and protocol types

* migrate default acl policies

* define constants for service types

* add adv options for user rules on ingress gw

* debug log

* allow whole network

* add static nodes to acl rules

* replace peers on acl updates

* initiliase rule map

* add user acl rules on target node

* revert acl check on extclient

* handle static node rules on ingress gw

* update multiple policies for users

* check allowed direction

* remove debug logs
2024-12-10 11:21:14 +04:00
Abhishek Kandthe_aceix 31c2311bef NET-1782: Fetch Node Connection Status from metrics (#3237)
* add live status of node

* handle static node status

* add public IP field to server configuration

* get public Ip from config

* improve node status logic

* improvise status check

* use only checkin status on old nodes

---------

Co-authored-by: the_aceix <[email protected]>
2024-12-10 10:46:05 +04:00
abhishek9686 6c1208ad81 resolve merge conflicts 2024-10-17 15:15:54 +04:00
abhishek9686 b41353b415 add static nodes to api resp 2024-10-14 16:36:25 +04:00
abhishek9686 04b8737a02 move tags to node model 2024-09-22 13:37:58 +04:00
Max Ma 189ac27527 NET-1509:add ingresspersistentkeepalive and ingressmtu for extClient/RAC config (#3107)
* add ingresspersistentkeepalive and ingressmtu for extClient/RAC config

* add ingressmtu and PKA in api response

* add pka and mtu in api/nodes PUT call

* add default value for PKA and mtu for extClients
2024-09-12 12:19:27 +04:00
Sayan Mallick c551c487ca New Docs (#3034)
* New Docs

CSS update and Dockerfile to include docs folder

flash of unrendered text fix

markdown docs

ignore docs/docs.go

improving the docs generation

github actions for docs generation

go runner version fix

updated docs.yml

update repo action updated

updated actions and dns docs

dns complete

More docs update

Complete docs and updated workflow

Update documentation Tue Aug  6 11:17:42 UTC 2024

Update documentation Thu Aug  8 12:26:57 UTC 2024

clean up

clean up

Dockerfile clean up

Updated workflow

Updated workflow

Update docs.yml

Update docs.yml

* requested changes

* changed ingress gateway to remote access gateway
2024-08-15 11:55:01 +05:30
Aceix abf3f4f55d feat(NET-1106): support additional RAG endpoint IPs (#2907) 2024-05-08 14:45:05 +05:30
Max Ma e0bde5b67f Net 1088 (#2872)
* fix ipv6 only issue

* fix InetGw ipv6 issue

* refactor constants for NET1088

* fix extClient IPv6 empty issue
2024-04-09 11:50:07 +05:30
Abhishek K 0638dcac49 NET-940: Inet Gws (#2828)
* internet gws apis

* add validate check for inet request

* add default gw changes to peer update

* update json tag

* add OS checks for inet gws

* add set defaul gw pro func

* allow disable and enable inet gw

* add inet handlers to pro

* add fields to api node

* add inet allowed ips

* add default gw to pull

* unset node inet details on deletion

* unset internet gw on network nodes

* unset inet gw fix

* unset inet gw fix

* send default gw ip

* fix inet node endpoint

* add default gw endpoint ip to pull resp

* validate after unset gws

* add inet client peer allowedips to inet node

* validate after unset gws

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix inet gw and relayed conflict

* fix inet gw and relayed conflict

* fix update req

* fix update inet gw api

* when inet gw is peer ignore other allowedIps

* test relay

* revert test relay

* revert inet peer update changes

* channel internet traffic of relayed node to relay's inetgw

* channel internet traffic of relayed node to relay's inetgw

* channel internet traffic of relayed node to relay's inetgw

* add check for relayed node

* add inet info to peer update

* add inet info to peer update

* fix update node to persist inet info

* fix go tests

* egress ranges with inet gw fix

* egress ranges with inet gw fix

* disallow node acting using inet gw to act as inet gw

* add check to validate inet gw

* fix typos

* add firewall check

* set inetgw on ingress req on community

* set inetgw to false on community on ingress del
2024-02-28 08:46:51 +07:00
the_aceix 5882b86d40 feat(NET-709): add node metadata for remote gws 2024-01-26 10:06:36 +00:00
Abhishek K 718f739b7d NET-891: Send deleted node peer update when expired (#2763)
* send deleted node peer update when expired

* set default expiration to 100 years

* pruge expired nodes

* send mq node delete update before deleting node

* initalise to new var

* revert expiration time interval
2024-01-12 16:01:40 +05:30
Abhishek Kandthe_aceix 530dbdc65c NET-710: Internet Gws Re-Design (#2718)
* add internet gateway to client gateway

* migration func to remove internet egress range from egress gateway

* add internet gateways ranges to firewall update

* add internet gw ranges to extcleint conf

* add ipv6 internet address

* remove failover field from ingress req

* only let normal to be created on PRO (#2716)

* feat(NET-805): send internet gw props to rac

* set inet gw field on node update api

* move internet gws to EE

---------

Co-authored-by: the_aceix <[email protected]>
2023-12-06 23:57:58 +04:00
Abhishek K b78cc0a8a1 NET-725: Failovers (#2685)
* api to  to get host relayed from client

* add auto relay to api host

* add peer nat type

* set pro field on signal

* rm net check on relay me handler

* return success response

* re-establish failover logic

* set failOver ctx

* failOver with peer pub key

* failovered peer updates

* failover handlers, reset failovered peer on deletion

* rm unused funcs

* initialize failover handler on EE

* ignore failover node on signal

* failover changes

* set host id on signal

* extend signal model to include node ids

* add backwards compatibility

* add failover as node api

* set json response on failover handers

* add failover field to api node

* fix signal data check

* initialize failover peer map

* reset failovered status when relayed or deleted

* add failover info to api node

* reset network failover

* only proceed furtuer if failover exists in the network

* set failOver node defaults

* cannot set failover node as relayed

* debug log

* debug log

* debug changes

* debug changes

* debug changes

* revert debug changes

* don't add peers to idmap when removed

* reset failed Over

* fix static checks

* rm debug log

* add check for linux host
2023-11-29 20:10:07 +04:00
Abhishek Kondur cf0b4cbd2b fix extclient route conv 2023-11-27 11:19:46 +04:00
Abhishek Kondur b6b71c951e add extclient extra allowedIPs and attach them to egress routes 2023-11-24 18:32:43 +04:00
Gabriel de Souza Seibel cb4b99ffcb [NET-562] Persistent Keep Alive from node to host (#2604)
* Move PKA field from models node to host level

* Move PKA field from api models node to host level

* Adapt logic package to node->host PKA

* Adapt migration-related code to node->host PKA

* Adapt cli code to node->host PKA

* Change host PKA default to 20s

* On IfaceDelta, check for PKA on host

* On handleHostRegister, set default PKA

* Use a default PKA

* Use int64 for api host pka

* Reorder imports

* Don't use host pka in iface delta

* Fix ConvertAPIHostToNMHost

* Add swagger doc for host PKA field

* Fix swagger.yml

* Set default PKA only for new hosts

* Remove TODO comment

* Remove redundant check

* Have api-host pka be specified in seconds
2023-10-06 10:09:19 +04:00
Gabriel de Souza SeibelandAbhishek Kondur 1a1ba1ccf4 [NET-546] Move ee code to ee package, unify ee status and terminology (#2538)
* Move ee code to ee package and unify ee status to IsPro

* Consolidate naming for paid/professional/enterprise version as "pro". Notes:

- Changes image tags
- Changes build tags
- Changes package names
- Doesn't change links to docs that mention "ee"
- Doesn't change parameters sent to PostHog that mention "ee"

* Revert docker image tag being -pro, back to -ee

* Revert go build tag being pro, back to ee

* Add build tags for some ee content

* [2] Revert go build tag being pro, back to ee

* Fix test workflow

* Add a json tag to be backwards compatible with frontend "IsEE" check

* Add a json tag for the serverconfig struct for IsEE

* Ammend json tag to Is_EE

* fix ee tags

---------

Co-authored-by: Abhishek Kondur <[email protected]>
2023-09-01 07:42:05 +05:30
5c38b5b2d0 migration (#2509)
* create gateways during migration

* set version for testing

* restruct migration

* debug logging

* enforce unique names for ext client names (#2476)

* enforce unique names for ext client names

* only check for unique id on creation

* check for unique id if changed

* prune(NET-483): remove defunct host.internetgateway field (#2487)

* don't reference host on err (#2493)

* deprecrate netclient install scripts (#2490)

* Net 500: validate network parameter passed to node endpoints (#2480)

* enforce unique names for ext client names

* only check for unique id on creation

* check for unique id if changed

* validate network parameter passed to node endpoints

---------

Co-authored-by: Abhishek K <[email protected]>

* NET-513 (#2492)

nm-certs.sh now requests certificate for EE and CE edition domains accordingly.

* [NET-404] Run in limited mode when ee checks fail (#2474)

* Add limited http handlers functionality to rest handler

* Export ee.errValidation (ee.ErrValidation)

* Export a fatal error handled by the hook manager

* Export a new status variable for unlicensed server

* Mark server as unlicensed when ee checks fail

* Handle license validation failures with a (re)boot in a limited state

* Revert "Export a fatal error handled by the hook manager"

This reverts commit 069c21974a8d36e889c73ad78023448d787d62a5.

* Revert "Export ee.errValidation (ee.ErrValidation)"

This reverts commit 59dbab8c79773ca5d879f28cbaf53f3dd4297b9b.

* Revert "Add limited http handlers functionality to rest handler"

This reverts commit e2f1f28facaca54713db76a588839cd2733cf673.

* Revert "Handle license validation failures with a (re)boot in a limited state"

This reverts commit 58cfbbaf522a1345aac1fa67964ebff0a6d60cd8.

* Revert "Mark server as unlicensed when ee checks fail"

This reverts commit 77c6dbdd3c9cfa6e7d6becedef6251e8617ae367.

* Handle license validation failures with a middleware

* Forbid responses if unlicensed ee and not in status api

* Remove unused func

* feat(NET-449): add sync feature to request a host pull from server (#2491)

* fix(NET-486): change client name length validation (#2498)

set limit to 5<=x<=32

* [NET-477] Pick AMB URL dynamically (#2489)

* Introduce config for environment

* Introduce func to get environment

* Choose accounts api host from environment

* Test the ee package on workflows

* Use build tag ee for license_test.go

* [Feature]: nm-quick script tackling arm TODO support (#2488)

* domain flag for auto installs

* use static servers with custom domain (#2421)

* send delete peer update always

* fix add/remove host api calls

* keep mq updates in a single go func

* move branch test logic to devops (#2443)

* handle IOT OS

* save server name to env (#2460)

* ensure branch test servers available after test runs (#2467)

* save server name to env

* free server always; add PR to discord messages

* use correct method to delete droplets (#2468)

* quick fix for the launcher

* removed exit when triggering not supported exit and removed the TODO comments related to this issue

---------

Co-authored-by: Matthew R Kasun <[email protected]>
Co-authored-by: Alex Feiszli <[email protected]>
Co-authored-by: Christopher Blaha <[email protected]>
Co-authored-by: Abhishek Kondur <[email protected]>
Co-authored-by: Abhishek K <[email protected]>

* rebase conflict

* include pass and os in mirgration data

* node network ranges

* remove debugging logs

* add gateways

* use sent node

* upgrade shell script

* associate node to host during migration

* add node to host.Nodes and publish peer update

* save host outside loop

* fix script name

* simplify upgrade script

* don't migrate relays

* simplify upgrade script even more

* guard against blank address or address6

* typos

---------

Co-authored-by: Aceix <[email protected]>
Co-authored-by: Abhishek K <[email protected]>
Co-authored-by: Farukh Khan <[email protected]>
Co-authored-by: Gabriel de Souza Seibel <[email protected]>
Co-authored-by: bornav <[email protected]>
Co-authored-by: Alex Feiszli <[email protected]>
Co-authored-by: Christopher Blaha <[email protected]>
Co-authored-by: Abhishek Kondur <[email protected]>
2023-08-14 23:14:10 +05:30
Farukh Khan a9a8596d71 NET-507 (#2506)
* NET-507

* Fixed server restart changing the node expiration date to the defaults.

* Removed expired nodes removal from zombie cleanup routine.

* Added a new expired nodes deletion routine which removes expired nodes every hour.

* NET-507 suggested changes

* Possible fix for zombie nodes upon node deletion from the UI.

* Suggested changes implemented for expired nodes deletion go routine.

* NET-507 typo fix

* typo fix for DeleteNode purge parameter
2023-08-10 20:27:44 +05:30
Aceix 91859c2490 prune(NET-483): remove defunct host.internetgateway field (#2487) 2023-08-02 17:26:44 +05:30
Aceix a7914b1631 fix(NET-388): fix to update node dns toggle (#2472) 2023-07-28 21:28:36 +05:30
Abhishek K 645bc4c126 NET-447: Removed proxy related fields and code (#2459)
* remove related fields and code

* remover metrics collection from server code

* fw update struct

* add ext client flag to metrics data

* simply nat types

* rm proxy update from cli
2023-07-26 11:22:49 +05:30
Abhishek K 38454ece79 IOT-66: Single Peer update for IOT client (#2424)
* send peer update to IOT client only when it is relayed

* move node check

* send relay del update for iot client

* fix relay delete logic for iot

* set relay node to true for iot peer update

* add node addrs to peer update

* revert tag
2023-06-28 22:08:49 +05:30
Abhishek KandMatthew R Kasun 6c2582651c Net-137: Node relays (#2401)
* revert relays

* initial relay commit

* get relayed allowed ips

* add more relay validation checks, peer logic cleanup

* rm relayed nodes from relay node when relayed node is deleted

* fix egress updates for relayed nodes

* rm unused func

* remove  debug logs

* avoid adding egress ranges on the relayed gw node

---------

Co-authored-by: Matthew R Kasun <[email protected]>
2023-06-19 12:26:14 -04:00
Matthew R KasunandAceix 78640f1342 Extclient NET-63x (#2286)
* model changes

* additional fields for extclient create

* add DNS to extclient config

* extclient name checks

* update extclient

* nmctl extclient

* final tweaks

* review comments

* add extclientdns to node on ingress creation

* fix to add ingress dns to api (#2296)

---------

Co-authored-by: Aceix <[email protected]>
2023-05-17 10:58:03 -04:00
Matthew R Kasun 63552891ee refactor key updates on host basis vice node 2023-04-13 15:02:14 -04:00