gravitl/netmaker
1
0
Fork 0
mirror of https://github.com/gravitl/netmaker.git synced 2026-10-12 02:15:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,894 Commits 1,843 Branches 91 Tags
develop
Commit Graph
14 Commits
Author SHA1 Message Date
Vishal Dalwadi 132aee0bb3 NM-341-v6: AMB Multi-tenancy Management (#4101) 
* wip(go): return auth token with user on create org owner;

* wip(go): prevent use of default tenant slug ;

* wip(go): upsert settings for tenant in ctx;

* wip(go): use passed ctx when updating user; remove password validation from ValidateUser (already validated in UpdateUser);

* wip(go): qualify all tenant id usages with table name;

* wip(go): use passed ctx

* wip(go): return old enrollment key type on create;

* wip(go): prevent null networks return;

* wip(go): wire ctx in dns/nameserver functions; wire ctx in host registration functions;

* wip(go): wire ctx in node functions;

* wip(go): wire ctx in node functions;

* wip(go): wire ctx in extclient functions;

* wip(go): remove not needed field;

* wip(go): use passed ctx;

* wip(go): use unique index instead of primary key;

* wip(go): wire ctx in extclient apis;

* wip(go): wire ctx in extclient apis;

* wip(go): remove org id check from global scoped get tenant api;

* wip(go): add scope slug to discover response;

* wip(go): wire ctx in get users;

* wip(go): set tenant id in posture check violations;

* wip(go): upsert membership instead of user;

* wip(go): pass correct len;

* wip(go): optimize (single loop);

* wip(go): unify todos;

* wip(go): add todo;

* wip(go): wire context for network get;

* wip(go): wire context;

* wip(go): wire context;

* wip(go): wire ctx when getting tenant superadmin;

* wip(go): add network creator to network admin group;

* wip(go): check for user membership on sync;

* wip(go): add methods with strict membership check;

* wip(go): remove unwanted feature flags;

* wip(go): report per-tenant usage during license validation;

* wip(go): move email setting from tenant to org;

* wip(go): pass org scope to email init;

* wip(go): remove old tenant email settings usages; use org settings to get email settings

* wip(go): use get email settings from org settings;

* wip(go): remove email settings migration;

* wip(go): migrate email settings from server to org settings;

* wip(go): assign global network admin group membership only for super-admin and admins;

* wip(go): assign global network admin group membership only for super-admin and admins;

* wip(go): add membership alias to filter;

* wip(go): skip clearing local license validation response cache;

* wip(go): extend license validation response for multi-tenancy;

* wip(go): add functions to get sole tenant and org;

* wip(go): use tenant-specific feature flags;

* wip(go): add tenant status type;

* wip(go): remove old hook comment;

* wip(go): expect ctx during license validation;

* wip(go): always create default org and tenant for ce;

* wip(go): add tenant-specific check for license validation error;

* wip(go): cache license validation response in memory;

* wip(go): add todo to handle multiple-tenant for integration pull;

* wip(go): pass ctx during license validation;

* wip(go): add todo for making changes later; run hook separately for each tenant;

* wip(go): return specific errors when license validation response not in cache;

* wip(go): use retryable.httpclient for calling license validation api;

* wip(go): process license validation;

* wip(go): license validation function;

* wip(go): migration on license validation;

* wip(go): org settings not expected;

* wip(go): remove expiry from license validation response;

* wip(go): add limits and limits enforcement to license validation response;

* wip(go): remove expiry usages;

* wip(go): add arg to clear cache;

* wip(go): add arg to clear cache;

* wip(go): skip license validation if cached resp;

* wip(go): allow org_id and tenant_id query params;

* wip(go): upsert tenant membership on sync;

* wip(go): pass ctx to delete network roles;

* wip(go): create tenant membership on org owner creation on ce;

* wip(go): add tenant orchestrator;

* wip(go): use tenant orchestrator;

* wip(go): add default settings for tenant;

* wip(go): make org owner superadmin on all tenants;

* wip(go): simplify SyncOrgAndTenants;

* wip(go): retry license validation only 3 times

* wip(go): store invalid err;

* wip(go): add function to check if limits are to be enforced on a tenant;

* wip(go): pass tenant scope if possible when checking for license validation error;

* wip(go): remove merge errors;

* wip(go): add functions for limit checks;

* wip(go): wire host limit check;

* wip(go): wire ingress limit check;

* wip(go): wire egress limit check;

* wip(go): wire client limit check;

* wip(go): wire network limit check;

* wip(go): wire user limit check;

* wip(go): use correct errors;

* wip(go): skip telemetry for msps;

* wip(go): disable telemetry on msp tenants;

* wip(go): err check;

* wip(go): uncomment log event;

* wip(go): get sso state to get scope for non-headless oauth login;

* wip(go): inline delete pending user;

* wip(go): allow both org and tenant scope for pending users table;

* wip(go): add migration for pending user table during set tenant id and rekey org/tenant id;

* wip(go): update queries to use new scope fields in pending users table;

* wip(go): allow both org and tenant scope for user invites table;

* wip(go): update queries to use new scope fields;

* wip(go): add migration for rekey-ing org/tenant id and setting tenant id;

* wip(go): inline delete user invite;

* wip(go): set scope when creating invite;

* wip(go): get user count with membership is ctx is tenant scoped;

* wip(go): add methods to upsert and delete org memberships;

* wip(go): add global valid tenant and org roles map;

* wip(go): add scope query in user invite get by email;

* wip(go): add support for upsert and delete of org and tenant membership both;

* wip(go): infer scope for user invite apis;

* wip(go): wire ctx in get user invite by email;

* wip(go): allow no scope headers to be passed in infer scope apis;

* wip(go): add validation and checks for org-scoped invite api handlers;

* wip(go): wire ctx to is pending user check;

* wip(go): infer scope for pending user apis;

* wip(go): set default role based on ctx scope;

* wip(go): move mfa to tenant membership table; move account status to membership tables;

* wip(go): migrate account status and mfa settings;

* wip(go): update account status on sync;

* wip(go): use update mfa to update mfa setting;

* wip(go): set new membership settings on user creation;

* wip(go): use membership where required;

* wip(go): wire ctx on key unlink refs;

* wip(go): use r.Context directly without reinitializing db;

* wip(go): wire ctx in acls;

* wip(go): add tenant scoped primary key constructor;

* wip(go): use tenant scoped primary key for acls, dns, extclients and tags;

* wip(go): wire ctx for get groups in network;

* wip(go): wire ctx for get groups;

* wip(go): scope non-uuid group ids by tenant id;

* wip(go): migrate non-uuid group ids to scoped ids;

* wip(go): don't expect scope id in invite validation and signup apis;

* wip(go): wire ctx for get matching user; allow org scope when querying by external idp id;

* wip(go): add mfa to org;

* wip(go): scope preauth token;

* wip(go): add fallback for older auth tokens;

* wip(go): wire ctx for preauth token creation;

* wip(go): migrate mfa settings;

* wip(go): resolve mfa settings for inherited auth;

* wip(go): update mfa for org;

* wip(go): infer scope for totp apis;

* wip(go): use ctx when passed;

* wip(go): infer scope for list user apis;

* wip(go): ensure set primary keys don't cause errors during fetch;

* wip(go): make primary key for integrations uuid and store provider in separate key;

* wip(go): migrate integration ids;

* wip(go): use provider instead of id everywhere;

* wip(go): remove gorm column;

* wip(go): wire ctx for igw egress creation;

* wip(go): wire ctx for igw egress update;

* wip(go): wire ctx for host registration;

* wip(go): wire ctx for host registration;

* wip(go): merge errors;

* wip(go): merge errors;

* wip(go): wire ctx; remove default scope usages;

* wip(go): use get with membership;

* wip(go): org scope license validation;

* wip(go): remove debug log;

* wip(go): remove scoping on list roles;

* wip(go): ignore invite creation if already exists;

* wip(go): add is msp func to check if deployment is msp deployment;

* wip(go): ensure username is email for msp users;

* wip(go): add email validated check on user table;

* wip(go): mark email validated when oauth user;

* wip(go): handle gorm error;

* wip(go): add tenant id to siem grpc schema;

* wip(go): read tenant from scoped ctx;

* wip(go): pass scoped ctx to siem grpc client;

* wip(go): remove unused publisher;

* wip(go): publish all tenant exporter updates on pull;

* wip(go): fix wg usage;

* wip(go): tenant scope the ctx when getting server info;

* wip(go): add tenant id to enrollment key token;

* wip(go): make host register tenant scope;

* wip(go): add default settings;

* wip(go): use correct checks for is msp tenant;

* wip(go): add email validation extension;

* wip(go): add pages for validation check;

* wip(go): use extension for validating user;

* wip(go): add invite type;

* wip(go): create only one validation invite per user email;

* wip(go): add validate email handler;

* wip(go): add email builder;

* wip(go): add user invite type migration logic;

* wip(go): only log email send failures;

* wip(go): add tenant id to flows table;

* wip(go): add func to check if connected to clickhouse;

* wip(go): migrate clickhouse;

* wip(go): add tenant id to mdm and edr state;

* wip(go): add delete methods for tables;

* wip(go): add tenant teardown method;

* wip(go): handle tenant deletion;

* wip(go): clean up flow logs on tenant deletion;

* wip(go): add update org membership role id method;

* wip(go): add transfer org ownership api;

* wip(go): move audit logs retention period to org settings;

* wip(go):get retention period from org settings;

* wip(go): remove old audit logs usage;

* wip(go): migrate audit logs retention period;

* wip(go): unmarshal decodes nums as float64;

* wip(go): run flow logs cleanup for all tenants in one go;

* wip(go): add audit logs retention period and smtp settings to tenant settings table for non-msp tenants;

* wip(go): resolve retention period and smtp settings correctly;

* wip(go): handle retention period and smtp settings updates and fetches correctly;

* wip(go): overlay retention period and smtp settings on server settings;;

* wip(go): upsert overlayed settings;

* wip(go): add azure tenant id to org settings;

* wip(go): tenant scope list roles;

* wip(go): add noop creds;

* wip(go): use first instead of find;

* wip(go): use correct roles when transfering ownership;

* wip(go): delete access tokens only if tenant scoped;

* wip(go): make delete user work for both org and tenant scope;

* wip(go): delete extclients only if tenant scoped;

* wip(go): check if caller is allowed to delete user;

* wip(go): infer scope for update user;

* wip(go): prevent org-owner elevation;

* wip(go): permissions check for update;

* wip(go): delete user access tokens only if tenant scope;

* wip(go): delete extclients only if tenant scope;

* wip(go): move upsert membership in upsert user;

* wip(go): set api host when constructing server config;

* wip(go): fix static errors;

* wip(go): fix test cases;

* wip(go): use correct get ns order;

* wip(go): use correct get pending host order;

* wip(go): use correct get access token order;

* wip(go): use correct get event order;

* wip(go): return error response on emqx user creation error;

* wip(go): store correct value in cache;

* wip(go): return better error response;

* wip(go): return better error response;

* wip(go): use cache when listing egress acls;

* wip(go): wire ctx in UserHasNetworkGroupAccess;

* wip(go): remove double scope wrapping;

* wip(go): add security check for list and get org;

* wip(go): ratelimit discover api;

* wip(go): expect sole org id in get and create org owner;

* wip(go): wrap create org owner in tx;
 2026-08-05 21:25:02 +08:00
Vishal DalwadiandAbhishek Kondur f9bbbd7efa NM-341: Schema Migration for Serverconf, Generated and Enrollment Keys Table (#4065) 
* feat(go): remove enrollment key cache;

* feat(go): add internal schema;

* feat(go): use the new internal table to fetch serverconf records;

* feat(go): move internal key constants to schema/internal.go;

* feat(go): add server conf table migration;

* feat(go): use the new internal table for generated records;

* feat(go): add generated table migration;

* feat(go): use the new internal table for serveruuid records;

* feat(go): add server uuid table migration;

* chore(go): remove empty file;

* feat(go): store license validation cached response in the new internal table;

* chore(go): move table name constants;

* feat(go): use migration job name v1.7.0;

* wip(go): add schema definition for enrollment keys;

* wip(go): use schema.EnrollmentKey in orchestrator pkg;

* wip(go): use schema.EnrollmentKey in host registration and pending host approval;

* wip(go): use schema.EnrollmentKey in gateway/node cleanup;

* wip(go): use schema.EnrollmentKey in network cleanup;

* wip(go): use schema.EnrollmentKey in creating default network enrollment key;

* wip(go): use schema.EnrollmentKey in enrollment key logic/controller;

* wip(go): add schema migration for enrollment keys;

* fix(go): remove ref to non-existent constant;

* fix(go): use correct package+type for orchestrator test;

* feat(go): add internal table to list models function;

* fix(go): use int for enrollment key type;

* fix(go): remove tags from networks;

* fix(go): ensure same response for get enrollment keys as before;

* fix(go): enrollment key migration;

* wip(go): remove sqlite, rqlite, postgres and utils from database package;

* wip(go): remove unused database pkg method DeleteAllRecords;

* wip(go): remove db connection management code and accompanying consts from database/ pkg;

* wip(go): add schema definitions for org, tenant and org settings tables;

* wip(go): add tenant id to add tables in schema/ pkg; add tenant-level unique constraints for network, user invite and pending users;

* wip(go): remove usages of host actions table;

* wip(go): create default org and tenant on migration;

* wip(go): manage kv tables with gorm;

* wip(go): bring network id out of value for acls, dns, extclients, metrics and tags tables;

* wip(go): store user settings in user table;

* wip(go): add server settings migration logic;

* wip(go): replace database/ pkg functions with methods from the schema/ pkg;

* wip(go): add schema definitions for tenant settings; replace database/ pkg functions with methods from the schema/ pkg;

* wip(go): add index on tenant id;

* wip(go): remove tenant id column from user role table;

* wip(go): set tenant_id to default tenant's id on all records in all tables on migration;

* wip(go): extract out network_id on acls, dns, extclients, metrics and tags tables on migration;

* wip(go): add scope pkg with middleware to extract scope from headers or set default scope;

* wip(go): scope (most) apis to tenant;

* wip(go): remove users and user roles table from set tenant id migration function;

* wip(go): handle test case errors;

* wip(go): cache default tenant and org ids;

* feat(go): add utils to get server settings during migration;

* feat(go): check nil before conversion;

* feat(go): use correct arg order;

* wip(go): pass db context;

* wip(go): correct api handler router tag;

* wip(go): add schema definitions for org and tenant membership tables;

* wip(go): use middleware from middleware/ package;

* wip(go): add org role constants;

* wip(go): read user tenant role and groups from tenant membership table;

* wip(go): add migration logic for memberships;

* wip(go): undo inadvertent paste;

* wip(go): add migration logic for migrating user's role and groups to tenant membership table and setting org owner;

* wip(go): use tenant membership table to set user role and groups;

* wip(go): test cases;

* feat(go): check for nil;

* feat(go): move auth type, password and external idp id to tenant membership table;

* wip(go): use middleware from middleware/ package;

* wip(go): add schema definitions for org and tenant membership tables;

* wip(go): add temp default function to use default tenant scoped contexts everywhere;

* wip(go): use scope.Default to set tenant id before db ops;

* wip(go): create default org and tenant for tests as well;

* wip(go): move Default to logic to remove import cycle;

* wip(go): scope auth request to default context;

* wip(go): migration fixes;

* wip(go): use correct column; migrate membership on user migrate;

* fix(go): add smtp tls verify skip setting;

* fix(go): add smtp tls verify skip setting;

---------

Co-authored-by: Abhishek Kondur <[email protected]>
 2026-07-06 23:00:21 +05:30
VishalDalwadi b8016780b9 fix(go): remove debug logs; 2026-04-02 22:35:54 +05:30
VishalDalwadi bb7f640df4 fix(go): pass array as arg; 2026-04-02 22:34:23 +05:30
VishalDalwadi 85c1ecee35 fix(go): use start_ts instead of version; 2026-04-02 22:14:27 +05:30
VishalDalwadi f07ce69839 fix(go): allow multivalued filters; 2026-04-02 17:35:41 +05:30
VishalDalwadi 8dc250bcd1 fix(go): remove incomplete flow logs; 2026-04-02 15:22:30 +05:30
Abhishek KondurVishalDalwadiVishal Dalwadi
 		12cc967ba1 Fixes/v1.5.1 (#3938) 
* fix(go): set persistent keep alive when registering host using sso;

* fix(go): run posture check violations on delete;

* fix(go): upsert node on approving pending host;

* fix(go): resolve concurrency issues during group delete cleanup;

* fix(go): update doc links;

* fix(go): add created and updated fields to host;

* fix(go): skip delete and update superadmin on sync users;

* fix(go): use conn directly for now;

* fix(go): remove acl for idp groups;

* fix(go): quote fields;

* fix(go): use filters with count;

* feat(go): add a search query;

* fix(go): cleanup acls;

* fix(go): review fixes;

* fix(go): remove additional loop;

* fix(go): fix

* v1.5.1: separate out idp sync and reset signals for HA

* v1.5.1: add grps with name for logging

* v1.5.1: clear posture check violations when all checks are deleted

* v1.5.1: set static when default host

* v1.5.1: fix db status check

* rm set max conns

* v1.5.1: reset auto assigned gw when disconnected

* fix(go): skip global network admin and user groups when splitting;

* v1.5.1: fix update node call from client

* fix(go): separate out migration from normal usage;

* fix(go): skip default groups;

* fix(go): create policies for existing groups on network create;

* fix(go): skip fatal log on clickhouse conn;

* fix(go): add posture check cleanup;

---------

Co-authored-by: VishalDalwadi <[email protected]>
Co-authored-by: Vishal Dalwadi <[email protected]>
 2026-03-28 01:08:59 +05:30
Abhishek Kondur 9b4f9a4a50 NM-240: update api swagger docs (#3861) 
* NM-240: update api swagger docs

* NM-240: remove api doc for debug api

* NM-240: generate swagger

* NM-240: remove deprecated and unused apis, update jit models

* NM-240: deprecate ununsed apis from api docs
 2026-02-11 10:03:28 +04:00
Abhishek Kondur b935e3795a NM-228: add middleware checks for activity logs, remove all users get… (#3853) 
* NM-228: add middleware checks for activity logs, remove all users get access platform users

* NM-228: fix middleware checks for platform users

* NM-228: add middleware check for user activity
 2026-02-10 21:13:55 +04:00
Vishal Dalwadi d476e79278 NM-220: Flow Logs Improvements (#3812) 
* feat(go): add name to src and dst info;

* feat(go): populate names to share with netclient;

* feat(go): add hostname to flow event;
 2026-01-28 09:05:51 +04:00
Vishal Dalwadi 2062da5081 fix: construct a and-or clause instead of an and-and clause; (#3796) 2025-12-22 16:16:55 +04:00
Abhishek KondurandVishalDalwadi 0c9ed2542b NM-202: fix egress domain routing (#3793) 
* NM-195: add key tags info to posture check on join

* NM-195: add network user grps to posture check

* NM-195: add posture checks to middleware

* fix: return error when group network roles are set for specific networks and all networks;

* add all posture check to rsrc permission check func

* NM-202: fix egress domain routing

* fix: add username filter;

* feat: add fallback nameserver support;

* fix: add validation for pro as well;

* fix: skip fallback domains for user gws;

* fix: don't set domains for fallback dns servers;

* fix: validation fixes;

* fix: empty match domains for fallback nameservers;

---------

Co-authored-by: VishalDalwadi <[email protected]>
 2025-12-22 10:10:40 +04:00
Vishal Dalwadi a4981ffd26 NM-168: Network Flow Logs (#3754) 
* feat(go): define flow events;

* feat(go): improve structure;

* feat(go): improve structure;

* feat(go): remove old flow definitions;

* feat(sql): add clickhouse init scripts;

* feat(sql): add protobuf spec;

* fix(sql): store ip as string;

* feat(go): move proto def to grpc dir;

* feat(go): use node instead of host as type; optimize protobuf defs;

* feat(go): add clickhouse db support; add endpoint to query flows;

* fix(go): fix clickhouse config;

* fix(go): use error response structure to report error;

* feat(go): pass flow logging status to netclient;

* feat(go): add peer ip identity map to host peer info;

* feat(go): remove prefix from participant obj fields;

* feat(go): add flow logs enabled field to host;

* feat(go): add filtering to get flow api;

* feat(go): fix record struct;

* feat(go): add exporter url to server config;

* feat(go): add exporter url to server config;

* feat(go): enable flow logs by default;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): update nm-quick.sh;

* feat(go): add db initialization logic;

* feat(go): filter by network id;

* fix(go): connection issue;

* fix(go): connection issue;

* fix(go): golang builder version;

* feat(go): add server settings for flow logs;

* feat(go): initialize clickhouse in pro; check for retention;

* feat(go): add exporter feature flags;

* feat(go): add grpc behind caddy;

* feat(go): expose ports correctly;

* fix(go): grpc caddyfile config;

* fix(go): publish exporter feature flags on license validation;

* fix(go): set server name for netmaker exporter;

* fix(go): set server name for netmaker exporter;

* fix(go): check for nil cancel func;

* fix(go): add flow logs field to api host;

* fix(go): add flow logs field to api host;

* fix(go): remove port from grpc setting;

* chore(go): tabs;

* feat(go): introduce egress range participant type;.

* feat(go): rename egress range to egress route for uniform language;

* feat(go): rename egress range to egress route for uniform language;

* feat: add peer addr identity map to host peer update;

* feat: add address identity map to host peer update;

* feat: add address identity map to host peer update;

* feat: set correct from and to args;

* feat: add support for filtering by node;

* feat: use corresponding base image;

* feat: update dockerfile base image version;

* fix: disable flow logs for all host when global settings are changed;
 2025-12-12 14:12:00 +04:00