* wip(go): return auth token with user on create org owner;
* wip(go): prevent use of default tenant slug ;
* wip(go): upsert settings for tenant in ctx;
* wip(go): use passed ctx when updating user; remove password validation from ValidateUser (already validated in UpdateUser);
* wip(go): qualify all tenant id usages with table name;
* wip(go): use passed ctx
* wip(go): return old enrollment key type on create;
* wip(go): prevent null networks return;
* wip(go): wire ctx in dns/nameserver functions; wire ctx in host registration functions;
* wip(go): wire ctx in node functions;
* wip(go): wire ctx in node functions;
* wip(go): wire ctx in extclient functions;
* wip(go): remove not needed field;
* wip(go): use passed ctx;
* wip(go): use unique index instead of primary key;
* wip(go): wire ctx in extclient apis;
* wip(go): wire ctx in extclient apis;
* wip(go): remove org id check from global scoped get tenant api;
* wip(go): add scope slug to discover response;
* wip(go): wire ctx in get users;
* wip(go): set tenant id in posture check violations;
* wip(go): upsert membership instead of user;
* wip(go): pass correct len;
* wip(go): optimize (single loop);
* wip(go): unify todos;
* wip(go): add todo;
* wip(go): wire context for network get;
* wip(go): wire context;
* wip(go): wire context;
* wip(go): wire ctx when getting tenant superadmin;
* wip(go): add network creator to network admin group;
* wip(go): check for user membership on sync;
* wip(go): add methods with strict membership check;
* wip(go): remove unwanted feature flags;
* wip(go): report per-tenant usage during license validation;
* wip(go): move email setting from tenant to org;
* wip(go): pass org scope to email init;
* wip(go): remove old tenant email settings usages; use org settings to get email settings
* wip(go): use get email settings from org settings;
* wip(go): remove email settings migration;
* wip(go): migrate email settings from server to org settings;
* wip(go): assign global network admin group membership only for super-admin and admins;
* wip(go): assign global network admin group membership only for super-admin and admins;
* wip(go): add membership alias to filter;
* wip(go): skip clearing local license validation response cache;
* wip(go): extend license validation response for multi-tenancy;
* wip(go): add functions to get sole tenant and org;
* wip(go): use tenant-specific feature flags;
* wip(go): add tenant status type;
* wip(go): remove old hook comment;
* wip(go): expect ctx during license validation;
* wip(go): always create default org and tenant for ce;
* wip(go): add tenant-specific check for license validation error;
* wip(go): cache license validation response in memory;
* wip(go): add todo to handle multiple-tenant for integration pull;
* wip(go): pass ctx during license validation;
* wip(go): add todo for making changes later; run hook separately for each tenant;
* wip(go): return specific errors when license validation response not in cache;
* wip(go): use retryable.httpclient for calling license validation api;
* wip(go): process license validation;
* wip(go): license validation function;
* wip(go): migration on license validation;
* wip(go): org settings not expected;
* wip(go): remove expiry from license validation response;
* wip(go): add limits and limits enforcement to license validation response;
* wip(go): remove expiry usages;
* wip(go): add arg to clear cache;
* wip(go): add arg to clear cache;
* wip(go): skip license validation if cached resp;
* wip(go): allow org_id and tenant_id query params;
* wip(go): upsert tenant membership on sync;
* wip(go): pass ctx to delete network roles;
* wip(go): create tenant membership on org owner creation on ce;
* wip(go): add tenant orchestrator;
* wip(go): use tenant orchestrator;
* wip(go): add default settings for tenant;
* wip(go): make org owner superadmin on all tenants;
* wip(go): simplify SyncOrgAndTenants;
* wip(go): retry license validation only 3 times
* wip(go): store invalid err;
* wip(go): add function to check if limits are to be enforced on a tenant;
* wip(go): pass tenant scope if possible when checking for license validation error;
* wip(go): remove merge errors;
* wip(go): add functions for limit checks;
* wip(go): wire host limit check;
* wip(go): wire ingress limit check;
* wip(go): wire egress limit check;
* wip(go): wire client limit check;
* wip(go): wire network limit check;
* wip(go): wire user limit check;
* wip(go): use correct errors;
* wip(go): skip telemetry for msps;
* wip(go): disable telemetry on msp tenants;
* wip(go): err check;
* wip(go): uncomment log event;
* wip(go): get sso state to get scope for non-headless oauth login;
* wip(go): inline delete pending user;
* wip(go): allow both org and tenant scope for pending users table;
* wip(go): add migration for pending user table during set tenant id and rekey org/tenant id;
* wip(go): update queries to use new scope fields in pending users table;
* wip(go): allow both org and tenant scope for user invites table;
* wip(go): update queries to use new scope fields;
* wip(go): add migration for rekey-ing org/tenant id and setting tenant id;
* wip(go): inline delete user invite;
* wip(go): set scope when creating invite;
* wip(go): get user count with membership is ctx is tenant scoped;
* wip(go): add methods to upsert and delete org memberships;
* wip(go): add global valid tenant and org roles map;
* wip(go): add scope query in user invite get by email;
* wip(go): add support for upsert and delete of org and tenant membership both;
* wip(go): infer scope for user invite apis;
* wip(go): wire ctx in get user invite by email;
* wip(go): allow no scope headers to be passed in infer scope apis;
* wip(go): add validation and checks for org-scoped invite api handlers;
* wip(go): wire ctx to is pending user check;
* wip(go): infer scope for pending user apis;
* wip(go): set default role based on ctx scope;
* wip(go): move mfa to tenant membership table; move account status to membership tables;
* wip(go): migrate account status and mfa settings;
* wip(go): update account status on sync;
* wip(go): use update mfa to update mfa setting;
* wip(go): set new membership settings on user creation;
* wip(go): use membership where required;
* wip(go): wire ctx on key unlink refs;
* wip(go): use r.Context directly without reinitializing db;
* wip(go): wire ctx in acls;
* wip(go): add tenant scoped primary key constructor;
* wip(go): use tenant scoped primary key for acls, dns, extclients and tags;
* wip(go): wire ctx for get groups in network;
* wip(go): wire ctx for get groups;
* wip(go): scope non-uuid group ids by tenant id;
* wip(go): migrate non-uuid group ids to scoped ids;
* wip(go): don't expect scope id in invite validation and signup apis;
* wip(go): wire ctx for get matching user; allow org scope when querying by external idp id;
* wip(go): add mfa to org;
* wip(go): scope preauth token;
* wip(go): add fallback for older auth tokens;
* wip(go): wire ctx for preauth token creation;
* wip(go): migrate mfa settings;
* wip(go): resolve mfa settings for inherited auth;
* wip(go): update mfa for org;
* wip(go): infer scope for totp apis;
* wip(go): use ctx when passed;
* wip(go): infer scope for list user apis;
* wip(go): ensure set primary keys don't cause errors during fetch;
* wip(go): make primary key for integrations uuid and store provider in separate key;
* wip(go): migrate integration ids;
* wip(go): use provider instead of id everywhere;
* wip(go): remove gorm column;
* wip(go): wire ctx for igw egress creation;
* wip(go): wire ctx for igw egress update;
* wip(go): wire ctx for host registration;
* wip(go): wire ctx for host registration;
* wip(go): merge errors;
* wip(go): merge errors;
* wip(go): wire ctx; remove default scope usages;
* wip(go): use get with membership;
* wip(go): org scope license validation;
* wip(go): remove debug log;
* wip(go): remove scoping on list roles;
* wip(go): ignore invite creation if already exists;
* wip(go): add is msp func to check if deployment is msp deployment;
* wip(go): ensure username is email for msp users;
* wip(go): add email validated check on user table;
* wip(go): mark email validated when oauth user;
* wip(go): handle gorm error;
* wip(go): add tenant id to siem grpc schema;
* wip(go): read tenant from scoped ctx;
* wip(go): pass scoped ctx to siem grpc client;
* wip(go): remove unused publisher;
* wip(go): publish all tenant exporter updates on pull;
* wip(go): fix wg usage;
* wip(go): tenant scope the ctx when getting server info;
* wip(go): add tenant id to enrollment key token;
* wip(go): make host register tenant scope;
* wip(go): add default settings;
* wip(go): use correct checks for is msp tenant;
* wip(go): add email validation extension;
* wip(go): add pages for validation check;
* wip(go): use extension for validating user;
* wip(go): add invite type;
* wip(go): create only one validation invite per user email;
* wip(go): add validate email handler;
* wip(go): add email builder;
* wip(go): add user invite type migration logic;
* wip(go): only log email send failures;
* wip(go): add tenant id to flows table;
* wip(go): add func to check if connected to clickhouse;
* wip(go): migrate clickhouse;
* wip(go): add tenant id to mdm and edr state;
* wip(go): add delete methods for tables;
* wip(go): add tenant teardown method;
* wip(go): handle tenant deletion;
* wip(go): clean up flow logs on tenant deletion;
* wip(go): add update org membership role id method;
* wip(go): add transfer org ownership api;
* wip(go): move audit logs retention period to org settings;
* wip(go):get retention period from org settings;
* wip(go): remove old audit logs usage;
* wip(go): migrate audit logs retention period;
* wip(go): unmarshal decodes nums as float64;
* wip(go): run flow logs cleanup for all tenants in one go;
* wip(go): add audit logs retention period and smtp settings to tenant settings table for non-msp tenants;
* wip(go): resolve retention period and smtp settings correctly;
* wip(go): handle retention period and smtp settings updates and fetches correctly;
* wip(go): overlay retention period and smtp settings on server settings;;
* wip(go): upsert overlayed settings;
* wip(go): add azure tenant id to org settings;
* wip(go): tenant scope list roles;
* wip(go): add noop creds;
* wip(go): use first instead of find;
* wip(go): use correct roles when transfering ownership;
* wip(go): delete access tokens only if tenant scoped;
* wip(go): make delete user work for both org and tenant scope;
* wip(go): delete extclients only if tenant scoped;
* wip(go): check if caller is allowed to delete user;
* wip(go): infer scope for update user;
* wip(go): prevent org-owner elevation;
* wip(go): permissions check for update;
* wip(go): delete user access tokens only if tenant scope;
* wip(go): delete extclients only if tenant scope;
* wip(go): move upsert membership in upsert user;
* wip(go): set api host when constructing server config;
* wip(go): fix static errors;
* wip(go): fix test cases;
* wip(go): use correct get ns order;
* wip(go): use correct get pending host order;
* wip(go): use correct get access token order;
* wip(go): use correct get event order;
* wip(go): return error response on emqx user creation error;
* wip(go): store correct value in cache;
* wip(go): return better error response;
* wip(go): return better error response;
* wip(go): use cache when listing egress acls;
* wip(go): wire ctx in UserHasNetworkGroupAccess;
* wip(go): remove double scope wrapping;
* wip(go): add security check for list and get org;
* wip(go): ratelimit discover api;
* wip(go): expect sole org id in get and create org owner;
* wip(go): wrap create org owner in tx;
* feat(go): remove enrollment key cache;
* feat(go): add internal schema;
* feat(go): use the new internal table to fetch serverconf records;
* feat(go): move internal key constants to schema/internal.go;
* feat(go): add server conf table migration;
* feat(go): use the new internal table for generated records;
* feat(go): add generated table migration;
* feat(go): use the new internal table for serveruuid records;
* feat(go): add server uuid table migration;
* chore(go): remove empty file;
* feat(go): store license validation cached response in the new internal table;
* chore(go): move table name constants;
* feat(go): use migration job name v1.7.0;
* wip(go): add schema definition for enrollment keys;
* wip(go): use schema.EnrollmentKey in orchestrator pkg;
* wip(go): use schema.EnrollmentKey in host registration and pending host approval;
* wip(go): use schema.EnrollmentKey in gateway/node cleanup;
* wip(go): use schema.EnrollmentKey in network cleanup;
* wip(go): use schema.EnrollmentKey in creating default network enrollment key;
* wip(go): use schema.EnrollmentKey in enrollment key logic/controller;
* wip(go): add schema migration for enrollment keys;
* fix(go): remove ref to non-existent constant;
* fix(go): use correct package+type for orchestrator test;
* feat(go): add internal table to list models function;
* fix(go): use int for enrollment key type;
* fix(go): remove tags from networks;
* fix(go): ensure same response for get enrollment keys as before;
* fix(go): enrollment key migration;
* wip(go): remove sqlite, rqlite, postgres and utils from database package;
* wip(go): remove unused database pkg method DeleteAllRecords;
* wip(go): remove db connection management code and accompanying consts from database/ pkg;
* wip(go): add schema definitions for org, tenant and org settings tables;
* wip(go): add tenant id to add tables in schema/ pkg; add tenant-level unique constraints for network, user invite and pending users;
* wip(go): remove usages of host actions table;
* wip(go): create default org and tenant on migration;
* wip(go): manage kv tables with gorm;
* wip(go): bring network id out of value for acls, dns, extclients, metrics and tags tables;
* wip(go): store user settings in user table;
* wip(go): add server settings migration logic;
* wip(go): replace database/ pkg functions with methods from the schema/ pkg;
* wip(go): add schema definitions for tenant settings; replace database/ pkg functions with methods from the schema/ pkg;
* wip(go): add index on tenant id;
* wip(go): remove tenant id column from user role table;
* wip(go): set tenant_id to default tenant's id on all records in all tables on migration;
* wip(go): extract out network_id on acls, dns, extclients, metrics and tags tables on migration;
* wip(go): add scope pkg with middleware to extract scope from headers or set default scope;
* wip(go): scope (most) apis to tenant;
* wip(go): remove users and user roles table from set tenant id migration function;
* wip(go): handle test case errors;
* wip(go): cache default tenant and org ids;
* feat(go): add utils to get server settings during migration;
* feat(go): check nil before conversion;
* feat(go): use correct arg order;
* wip(go): pass db context;
* wip(go): correct api handler router tag;
* wip(go): add schema definitions for org and tenant membership tables;
* wip(go): use middleware from middleware/ package;
* wip(go): add org role constants;
* wip(go): read user tenant role and groups from tenant membership table;
* wip(go): add migration logic for memberships;
* wip(go): undo inadvertent paste;
* wip(go): add migration logic for migrating user's role and groups to tenant membership table and setting org owner;
* wip(go): use tenant membership table to set user role and groups;
* wip(go): test cases;
* feat(go): check for nil;
* feat(go): move auth type, password and external idp id to tenant membership table;
* wip(go): use middleware from middleware/ package;
* wip(go): add schema definitions for org and tenant membership tables;
* wip(go): add temp default function to use default tenant scoped contexts everywhere;
* wip(go): use scope.Default to set tenant id before db ops;
* wip(go): create default org and tenant for tests as well;
* wip(go): move Default to logic to remove import cycle;
* wip(go): scope auth request to default context;
* wip(go): migration fixes;
* wip(go): use correct column; migrate membership on user migrate;
* fix(go): add smtp tls verify skip setting;
* fix(go): add smtp tls verify skip setting;
---------
Co-authored-by: Abhishek Kondur <[email protected]>
* feat(go): add schema for pending users table;
1. Schema Definition for Pending Users table.
2. Use the newer table everywhere.
3. Migration stubs for v1.5.2.
4. Migration code for Pending Users table;
* feat(go): add schema for user invites table;
1. Schema Definition for User Invites table.
2. Use the newer table everywhere.
3. Migration code for User Invites table;
* feat(go): add schema definitions for nodes, gateways and posture check violations table;
* feat(go): remove legacy node usages;
* fix(go): address code review comments;
1. remove column names in primary key for PostureCheckViolation struct.
2. use correct column name for network_id.
3. handle rollback error.
* feat(go): check ip uniqueness using db query;
* feat(go): remove nodes and network nodes cache; remove unused functions and consts;
* feat(go): delete unused function GetExtClientHAConf;
* feat(go): optimize DoesHostExistInTheNetworkAlready;
* feat(go): remove legacy node and migration;
* feat(go): implement network orchestrator;
1. Implements IP Allocation for Nodes and Extclients;
* feat(go): implement gateway orchestrator and extensions;
1. Implements CreateGateway;
2. Pro and CE Gateway Extensions to ConfigureAutoRelay;
* feat(go): implement node orchestrator and extensions;
1. Implements CreateNode;
2. Pro and CE Node Extensions to ConfigureAutoAssignGateway;
* feat(go): implement orchestrator repository and extensions factory;
* feat(go): add repository initialization with correct factory;
* feat(go): remove persistent keep alive and mtu from gateway;
* feat(go): add auto relayed peers to node;
* feat(go): rename relay gateway to relaying node; use null string instead of string pointer;
* fix(go): merge errors;
* feat(go): add update node last checkin function;
* feat(go): wire orchestrator create node;
1. Enrollment Key Join.
2. API: Add Host to Network.
3. API: Approve Pending Host.
4. API: Update Host (Make Host Default).
5. API: Create Network (Add Default Hosts to Network).
* feat(go): inline create extclient;
Remove cyclic dependency between logic and orchestrator;
* feat(go): remove old create node functions;
* feat(go): optimize bulk update node connected status;
* feat(go): mark pending delete directly; move constants;
* feat(go): remove unused validations;
* feat(go): remove allocated ip map cache;
* feat(go): use host persistent keep alive and mtu for extclient config;
* feat(go): remove (almost) all failover "unused" code;
* feat(go): guard address allocation with mtx;
* feat(go): validate node ip uniqueness with network orchestrator;
* feat(go): convert schema node to models node for host update;
* feat(go): remove nodes cache initialization;
* feat(go): use mark for deletion for node soft delete and delete for hard delete; add preload options for node get;
* feat(go): removed unused code;
1. No calls for GetRelays and commented GetRelayedNodes.
2. GetNodeEgressInfo populates egress details for node but they aren't validated here.
* feat(go): convert duration to seconds;
* feat(go): remove gateway schema, extensions and orchestrator; move gateway management to nodes extension and orchestrator;
* feat(go): remove self-referential foreign key;
* feat(go): add converters for schema node; use net tables for getting a node and listing all nodes;
* feat(go): remove unused code;
* feat(go): use models.Node for delete channel;
* feat(go): optimize node telemetry gen;
* feat(go): pass schema.Node to check zombies;
* feat(go): add model.Node to schema.Node conversion;
* feat(go): upsert violations on node upsert;
* feat(go): remove all SetDNS usage;
* feat(go): use fetchAll instead of database.FetchRecords;
* feat(go): add nodes migration;
* feat(go): remove unused node methods;
* refactor(go): improve field names;
* feat(go): remove unused network methods;
* feat(go): move type NodeStatus and status constants to scheme package;
* feat(go): add list network nodes paginated api;
* feat(go): add filters to list nodes paginated api;
* feat(go): improve logs; manage tx outside of schema pkg;
* feat(go): ensure address empty for nil ips;
* feat(go): add json tags;
* feat(go): add json tags;
* feat(go): use string ptr instead of NullString;
* feat(go): use api host in list response;
* feat(go): optimize list posture check violations api;
* feat(go): list only current violations;
* feat(go): scope table field on join;
* NM-293: fix static checks
* NM-293: fix static checks
* NM-293: fix static checks
* Update orchestrator/network.go
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
* NM-293: fix invalid gorm struct tags
* NM-293: Only set RelayingNodeID when the node is actually relayed.
* NM-293: use RelayedByNodeID for clarity
* feat(go): add pkg for json expressions; add node methods to do json operations in db;
* feat(go): make options common for all orchestrators; use orchestrator create gateway in api;
* feat(go): add delete all methods; add set internet gateway in db;
* feat(go): set is ingress gateway to is gateway;
* feat(go): restore igws handlers;
* feat(go): use network name in tag;
* feat(go): add skip mq host and node update options;
* feat(go): add node orchestrator test suite;
* feat(go): set network addr range on conversion;
* feat(go): upsert node instead of update node;
* feat(go): remove old todos;
* feat(go): use only is_gw to determine whether gateway;
* feat(go): add create node with default host test case;
* feat(go): add create node with enrollment key test case;
* feat(go): add pro node orchestrator test suite;
* feat(go): update node and client status on update metrics;
* feat(go): add node status filters;
* feat(go): update node status on bulk update as well;
* feat(go): prevent reset of host password on upsert;
* feat(go): disable platform/service user device join;
* feat(go): disable platform/service user device join;
* feat(go): add search across fields for list network nodes api;
* feat(go): add field for additional gateway endpoints;
* feat(go): configure tags only if they belong to the node network;
* feat(go): optimize tag assignments and unassignments;
* feat(go): add posture check violations check before create node;
Check not added for default host on network create;
* feat(go): copy key before editing key networks to prevent changes in the cache;
* feat(go): construct another network slice instead of in-place delete;
* feat(go): error only when no networks left to join;
* feat(go): move set defaults to schema migration;
* feat(go): load server settings in cache before migration;
* feat(go): migrate gateway nameserver and egress along with nodes;
* feat(go): disable netclient auto update if disabled globally;
* refactor(go): move network nameserver migration to separate function;
* feat(go): calculate node status on migrate (skipping metrics check);
Previously, we weren't setting the node status in db, just recalculating on the fly. With this version, we expect the node status to be in the db and recalculated when metrics are pushed from netclient. So we have to recalculate the node status to initialize the status value.
* feat(go): update only if nodeIDs is non-empty;
WithFilter adds the where clause only if atleast one value is passed. To prevent updating status of all nodes, add a check for nodeIDs.
* fix(go): remove tag;
Execute assign tag only when at least one node has to be tagged;
* feat(go): add egress nat mode migration;
* fix(go): use correct network id;
* fix(go): make is_auto_relay string;
* fix(go): create tag before use to pass test;
* fix(go): move delete to testutils package;
---------
Co-authored-by: abhishek9686 <[email protected]>
Co-authored-by: tenki-reviewer[bot] <262613592+tenki-reviewer[bot]@users.noreply.github.com>
- Avoid referencing conditions we know are false/true
- Avoid using name of imported package as variable
- Avoid broken (see list item 1) if else statement in `ipservice.go` by refactoring to switch statement
- When assigning a pointer value to a variable along with an error, check that error before referencing that pointer. Thus avoiding de-referencing a nil and causing a panic.
*** This item is the most important ***
- Standard gofmt package sorting + linting; This includes fixing comment starts for go doc
- Explicit non-handling of unhandled errors where appropriate (assigning errs to _ to reduce linter screaming)
- Export ErrExpired in `netcache` package so that we can properly reference it using `errors.Is` instead of using `strings.Contains` against an `error.Error()` value
Vishal Dalwadi