The standalone install adds it in setup-wildfly.sh, but the image ran no
equivalent, so in-VM messaging connections failed authentication and
retried unthrottled, risking an OutOfMemoryError.
Co-authored-by: Claude Opus 5 <[email protected]>
## Configuration
Add the ability to enforce concurrent query limits across a group of
webservers. Zookeeper is used to track active queries and the following
data:
- The query ID
- The user who submitted the query
- The system the query was submitted on
- The query logic the query originated from
Query limit enforcement is done through the `QueryLimiter` class. Given
a user, system, and query logic, it can determine if any of the
following limits have been exceeded:
- The max allowed concurrent queries for the user.
- The max allowed concurrent queries of the query logic for the user.
- The max allowed concurrent queries for the system.
- The max allowed concurrent queries of the query logic for the system.
Limits may be defined and customized on a per-user and per-system basis.
They may also be defined for groups of query logics. The classes
`UserLimitProvider`, `SystemLimitProvider`, and
`QueryLogicGroupLimitProvider` are respectively responsible for
identifying the best limits to enforce for a user, system, and query
logic. They will be initialized in the `QueryLimiter` after providing a
`QueryLimitConfiguration` instance. The following can be configured:
On a system-wide basis:
- The default concurrent user query limit. This applies to the total
number of queries a user may run across all systems. May be overridden
per user.
- The default concurrent system query limit. Primarily to avoid a system
getting overloaded. May be overridden per system.
On a per-system basis:
- The system name/ids the configuration targets. Regex matching is
supported. Pattern uniqueness is enforced.
- The concurrent system query limit. Overrides the system-wide value.
- Whether queries submitted to the system count towards a user's
concurrent query total. This is always true unless specified otherwise.
- The concurrent system query limit for different query logic groups.
Regex matching against group names is supported. Pattern uniqueness is
enforced.
On a per-user basis:
- The user DN.
- The user's concurrent query limit. Overrides the system-wide
configuration.
- The user's concurrent query limit for different query logic groups.
Regex matching against group names is supported.
On a per-query-logic-group basis:
- The group name.
- The query logics included in the group. Regex matching is supported.
Pattern uniqueness is enforced.
- The default concurrent user query limit. This applies to the total
concurrent queries a user may run that originate from a query logic in
the group across all systems.
Given the possibilities for exact matches, partial regex matches, and
wildcard regex matches, the determination of the best limit to use for
any particular system, query logic, or query logic group is done by
sorting matches into the following 'matching buckets' (in best-match
priority):
1. Exact match: We attempt to find an exact match first and use the
associated limit.
2. Partial regex (non-wildcard-only): If we cannot find an exact match,
then we attempt to find all partial matches, and see if any of their
limits are met.
3. Wildcard-only regex: In the case of no exact or partial matches, we
use the wildcard match with the lowest limit.
## Implementation
Checking limits and marking as active/inactive is done through the
`QueryLimiter` class. The three main methods to know are:
- `QueryLimiter.checkForLimits()`
- `QueryLimiter.countQueryTowardsLimits()`
- `QueryLimiter.stopCountingQueryTowardsLimits()`
When a query is marked as active via
`QueryLimiter.countQueryTowardsLimits()`, it will delegate to the
ActiveQueryTracker, which will in turn create nodes in Zookeeper under
the `ActiveQueries` namespace. When `ActiveQueryTracker.trackQuery()` is
called, the following nodes are created:
```
# Container nodes
/users/<userDn>/<queryLogic> # Only for queries on systems that count towards user limit
/systems/<system>/<queryLogic>
/distinctQueryLogics/<queryLogic> [Only created if it does not exist.]
# Ephemeral nodes. These will auto-delete themselves if their associated Zookeeper connection ever goes down.
/users/<userDn>/<queryLogic>/<queryId> # Only for queries on systems that count towards user limit
/systems/<system>/<queryLogic>/<queryId>
```
`ActiveQueryTracker.trackQuery()` will return a `QueryHeartbeat` that
contain a list of `PersistentNode` (provided by the Apache Curator
library) wrappers around the ephemeral nodes listed above. The
`QueryHeartbeat` will maintain the connection to Zookeeper and attempt
to keep the ephemeral nodes present in Zookeeper until
`QueryHeartbeat.stop()` is called. If `QueryHeartbeat.stop()` is called,
or the webserver crashes, the ephemeral nodes will automatically be
deleted by Zookeeper.
The following error codes have been added:
```
412-20 - Concurrent query limit exceeded
500-164 - Error checking concurrent query limits
```
Closes#3100
---------
Co-authored-by: Bill Oley <[email protected]>
Co-authored-by: foster33 <[email protected]>
Co-authored-by: Ivan Bella <[email protected]>
Use correct slf4j impl in wildfly and services, update versions of
type-utils, metadata-utils, common-utils, accumulo-utils,
base-rest-responses in microservice starters, apis, services to track
datawave version
The use of these dependencies under microservices (starters, services,
api) should track the datawave version instead of the version number
from when they were in separate repositories
Because the prior versions used a different groupId, we have the case
where multiple versions of the same artifact are being included in some
microservices.
For instance, this caused a serialization conflict once the
serializationId in BaseType (type-utils) changed in datawave version
7.30.0. When both versions are present on both sides of a serialization,
whether a serialization exception occurs is indeterminate.
gov.nsa.datawave.core:accumulo-utils
gov.nsa.datawave.core:type-utils
gov.nsa.datawave.core:metadata-utils
gov.nsa.datawave.core:common-utils
gov.nsa.datawave.core:base-rest-responses
After this PR is merged, the following should be done to make the
starters, apis, modules, and services depend on released version of each
other.
Release datawave-microservice-service-parent:5.0.8
In all modules with parent datawave-microservice-service-parent, update
to datawave-microservice-service-parent:5.0.8
Release authorization-api:4.0.2
In starter-datawave update to authorization-api:4.0.2
Release starter-datawave:4.0.6
Release query-metric-api:4.1.3
Release query-api:1.0.1
Release accumulo-api:4.0.1
Release modification-api:1.0.1
Release dictionary-api:4.0.6
In hazelcast-common update to starter-datawave:4.0.6
Release hazelcast-common:4.0.3
In hazelcast-client update to hazelcast-common:4.0.3
In starter-audit update to starter-datawave:4.0.6,
authorization-api:4.0.2
In starter-metadata update to starter-datawave:4.0.6
In starter-query-metric update to starter-datawave:4.0.6,
query-metric-api:4.1.3, authorization-api:4.0.2
Release hazelcast-client:4.0.3
Release starter-audit:4.0.4
Release starter-metadata:3.0.3
Release starter-query-metric:3.0.4
In starter-query update to starter-datawave:4.0.6, query-api:1.0.1,
hazelcast-client:4.0.3, starter-metadata:3.0.3,
starter-query-metric:3.0.4
Release starter-query:1.0.9
In starter-cached-query update to starter-datawave:4.0.6,
hazelcast-client:4.0.3, starter-audit:4.0.4, starter-query:1.0.9
Release starter-cached-query:1.0.11
In mapreduce-query-core-job update to starter-query:1.0.9
Release mapreduce-query-core-job:1.0.5
In query-executor-api update to starter-datawave:4.0.6,
starter-query:1.0.9
Release query-executor-api:1.0.8
In accumulo-service update to starter-datawave:4.0.6, accumulo-api:4.0.1
In audit-service update to starter-datawave:4.0.6,
hazelcast-client:4.0.3
In authorization-service update to starter-datawave:4.0.6,
hazelcast-client:4.0.3, authorization-api:4.0.2
In config-service update to starter-datawave:4.0.6
In dictionary-service update to starter-datawave:4.0.6,
dictionary-api:4.0.6, starter-metadata:3.0.3
In file-provider-service update to starter-datawave:4.0.6
In hazelcast-service update to hazelcast-common:4.0.3
In mapreduce-query-service update to mapreduce-query-core-job:1.0.5,
starter-query:1.0.9
In modification-service update to starter-metadata:3.0.3,
query-api:1.0.1, modification-api:1.0.1
In query-service update to starter-datawave:4.0.6, query-api:1.0.1,
starter-cached-results:1.0.11, starter-query:1.0.9
In query-executor-service update to starter-query:1.0.9,
query-executor-api:1.0.8
In query-metric-service update to starter-datawave:4.0.6,
starter-metadata:3.0.3, hazelcast-common:4.0.3
Release accumulo-service:4.0.3
Release audit-service:4.0.3
Release authorization-service:4.0.5
Release config-service:4.0.2
Release dictionary-service:4.0.10
Release file-provider-service:1.0.1
Release hazelcast-service:4.0.3
Release mapreduce-query-service:1.0.5
Release modification-service:1.0.3
Release query-service:1.0.12
Release query-executor-service:1.0.10
Release query-metric-service:4.1.9
renamed `docker.image.prefix` to `docker.registry` to make purpose clear
resolves image push failures
---------
Co-authored-by: Joe Alphonso <[email protected]>
* cleanup quickstart and jkube docker configuration
set compose prefix
stop boostrapping maven twice
add default back for microservices since they do not load the properties files
add latest tag to compose tests
# Conflicts:
# contrib/datawave-quickstart/docker/docker-build.sh
# contrib/datawave-quickstart/docker/pom.xml
# Conflicts:
# microservices/microservice-parent/pom.xml
* remove cdn download from pom and set file urls to local paths
* fix file paths
* testing
# Conflicts:
# microservices/microservice-service-parent/pom.xml
* cleanup and add quickstart web tests back to CI
* update quickstart ci test
* testing fixes
* fix bug
* address compose warning
* bugfix
* dev profile clean up and docker-entrypoint.sh fix
* bootstrap only when install is called
* kill process when there is a fatal error
* update maven to a version we actually have
* install maven via os pkg mgr to start
* move quickstart jdk check to the install.sh scripts
* copy maven from latest maven-3 image at container build time
* remove maven download and bootstrap code
* fix returns and exits so we terminate quickstart build upon first fatal error
* remove quickstart ci job
* remove broken maven download from inside quickstart
* remove obsolete arg
* bump microservice-parent version in child poms
* bump microservice-service-parent version in child poms
* use mvn --settings arg
* rename vars and add to pom for warehouse docker image
* add web tests to CI
* reenable compose snapshot ci job
* add missed mvn --settings arg
* testing
* fix bug
* remove testing changes
* fix web tests
* fixes so users can use scripts outside of the container
---------
Co-authored-by: Joe Alphonso <[email protected]>
Kevin Toth