## Summary
- Replaces all 120 uses of `datawave.util.TableName` (deprecated since
7.40.0, `forRemoval=true`) with the new
`datawave.table.constants.TableName` class in `core/tables`
- Adds an explicit `datawave-core-tables` dependency to
`core/query/pom.xml` — the only affected module that didn't already have
it transitively
## Test plan
- [x] Formatting build passes (`mvn clean install -DskipTests
-DskipITs`)
- [x] Full test suite passes on CI
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
update accumulo-utils to use a POJO for storing markings and then
updated the rest of the codebase that referenced it.
replaced some of the ColumnVisibility usage with AccessExpression to
improve performance.
---------
Co-authored-by: Joe Alphonso <[email protected]>
Resolves race condition by adding CountDownLatch and removing the fixed
sleep.
Fixing intermittent test failures enables us to run tests faster with
`-T1.0C` more reliably.
* Adds KeywordUUIDQuery that pairs id lookup with keyword extraction.
* Creates a chained query logic that first does lookup uuid, followed by document keyword extraction.
* Adds the ability to generate a word cloud across all keyword results, currently stored in messages portion of the response.
* Adds Datawave TagCloudResponse and supporting classes.
* Adds two endpoints to /DataWave/Query - /extractKeywords and /generateTagCloud - each take a POST request and return keywords from individual documents or consolidated tag coluds, respectively.
* Adds response type for /generateTagCloud, the TagCloudResponseBase and accompanying data classes.
* Updates chained query and improves serialization of objects returned from iterators.
* Addressed formatting, review comments, and completed bugfix.
* Implement query validation endpoint
Implement the query/{logicName}/validate endpoint. This feature supports
the ability to configure validation rules that will validate LUCENE and
JEXL queries against a number of criteria and provide meaningful
feedback to customers.
Closes#2585
---------
Co-authored-by: Seth Smucker <[email protected]>
Co-authored-by: Ivan Bella <[email protected]>
* Initial commit for scan hint overrides for shard index methods
* Update mocks for copy constructor test
* SessionOptions has methods to apply scan hints and scan consistency options, fixed several instances where new session options were created and scan hints were not applied
* Refactor ScannerFactory to support setting execution hints by table name or functional name
* clean up unused variable, import
* add reserved 'expansion' hint key, refactor ScannerFactory.applyConfigs to support falling back to secondary hint key
* Enrich 'i' and 'ri' rows in metadata table with event date
Modify the generation of 'i' (indexed rows) and 'ri' (reverse indexed
rows) in the metadata table such that the column qualifier contains the
event date. This is required as a first step to support efforts for
issue #825 so that we can identify dates when an event was ingested and
included in a frequency count for an associated 'f' row, but was not
indexed.
* Add counts to 'i' and 'ri' rows
* Initial federated query planner implementation
* code formatting
* Fixed issues with FederatedQueryIterable
* Fix test failures
* Fix failing tests
* Additional test fixes
* pr feedback
* Use new MetadataHelper function version
* Extract fields to filter index holes
* Correct logic for determining sub date ranges
* Remove unnecessary check
* code formatting
* Add check for null query model
* Limit config arg to function scope
* Update metadata-utils submodule commit
* code formatting
* Fix failing tests
* Additional test fixes
* Ensure all original tests pass
* Add federated planner tests and chained schedulers
* pr feedback
* metadata-utils 3.0.3 tag
* Fixed the index hole data ingest to set appropriate time stamps on the keys
Removed some of the code which I believe was trying to diagnose the test issues
* Updated applyModel to use the passed in script
* Remove unneeded changes
* Make FederatedQueryPlanner the default
* Restore original log4j.properties
* code formatting
* Fix QueryPlanTest
* Updated to test with teardown
* Test debugging edits
* Updated formatting
* Concatenate sub-plans
* Make FederatedQueryPlanner implement Cloneable
* code formatting
* * Updated with metadata-utils 4.0.5 (index markers and avoid non-indexed fields for holes)
* Fixed test cases with correct responses and periodic failing test cases
* Updated AncestorQueryLogic to handle federate query planner
* * Allow subclasses of ShardQueryConfiguration
* Updated to throw a NoResultsException for am empty query.
* import reorg
* Updated to avoid expanding unfielded if disabled, and to assume no index holes if no query fields.
* Add tests for default query planner with ne and not-eq
* Revert changes to test data format
* Revert changes to log4j.properties
* Ensure query plan updated after any exception type
* Revert all changes to test data format
---------
Co-authored-by: Ivan Bella <[email protected]>
Co-authored-by: hgklohr <[email protected]>
The query metrics API expects QueryExceptions to be found in the cause
of the original exception passed to BaseQueryMetric.setError(Throwable).
In the case of CompositeQueryLogic, due to the additional wrapping of
exceptions passed from its defined internal logics in a
CompositeLogicException, query exceptions were not in the expected place
in the stack hierarchy.
Updateid the CompositeLogicException to identify any QueryExceptions
present in the stacks of exception arguments, and ensure that they are
either already a QueryException, or that if they contain a
QueryException in the stack, that they are subsequently wrapped in a
CompositeRaisedQueryException with a copy of the desired error code.
* Add support for setting scan consistency level on a query logic
Additional updates
Update a few instances that still relied on old constructors
Only update scanner factory hints and consistency if set on logic config
* merge request feedback
* clarify names are per-table
* Modified a couple of things and fixed a few tests with the integration merge.
* pulling in latest audit changes
* pulled in latest audit updates
* Fixed metrics version in docker compose
* Updated accumulo utils and datawave starter versions.
* Updated the audit api and audit starter versions
* Updated authorization api version
* Updated accumulo api version.
* Updated config service with latest changes
* updated dictionary and accumulo utils version
* Updated hazelcast version
* bumped query metric api version
* Updated pki for query microservices docker deployment.
* Fixing the docker compose bootstrap to avoid duplicate extra hosts issue.
* adding latest datawave changes
* updated metadata utils
* updated docker compose to use the latest version of each service.
* Added a cached results spring boot starter and updated associated classes accordingly.
* fixing spotbugs error
* fixed install-ingest.sh for datawave.
* Updates to get bulk results map reduce query working again.
* formatting
* Updates per a race condition discovered in the kafka variant of the
query executor tests.
* Update README.md
* accumulo service merge from main
* fixed a test that broke after merging integration
* Added logic from Datawave Principal Login Module to the datawave spring boot starter.
* Updated datawave to 5.4.0-SNAPSHOT
* Updated ProxiedUserDetails to use dynamic type for newInstance method.
* updated audit service to stop intermittent test failures.
* Updated the query results publisher and setup the kafka manager to create topics manually.
* Added github actions for some submodules
* Updated github actions to avoid running out of disk space.
* Updated import sorting for microservices using impsort plugin.
* fixing merge conflicts
* updated metadata utils
* Moved Query, QueryImpl to query-api
* Removed usage of zookeeper stringutils
* added package-info.java for Query, QueryImpl
* updated modification service logic to use ProxiedUserDetails.
* Removed unnecessary jackson annotations to address enunciate issues
* Updated profile ids for submodules
* changes to get the github actions working again
* attempting to fix github actions
* typo
* udpated kafka config
* removed banned dependency
* bumped release version
* formatting
* fixing docker compose kafka setup
* updated deploy poms to work with the latest version of maven
* bumped version for some modules
* bumped microservice service parent version
* Updated with latest changes from main/integration
* fixed the composite query logic for the query microservices
* updated submodules
* Added v1 to the model path
* For this branch we need to checkout all submodules when building tests.
* Added alerman's run cache settings for docker
* Updated github actions to build query microservices correctly
* fixed build cache config typo
* Updated query service to use pool header, and allow for admin override via parameter
* Updated github ci to make sure it runs against the queryMicroservices branch
* Updated lookup uuid to accept multiple terms per field
* Updated confirm ack configuration for query metric service, and added configuration to make rest metrics updates possible
* Added support for long running queries
* Added configcheck command line utility
* updated next call logic for long running queries
* updated query metric service and starter submodules
* Added some plugins to the config check service
* Updated configcheck tests to trim whitespace
* fixing unit tests for config check
* Changed from max long running query timeouts to long running query
timeout
* Updated the query starter submodule
* updated submodules
* fixed composite query logic test
* Refactored query scripts to use common code
* Updated quickstart docker-maven-plugin to not cleanup old tagged releases
* updated common-utils submodule
* Updated LookupService to behave the same as the webservice
* Fixed TypedValue (de)serialization, and added a result post processor to handle things like counts
* reset docker-compose quickstart to just be accumulo, not web
* Changed how the result postprocessor runs to support result aggregation for long running queries
* Renamed aggregateResults to reduceResults
* Added junit-jupiter engine to datawave core pom
* Updated query logic factory to add logic map
* Updated EdgeQueryLogicFactory configuration
* Removed unused RemoteQueryLogic and cleaned up RemoteEventQueryLogic class definition:
* Implemented authorization and query federation for the query microservices
* Fixed copy constructor parameter assignment in QueryData
* Formatting job fix
* Fixing build/formatter issues for latest changes pulled in from integration
* Fixed SSDeepIngestQueryTest
* Updated usage of remote user operations for query microservices
* Fixed QueryPlan constructor
* Moved filtering and delegating query logic functionality into datawave-core-query so it can be used by the microservices
* formatting
* Updated submodules
* Updated package names for commons.lang3 classes due to type-utils fix
* Moved the AuthorizationsPredicate class to authorization-api
* Removed hadoop dependencies from pom
* Updated edge configuration to reset param variables before loading new values
* Excluded log4j from our maven plugins
* Updated in-memory-accumulo javadoc plugin to exclude log4j
* Updated microservice-parent maven-dependency-plugin to exclude log4j
* Removed infinispan from query starter
* Remove validation call from LoadModel
* fixed formatting
* Updated datawave ingest jar libs
* QueryMetricQueryLogic must copy runAsMetricsAdministrator setting in copy constructor
* Revert query metric query logic back to it's original state
* pr feedback
* PR feedback
* PR Feedback
* PR feedback
* PR Feedback
* PR feedback
* PR feedback
* PR Feedback
* PR Feedback
* PR feedback
* PR feedback
* PR feedback
* pr feedback
* PR feedback
* pr feedback
* pr feedback
* pr feedback
* pr feedback
* pr feedback
* pr feedback
* PR feedback
* PR feedback
* pr feedback
* pr feedback
* Updated in-memory-accumulo commit
* in-memory-accumulo
* formatting
* query starter
* pr feedback
* pr feedback
* pr feedback
* pr feedback
* pr feedback
* pr feedback
* pr feedback
* Remoted CachedQueryStatus mechanism
* pr feedback
* pr feedback
* pr feedback
* pr feedback
* pr updates
* pr feedback
* pr review: found duplicate properties between configurations and the
super GenericQueryConfiguration class. Removed redundant copies in
ShardQueryConfiguration checkpoint copy constructor
* review comments
* review comments
* Reverted path changes in dictionary service
* pr review updates
* updated executor to use QueryStatusUpdateUtil
* updated query microservice submodule
* make github action tests match integration
---------
Co-authored-by: Ivan Bella <[email protected]>
Co-authored-by: GitHub Actions <[email protected]>
Co-authored-by: Bill Oley <[email protected]>
Moriarty