NationalSecurityAgency/datawave
1
0 Fork 0
mirror of https://github.com/NationalSecurityAgency/datawave.git synced 2026-11-10 11:41:46 -10:00
Code Issues Packages Projects Releases Wiki Activity
6,190 Commits 1,270 Branches 1,292 Tags
task/replace-sortedmapiterator
Commit Graph
7 Commits
Author SHA1 Message Date
Ivan Bella 246642ff6f Merge submodules (#2654) 
* git subrepo clone [email protected]:NationalSecurityAgency/datawave-type-utils.git core/utils/type-utils

subrepo:
  subdir:   "core/utils/type-utils"
  merged:   "867b89be18"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-type-utils.git"
  branch:   "main"
  commit:   "867b89be18"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-utils.git contrib/datawave-utils

subrepo:
  subdir:   "contrib/datawave-utils"
  merged:   "4348fc36a3"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-utils.git"
  branch:   "main"
  commit:   "4348fc36a3"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-base-rest-responses.git core/base-rest-responses

subrepo:
  subdir:   "core/base-rest-responses"
  merged:   "c5b77e8a35"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-base-rest-responses.git"
  branch:   "main"
  commit:   "c5b77e8a35"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-in-memory-accumulo.git core/in-memory-accumulo

subrepo:
  subdir:   "core/in-memory-accumulo"
  merged:   "8a9d2f46d2"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-in-memory-accumulo.git"
  branch:   "main"
  commit:   "8a9d2f46d2"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-metrics-reporter.git core/metrics-reporter

subrepo:
  subdir:   "core/metrics-reporter"
  merged:   "992378d629"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-metrics-reporter.git"
  branch:   "main"
  commit:   "992378d629"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-accumulo-utils.git core/utils/accumulo-utils

subrepo:
  subdir:   "core/utils/accumulo-utils"
  merged:   "638b3eda97"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-accumulo-utils.git"
  branch:   "main"
  commit:   "638b3eda97"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-common-utils.git core/utils/common-utils

subrepo:
  subdir:   "core/utils/common-utils"
  merged:   "c96ed21342"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-common-utils.git"
  branch:   "main"
  commit:   "c96ed21342"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone --branch=release/version4.0.9 [email protected]:NationalSecurityAgency/datawave-metadata-utils.git core/utils/metadata-utils

subrepo:
  subdir:   "core/utils/metadata-utils"
  merged:   "0d0872a05b"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-metadata-utils.git"
  branch:   "release/version4.0.9"
  commit:   "0d0872a05b"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-parent.git microservices/microservice-parent

subrepo:
  subdir:   "microservices/microservice-parent"
  merged:   "6207c9d657"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-parent.git"
  branch:   "main"
  commit:   "6207c9d657"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-service-parent.git microservices/microservice-service-parent

subrepo:
  subdir:   "microservices/microservice-service-parent"
  merged:   "8064d20ccf"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-service-parent.git"
  branch:   "main"
  commit:   "8064d20ccf"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-accumulo-service.git microservices/services/accumulo

subrepo:
  subdir:   "microservices/services/accumulo"
  merged:   "6d3839d289"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-accumulo-service.git"
  branch:   "main"
  commit:   "6d3839d289"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-audit-service.git microservices/services/audit

subrepo:
  subdir:   "microservices/services/audit"
  merged:   "2714c97b19"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-audit-service.git"
  branch:   "main"
  commit:   "2714c97b19"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-authorization-service.git microservices/services/authorization

subrepo:
  subdir:   "microservices/services/authorization"
  merged:   "292be85b63"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-authorization-service.git"
  branch:   "main"
  commit:   "292be85b63"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-config-service.git microservices/services/config

subrepo:
  subdir:   "microservices/services/config"
  merged:   "fc187c0e4b"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-config-service.git"
  branch:   "main"
  commit:   "fc187c0e4b"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-dictionary-service.git microservices/services/dictionary

subrepo:
  subdir:   "microservices/services/dictionary"
  merged:   "709957100d"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-dictionary-service.git"
  branch:   "main"
  commit:   "709957100d"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-file-provider-service.git microservices/services/file-provider

subrepo:
  subdir:   "microservices/services/file-provider"
  merged:   "27080f3b94"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-file-provider-service.git"
  branch:   "main"
  commit:   "27080f3b94"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-hazelcast-service.git microservices/services/hazelcast

subrepo:
  subdir:   "microservices/services/hazelcast"
  merged:   "8abc2ef7e9"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-hazelcast-service.git"
  branch:   "main"
  commit:   "8abc2ef7e9"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-map-service.git microservices/services/map

subrepo:
  subdir:   "microservices/services/map"
  merged:   "473ec43708"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-map-service.git"
  branch:   "main"
  commit:   "473ec43708"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-mapreduce-query-service.git microservices/services/mapreduce-query

subrepo:
  subdir:   "microservices/services/mapreduce-query"
  merged:   "22af8f93c8"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-mapreduce-query-service.git"
  branch:   "main"
  commit:   "22af8f93c8"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-modification-service.git microservices/services/modification

subrepo:
  subdir:   "microservices/services/modification"
  merged:   "768007f319"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-modification-service.git"
  branch:   "main"
  commit:   "768007f319"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-query-service.git microservices/services/query

subrepo:
  subdir:   "microservices/services/query"
  merged:   "14c65febd3"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-query-service.git"
  branch:   "main"
  commit:   "14c65febd3"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-query-executor-service.git microservices/services/query-executor

subrepo:
  subdir:   "microservices/services/query-executor"
  merged:   "4517c4c29e"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-query-executor-service.git"
  branch:   "main"
  commit:   "4517c4c29e"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-query-metric-service.git microservices/services/query-metric

subrepo:
  subdir:   "microservices/services/query-metric"
  merged:   "9123d49528"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-query-metric-service.git"
  branch:   "main"
  commit:   "9123d49528"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-spring-boot-starter-audit.git microservices/starters/audit

subrepo:
  subdir:   "microservices/starters/audit"
  merged:   "ef18c9e652"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-spring-boot-starter-audit.git"
  branch:   "main"
  commit:   "ef18c9e652"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone [email protected]:NationalSecurityAgency/datawave-spring-boot-starter-cache.git microservices/starters/cache

subrepo:
  subdir:   "microservices/starters/cache"
  merged:   "3019600791"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-spring-boot-starter-cache.git"
  branch:   "main"
  commit:   "3019600791"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-spring-boot-starter-cached-results.git microservices/starters/cached-results

subrepo:
  subdir:   "microservices/starters/cached-results"
  merged:   "b22a3d6a17"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-spring-boot-starter-cached-results.git"
  branch:   "main"
  commit:   "b22a3d6a17"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-spring-boot-starter.git microservices/starters/datawave

subrepo:
  subdir:   "microservices/starters/datawave"
  merged:   "081e0a028a"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-spring-boot-starter.git"
  branch:   "main"
  commit:   "081e0a028a"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-spring-boot-starter-metadata.git microservices/starters/metadata

subrepo:
  subdir:   "microservices/starters/metadata"
  merged:   "28337e6c23"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-spring-boot-starter-metadata.git"
  branch:   "main"
  commit:   "28337e6c23"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-spring-boot-starter-query.git microservices/starters/query

subrepo:
  subdir:   "microservices/starters/query"
  merged:   "ae730eab06"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-spring-boot-starter-query.git"
  branch:   "main"
  commit:   "ae730eab06"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* git subrepo clone (merge) [email protected]:NationalSecurityAgency/datawave-spring-boot-starter-query-metric.git microservices/starters/query-metric

subrepo:
  subdir:   "microservices/starters/query-metric"
  merged:   "fad3ed7859"
upstream:
  origin:   "[email protected]:NationalSecurityAgency/datawave-spring-boot-starter-query-metric.git"
  branch:   "main"
  commit:   "fad3ed7859"
git-subrepo:
  version:  "0.4.9"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "cce3d93"

* Updated profiles to optionally add former submodules into build/test
Updated test workflow to add all former submodules into build/.test
Changed README.md instructions from submodules to subrepo
Updated build readme
* Avoid periodic failures on a bunch of tests (force serial execution etc).
 2025-02-23 13:28:07 -05:00
awrigh11 791183af37 Initial File Provider Service update (#2608) 
* Initial File Provider Service update

* Updated Readme and removed docker dependency

* Update file-provider commit hash
 2024-10-31 11:08:38 -04:00
Whitney O'Meara a2a874116d added the map microservice as a submodule 2024-09-22 14:04:28 +00:00
Whitney O'MearaIvan BellaGitHub ActionsBill Oley
 		9dadf7b2c6 Feature/query microservices (#2192) 
* Modified a couple of things and fixed a few tests with the integration merge.

* pulling in latest audit changes

* pulled in latest audit updates

* Fixed metrics version in docker compose

* Updated accumulo utils and datawave starter versions.

* Updated the audit api and audit starter versions

* Updated authorization api version

* Updated accumulo api version.

* Updated config service with latest changes

* updated dictionary and accumulo utils version

* Updated hazelcast version

* bumped query metric api version

* Updated pki for query microservices docker deployment.

* Fixing the docker compose bootstrap to avoid duplicate extra hosts issue.

* adding latest datawave changes

* updated metadata utils

* updated docker compose to use the latest version of each service.

* Added a cached results spring boot starter and updated associated classes accordingly.

* fixing spotbugs error

* fixed install-ingest.sh for datawave.

* Updates to get bulk results map reduce query working again.

* formatting

* Updates per a race condition discovered in the kafka variant of the
query executor tests.

* Update README.md

* accumulo service merge from main

* fixed a test that broke after merging integration

* Added logic from Datawave Principal Login Module to the datawave spring boot starter.

* Updated datawave to 5.4.0-SNAPSHOT

* Updated ProxiedUserDetails to use dynamic type for newInstance method.

* updated audit service to stop intermittent test failures.

* Updated the query results publisher and setup the kafka manager to create topics manually.

* Added github actions for some submodules

* Updated github actions to avoid running out of disk space.

* Updated import sorting for microservices using impsort plugin.

* fixing merge conflicts

* updated metadata utils

* Moved Query, QueryImpl to query-api

* Removed usage of zookeeper stringutils

* added package-info.java for Query, QueryImpl

* updated modification service logic to use ProxiedUserDetails.

* Removed unnecessary jackson annotations to address enunciate issues

* Updated profile ids for submodules

* changes to get the github actions working again

* attempting to fix github actions

* typo

* udpated kafka config

* removed banned dependency

* bumped release version

* formatting

* fixing docker compose kafka setup

* updated deploy poms to work with the latest version of maven

* bumped version for some modules

* bumped microservice service parent version

* Updated with latest changes from main/integration

* fixed the composite query logic for the query microservices

* updated submodules

* Added v1 to the model path

* For this branch we need to checkout all submodules when building tests.

* Added alerman's run cache settings for docker

* Updated github actions to build query microservices correctly

* fixed build cache config typo

* Updated query service to use pool header, and allow for admin override via parameter

* Updated github ci to make sure it runs against the queryMicroservices branch

* Updated lookup uuid to accept multiple terms per field

* Updated confirm ack configuration for query metric service, and added configuration to make rest metrics updates possible

* Added support for long running queries

* Added configcheck command line utility

* updated next call logic for long running queries

* updated query metric service and starter submodules

* Added some plugins to the config check service

* Updated configcheck tests to trim whitespace

* fixing unit tests for config check

* Changed from max long running query timeouts to long running query
timeout

* Updated the query starter submodule

* updated submodules

* fixed composite query logic test

* Refactored query scripts to use common code

* Updated quickstart docker-maven-plugin to not cleanup old tagged releases

* updated common-utils submodule

* Updated LookupService to behave the same as the webservice

* Fixed TypedValue (de)serialization, and added a result post processor to handle things like counts

* reset docker-compose quickstart to just be accumulo, not web

* Changed how the result postprocessor runs to support result aggregation for long running queries

* Renamed aggregateResults to reduceResults

* Added junit-jupiter engine to datawave core pom

* Updated query logic factory to add logic map

* Updated EdgeQueryLogicFactory configuration

* Removed unused RemoteQueryLogic and cleaned up RemoteEventQueryLogic class definition:

* Implemented authorization and query federation for the query microservices

* Fixed copy constructor parameter assignment in QueryData

* Formatting job fix

* Fixing build/formatter issues for latest changes pulled in from integration

* Fixed SSDeepIngestQueryTest

* Updated usage of remote user operations for query microservices

* Fixed QueryPlan constructor

* Moved filtering and delegating query logic functionality into datawave-core-query so it can be used by the microservices

* formatting

* Updated submodules

* Updated package names for commons.lang3 classes due to type-utils fix

* Moved the AuthorizationsPredicate class to authorization-api

* Removed hadoop dependencies from pom

* Updated edge configuration to reset param variables before loading new values

* Excluded log4j from our maven plugins

* Updated in-memory-accumulo javadoc plugin to exclude log4j

* Updated microservice-parent maven-dependency-plugin to exclude log4j

* Removed infinispan from query starter

* Remove validation call from LoadModel

* fixed formatting

* Updated datawave ingest jar libs

* QueryMetricQueryLogic must copy runAsMetricsAdministrator setting in copy constructor

* Revert query metric query logic back to it's original state

* pr feedback

* PR feedback

* PR Feedback

* PR feedback

* PR Feedback

* PR feedback

* PR feedback

* PR Feedback

* PR Feedback

* PR feedback

* PR feedback

* PR feedback

* pr feedback

* PR feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* PR feedback

* PR feedback

* pr feedback

* pr feedback

* Updated in-memory-accumulo commit

* in-memory-accumulo

* formatting

* query starter

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* Remoted CachedQueryStatus mechanism

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* pr updates

* pr feedback

* pr review: found duplicate properties between configurations and the
super GenericQueryConfiguration class.  Removed redundant copies in
ShardQueryConfiguration checkpoint copy constructor

* review comments

* review comments

* Reverted path changes in dictionary service

* pr review updates

* updated executor to use QueryStatusUpdateUtil

* updated query microservice submodule

* make github action tests match integration

---------

Co-authored-by: Ivan Bella <[email protected]>
Co-authored-by: GitHub Actions <[email protected]>
Co-authored-by: Bill Oley <[email protected]>
 2024-05-20 16:33:34 +00:00
Whitney O'Meara 413b2f87ab removed the branch specification from .gitmodules 2023-03-14 17:23:27 -04:00
Whitney O'Meara b6327b122b Updated branch in .gitmodules. 2023-03-13 17:05:45 -04:00
Whitney O'Meara 11d0b5c29b Added the utils, microservices, and starters as submodules to make it easier to build the various projects 2023-03-13 16:43:54 -04:00