mirror of
https://github.com/NationalSecurityAgency/datawave.git
synced 2026-11-10 03:31:32 -10:00
9dadf7b2c6
* Modified a couple of things and fixed a few tests with the integration merge. * pulling in latest audit changes * pulled in latest audit updates * Fixed metrics version in docker compose * Updated accumulo utils and datawave starter versions. * Updated the audit api and audit starter versions * Updated authorization api version * Updated accumulo api version. * Updated config service with latest changes * updated dictionary and accumulo utils version * Updated hazelcast version * bumped query metric api version * Updated pki for query microservices docker deployment. * Fixing the docker compose bootstrap to avoid duplicate extra hosts issue. * adding latest datawave changes * updated metadata utils * updated docker compose to use the latest version of each service. * Added a cached results spring boot starter and updated associated classes accordingly. * fixing spotbugs error * fixed install-ingest.sh for datawave. * Updates to get bulk results map reduce query working again. * formatting * Updates per a race condition discovered in the kafka variant of the query executor tests. * Update README.md * accumulo service merge from main * fixed a test that broke after merging integration * Added logic from Datawave Principal Login Module to the datawave spring boot starter. * Updated datawave to 5.4.0-SNAPSHOT * Updated ProxiedUserDetails to use dynamic type for newInstance method. * updated audit service to stop intermittent test failures. * Updated the query results publisher and setup the kafka manager to create topics manually. * Added github actions for some submodules * Updated github actions to avoid running out of disk space. * Updated import sorting for microservices using impsort plugin. * fixing merge conflicts * updated metadata utils * Moved Query, QueryImpl to query-api * Removed usage of zookeeper stringutils * added package-info.java for Query, QueryImpl * updated modification service logic to use ProxiedUserDetails. * Removed unnecessary jackson annotations to address enunciate issues * Updated profile ids for submodules * changes to get the github actions working again * attempting to fix github actions * typo * udpated kafka config * removed banned dependency * bumped release version * formatting * fixing docker compose kafka setup * updated deploy poms to work with the latest version of maven * bumped version for some modules * bumped microservice service parent version * Updated with latest changes from main/integration * fixed the composite query logic for the query microservices * updated submodules * Added v1 to the model path * For this branch we need to checkout all submodules when building tests. * Added alerman's run cache settings for docker * Updated github actions to build query microservices correctly * fixed build cache config typo * Updated query service to use pool header, and allow for admin override via parameter * Updated github ci to make sure it runs against the queryMicroservices branch * Updated lookup uuid to accept multiple terms per field * Updated confirm ack configuration for query metric service, and added configuration to make rest metrics updates possible * Added support for long running queries * Added configcheck command line utility * updated next call logic for long running queries * updated query metric service and starter submodules * Added some plugins to the config check service * Updated configcheck tests to trim whitespace * fixing unit tests for config check * Changed from max long running query timeouts to long running query timeout * Updated the query starter submodule * updated submodules * fixed composite query logic test * Refactored query scripts to use common code * Updated quickstart docker-maven-plugin to not cleanup old tagged releases * updated common-utils submodule * Updated LookupService to behave the same as the webservice * Fixed TypedValue (de)serialization, and added a result post processor to handle things like counts * reset docker-compose quickstart to just be accumulo, not web * Changed how the result postprocessor runs to support result aggregation for long running queries * Renamed aggregateResults to reduceResults * Added junit-jupiter engine to datawave core pom * Updated query logic factory to add logic map * Updated EdgeQueryLogicFactory configuration * Removed unused RemoteQueryLogic and cleaned up RemoteEventQueryLogic class definition: * Implemented authorization and query federation for the query microservices * Fixed copy constructor parameter assignment in QueryData * Formatting job fix * Fixing build/formatter issues for latest changes pulled in from integration * Fixed SSDeepIngestQueryTest * Updated usage of remote user operations for query microservices * Fixed QueryPlan constructor * Moved filtering and delegating query logic functionality into datawave-core-query so it can be used by the microservices * formatting * Updated submodules * Updated package names for commons.lang3 classes due to type-utils fix * Moved the AuthorizationsPredicate class to authorization-api * Removed hadoop dependencies from pom * Updated edge configuration to reset param variables before loading new values * Excluded log4j from our maven plugins * Updated in-memory-accumulo javadoc plugin to exclude log4j * Updated microservice-parent maven-dependency-plugin to exclude log4j * Removed infinispan from query starter * Remove validation call from LoadModel * fixed formatting * Updated datawave ingest jar libs * QueryMetricQueryLogic must copy runAsMetricsAdministrator setting in copy constructor * Revert query metric query logic back to it's original state * pr feedback * PR feedback * PR Feedback * PR feedback * PR Feedback * PR feedback * PR feedback * PR Feedback * PR Feedback * PR feedback * PR feedback * PR feedback * pr feedback * PR feedback * pr feedback * pr feedback * pr feedback * pr feedback * pr feedback * pr feedback * PR feedback * PR feedback * pr feedback * pr feedback * Updated in-memory-accumulo commit * in-memory-accumulo * formatting * query starter * pr feedback * pr feedback * pr feedback * pr feedback * pr feedback * pr feedback * pr feedback * Remoted CachedQueryStatus mechanism * pr feedback * pr feedback * pr feedback * pr feedback * pr updates * pr feedback * pr review: found duplicate properties between configurations and the super GenericQueryConfiguration class. Removed redundant copies in ShardQueryConfiguration checkpoint copy constructor * review comments * review comments * Reverted path changes in dictionary service * pr review updates * updated executor to use QueryStatusUpdateUtil * updated query microservice submodule * make github action tests match integration --------- Co-authored-by: Ivan Bella <[email protected]> Co-authored-by: GitHub Actions <[email protected]> Co-authored-by: Bill Oley <[email protected]>
32 lines
1020 B
Bash
Executable File
32 lines
1020 B
Bash
Executable File
#!/bin/bash
|
|
|
|
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
|
|
|
|
source ${SCRIPT_DIR}/common/common.sh
|
|
|
|
createTempPem
|
|
|
|
MODIFICATION_ENDPOINT=https://localhost:9343/modification/v1
|
|
|
|
FOLDER="modification_$(date +%Y%m%d_%I%M%S.%N)"
|
|
|
|
mkdir $FOLDER
|
|
cd $FOLDER
|
|
|
|
echo "$(date): polling modification service for configurations"
|
|
curl -s -D headers_1.txt -k -E ${TMP_PEM} \
|
|
-H "Accept: application/json" \
|
|
${MODIFICATION_ENDPOINT}/listConfigurations -o modificationConfigurationResponse.txt -w '%{http_code}\n'
|
|
|
|
echo "$(date): reloading modification service fields"
|
|
curl -s -D headers_2.txt -k -E ${TMP_PEM} \
|
|
-H "Accept: application/json" \
|
|
${MODIFICATION_ENDPOINT}/reloadCache -o modificationReloadCacheResponse.txt -w '%{http_code}\n'
|
|
|
|
echo "$(date): polling modification service fields"
|
|
curl -s -D headers_2.txt -k -E ${TMP_PEM} \
|
|
-H "Accept: application/json" \
|
|
${MODIFICATION_ENDPOINT}/getMutableFieldList -o modificationFieldListResponse.txt -w '%{http_code}\n'
|
|
|
|
cd ../
|