…s from NationalSecurityAgency/datawave-stack-docker-images
---------
Co-authored-by: Moriarty <[email protected]>
Co-authored-by: Claude Opus 5 <[email protected]>
Updates the read-properties and assert-properties maven plugins to use
Mojo assertions.
This removes the Mojo warning messages seen when using the plugins.
Updates the compiler options to use the release flag which enforces a
tighter java 11 byte code enforcement than using the source and target
flags.
https://maven.apache.org/plugins/maven-compiler-plugin/examples/set-compiler-release.html
Performed minor code quality cleanup:
* Used try-with-resources loops
* Refactored mixed string concat and append usage to only .append
methods with StringBuilder.
* Marked final objects as final
* Removed dep on StringUtils
Add a utility tool that is able to identify tablets that need
compaction. This tool is hosted in the project check-tablet-extents and
can be built as an executable JAR.
Closes#3581
---------
Co-authored-by: Laura Schanno <[email protected]>
This update forces maven to run in offline mode when resolving
environment variables for the bootstrap scripts in the quickstart.
Aside from being a good citizen and not hitting maven central, when
running in the context of the quickstart container, the maven executions
used to populate environment variables in the bootstrap scripts are
incredibly slow...
```
[root@74633f45d549 datawave-quickstart]# time mvn -q -f ${DW_CLOUD_HOME}/docker/pom.xml help:evaluate -DforceStdout -Dexpression=sha512.checksum.wildfly
fcbdff4bc275f478c3bf5f665a83e62468a920e58fcddeaa2710272dd0f1ce3154cdc371d5011763a6be24ae1a5e0bca0218cceea63543edb4b5cf22de60b485
real 0m34.699s
user 0m7.059s
sys 0m0.384s
[root@74633f45d549 datawave-quickstart]# time mvn -o -q -f ${DW_CLOUD_HOME}/docker/pom.xml help:evaluate -DforceStdout -Dexpression=sha512.checksum.wildfly
fcbdff4bc275f478c3bf5f665a83e62468a920e58fcddeaa2710272dd0f1ce3154cdc371d5011763a6be24ae1a5e0bca0218cceea63543edb4b5cf22de60b485
real 0m2.131s
user 0m5.634s
sys 0m0.342s
```
There's a delay when downloading group level plugin metadata. It is
unlikely necssary to do this when resolving a property in a pom to a
value.
```
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/xmlbeans/maven-metadata.xml
Downloading from central: https://repo.maven.apache.org/maven2/com/diffplug/spotless/maven-metadata.xml
Downloading from central: https://repo.maven.apache.org/maven2/gov/nsa/datawave/plugins/maven-metadata.xml
[WARNING] Could not transfer metadata gov.nsa.datawave.plugins/maven-metadata.xml from/to central (https://repo.maven.apache.org/maven2): status code: 429, reason phrase: Too Many Requests (429)
[WARNING] Could not transfer metadata com.diffplug.spotless/maven-metadata.xml from/to central (https://repo.maven.apache.org/maven2): status code: 429, reason phrase: Too Many Requests (429)
[WARNING] Could not transfer metadata org.apache.xmlbeans/maven-metadata.xml from/to central (https://repo.maven.apache.org/maven2): status code: 429, reason phrase: Too Many Requests (429)
```
In some cases, we receive multiple values that get mapped to the same
field name, e.g., CONTENT. When these fields are received independently
in separate ingest passes and then tokenized, we wind up with tokens in
the term frequency portion of the shard table that have offsets that
overlap. This will become more frequent as we implement annotation
indexing.
For example, one CONTENT field value might be CONTENT:"cat mat" and
another value might be "dog bog". In this case, it's possible under
certain circumstances that a query CONTENT:"cat bog" would hit on this
field and require further filtering to determine that it's a false
positive.
This feature adds the ability to employ grouping notation, similar to
how it's used on the event fields, to maintain a distinction between the
contexts within which a token appears as a part of the term frequency
entry. It will involve modifications to ingest and query to preserve
this extra context in the form of dot notation.
Replaces #2812 which was owned by @FineAndDandy so they can be a legit
reviewer for this feature.
---------
Co-authored-by: FineAndDandy <[email protected]>
When attempting to bring up the docker compose environment the
quickstart container sometimes is unable to bring up the web service.
Waiting for the dependent services to come up first seems to resolve
this.
Co-authored-by: Joe Alphonso <[email protected]>
Dan was having github issues so I pushed this for him in case anyone has
questions about this PR
---------
Co-authored-by: d-hwang <[email protected]>
Co-authored-by: Joseph Alphonso <[email protected]>
Currently, when query validation rules are not configured for a
ShardQueryLogic bean, the resulting exception and message is identical
to the default UnsupportedOperationException thrown by the QueryLogic
interface. This is misleading, since the operation is supported, but
additional configuration is required.
Update the ShardQueryLogic to instead throw a BadRequestQueryException
with the error code 412-19 and a message indicating that query
validation rules need to be configured for the underlying logic before
validation can proceed.
Additionally, add the option `-VQ | --validateQuery` to the quickstart
datawaveQuery function to make the command post a query to the validate
endpoint.
---------
Co-authored-by: foster33 <[email protected]>
Co-authored-by: Mr. Patrickus Agbius <[email protected]>
quickstart builds do not benefit from building source and javadoc
remove the dist arg from `DW_DATAWAVE_BUILD_COMMAND` to stop this
Co-authored-by: Joe Alphonso <[email protected]>
renamed `docker.image.prefix` to `docker.registry` to make purpose clear
resolves image push failures
---------
Co-authored-by: Joe Alphonso <[email protected]>
Kevin Toth