## Which issue does this PR close?
- part of https://github.com/apache/datafusion/issues/17207
## Rationale for this change
As suggested by @BlakeOrth in
https://github.com/apache/datafusion/pull/18045#issuecomment-3403692516
here is an attempt to improve the output of datafusion object store
trace profiling:
## What changes are included in this PR?
Update the output format when `\object_store_profiling summary` is set
Current format (on main, before this PR):
```sql
Summaries:
Get
count: 2
duration min: 0.024603s
duration max: 0.031946s
duration avg: 0.028274s
size min: 8 B
size max: 34322 B
size avg: 17165 B
size sum: 34330 B
```
New format (after this PR):
```sql
DataFusion CLI v50.2.0
> \object_store_profiling summary
ObjectStore Profile mode set to Summary
> select count(*) from 'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet';
+----------+
| count(*) |
+----------+
| 1000000 |
+----------+
1 row(s) fetched.
Elapsed 6.754 seconds.
Object Store Profiling
Instrumented Object Store: instrument_mode: Summary, inner: HttpStore
Summaries:
+-----------+----------+-----------+-----------+-----------+-----------+-------+
| Operation | Metric | min | max | avg | sum | count |
+-----------+----------+-----------+-----------+-----------+-----------+-------+
| Get | duration | 0.031645s | 0.047780s | 0.039713s | 0.079425s | 2 |
| Get | size | 8 B | 34322 B | 17165 B | 34330 B | 2 |
+-----------+----------+-----------+-----------+-----------+-----------+-------+
```
## Are these changes tested?
Yes
## Are there any user-facing changes?
Nicer datafusion-cli output
- Adds the ability for a user to choose a summary only output for an
instrumented object store when using the CLI
- The existing "enabled" setting that displays both a summary and a
detailed usage for each object store call has been renamed to `Trace`
to improve clarity
- Adds additional test cases for summary only and modifies existing
tests to use trace
- Updates user guide docs to reflect the CLI flag and command line
changes
* Adds Object Store Profiling options/commands to CLI
- Adds a CLI option and command to datafusion-cli to enable or disabled
object store profiling
- Integrates the command with the instrumented object stores to allow
the user input to change the mode of the instrumented stores
- Adds tests to exercise the expected behavior of the commands
- Adds user docs for the commands/CLI options
- Updates visibility of `InstrumentedObjectStore` now that it needs to
be interacted with outside of its module
* Improves InstrumentedObjectStoreRegistry ergonomics
- Adds better methods to build an InstrumentedObjectStoreRegistry to
reduce code duplication in common usage
- Enhances test success criteria
- Normalizes method names
* feat: Add the ability to review the contents of the Metadata Cache
* Remove e_tag from test_metadata_cache
* Add entry in the user doc about this function
* Change type to UInt64
* Fix prettier
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Andrew Lamb