mirror of https://github.com/langchain-ai/datafusion.git synced 2026-07-19 15:24:21 -04:00

Files

T

Andrew Lamb 5a074ea4c8 Improve datafusion-cli object store profiling summary display (#18085 )

## Which issue does this PR close?

- part of https://github.com/apache/datafusion/issues/17207

## Rationale for this change

As suggested by @BlakeOrth in
https://github.com/apache/datafusion/pull/18045#issuecomment-3403692516
here is an attempt to improve the output of datafusion object store
trace profiling:

## What changes are included in this PR?

Update the output format when `\object_store_profiling summary` is set

Current format (on main, before this PR):
```sql
Summaries:
Get
count: 2
duration min: 0.024603s
duration max: 0.031946s
duration avg: 0.028274s
size min: 8 B
size max: 34322 B
size avg: 17165 B
size sum: 34330 B
```


New format (after this PR):

```sql
DataFusion CLI v50.2.0
> \object_store_profiling summary
ObjectStore Profile mode set to Summary
> select count(*) from 'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet';
+----------+
| count(*) |
+----------+
| 1000000  |
+----------+
1 row(s) fetched.
Elapsed 6.754 seconds.

Object Store Profiling
Instrumented Object Store: instrument_mode: Summary, inner: HttpStore
Summaries:
+-----------+----------+-----------+-----------+-----------+-----------+-------+
| Operation | Metric   | min       | max       | avg       | sum       | count |
+-----------+----------+-----------+-----------+-----------+-----------+-------+
| Get       | duration | 0.031645s | 0.047780s | 0.039713s | 0.079425s | 2     |
| Get       | size     | 8 B       | 34322 B   | 17165 B   | 34330 B   | 2     |
+-----------+----------+-----------+-----------+-----------+-----------+-------+
```



## Are these changes tested?
Yes
## Are there any user-facing changes?
Nicer datafusion-cli output

2025-10-17 21:10:13 +00:00

examples

Adds Object Store Profiling options/commands to CLI (#18004 )

2025-10-10 21:28:32 +00:00

src

Improve datafusion-cli object store profiling summary display (#18085 )

2025-10-17 21:10:13 +00:00

tests

Improve datafusion-cli object store profiling summary display (#18085 )

2025-10-17 21:10:13 +00:00

Cargo.toml

refactor: add dialect enum (#18043 )