langchain-ai/datafusion
1
0
Fork 0
mirror of https://github.com/langchain-ai/datafusion.git synced 2026-07-01 21:24:06 -04:00
Code Issues 3 Packages Projects Releases Wiki Activity
12,768 Commits 4 Branches 0 Tags
main
Commit Graph

12768 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
dependabot[bot] bd31db9d16 chore(deps): bump sysinfo from 0.38.1 to 0.38.2 (#20411) 
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.38.1
to 0.38.2.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/GuillaumeGomez/sysinfo/blob/main/CHANGELOG.md">sysinfo's
changelog</a>.</em></p>
<blockquote>
<h1>0.38.2</h1>
<ul>
<li>Implement <code>Error</code> trait for
<code>MacAddrFromStrError</code>, <code>IpNetworkFromStrError</code> and
<code>KillError</code> types.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/e3e319f11b06b3a7db220f88eedac7904e0d6580"><code>e3e319f</code></a>
Merge pull request <a
href="https://redirect.github.com/GuillaumeGomez/sysinfo/issues/1626">#1626</a>
from GuillaumeGomez/update</li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/6673f0453ef7475781a81e3dab8e5492741d1580"><code>6673f04</code></a>
Update crate version to <code>0.38.2</code></li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/4239c851afcc55e7bfd4e7fbacb6404c51c26f88"><code>4239c85</code></a>
Update CHANGELOG for <code>0.38.2</code></li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/eb9160337977f5b9c6250741b376c5f3c114129d"><code>eb91603</code></a>
Merge pull request <a
href="https://redirect.github.com/GuillaumeGomez/sysinfo/issues/1625">#1625</a>
from GuillaumeGomez/error</li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/8aa5200e062dc62bbb92ab8e963151deb7099465"><code>8aa5200</code></a>
Implement <code>Error</code> trait for error types</li>
<li>See full diff in <a
href="https://github.com/GuillaumeGomez/sysinfo/compare/v0.38.1...v0.38.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sysinfo&package-manager=cargo&previous-version=0.38.1&new-version=0.38.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-17 17:35:40 +00:00
Peter L 6798dff0d7 Fix try_shrink not freeing back to pool (#20382) 
## Which issue does this PR close?

No issue raised

## Rationale for this change

`try_shrink` never released any memory from the pool, and so would cause
a "memory leak"

## What changes are included in this PR?

A few changes to `MemoryReservation`

## Are these changes tested?

Yes & new test has been added

## Are there any user-facing changes?

Nope
 2026-02-17 14:28:52 +00:00
Andrew Lamb 132b04369d Fix incorrect SortExec removal before AggregateExec (option 2) (#20247) 
## Which issue does this PR close?

- Fixes https://github.com/apache/datafusion/issues/20244

This is an alternatative to 
- https://github.com/apache/datafusion/pull/20245

## Rationale for this change

Wrong answers bug was exposed by
https://github.com/apache/datafusion/pull/19287 in 52. See
https://github.com/apache/datafusion/issues/20244 and backstory here
- https://github.com/apache/datafusion/pull/19287#discussion_r2789573897

## What changes are included in this PR?

Fix the bug by properly implemnting 

## Are these changes tested?

Yes, a new test is added

## Are there any user-facing changes?

A bug is fixed

---------

Co-authored-by: Mustafa Akur <akurmustafa@gmail.com>
 2026-02-16 22:00:54 +00:00
notashes b0566c57cc fix: update filter predicates for min/max aggregates only if bounds change (#20380) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Part of #20324 (dynamic filter update overhead for aggregates)

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

Right now `AggregateStream::maybe_update_dyn_filter()` unconditionally
updates filter predicates after every batch even without any change.
Which triggers predicate rebuilds for file pruners even though we
converge towards the bounds quite early on.

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

We only conditionally update filter predicates if the bounds change.

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

The existing test suits pass. The existing clickbench regression shows
virtually no change.

<details>
  <summary>Benchmark Results</summary>
The existing benchmarks don't reproduce this as it gets the data from
stats. So we can use a trick like `"EventDate" + 0` to force full table
scan.
 
  ```sql
  SET datafusion.execution.parquet.binary_as_string = true;
  SET datafusion.execution.parquet.pushdown_filters = false;

CREATE EXTERNAL TABLE hits STORED AS PARQUET LOCATION
'benchmarks/data/hits_partitioned';
CREATE VIEW hits_view AS SELECT "EventDate" + 0 AS "EventDate" FROM
hits;

  -- Warmup
  SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view;

  -- dyn_off x5: hard-coded filter, dynamic filter disabled
SET datafusion.optimizer.enable_aggregate_dynamic_filter_pushdown =
false;
SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view WHERE
"EventDate" > 0 AND "EventDate" < 99999999;
SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view WHERE
"EventDate" > 0 AND "EventDate" < 99999999;
SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view WHERE
"EventDate" > 0 AND "EventDate" < 99999999;
SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view WHERE
"EventDate" > 0 AND "EventDate" < 99999999;
SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view WHERE
"EventDate" > 0 AND "EventDate" < 99999999;

  -- dyn_on x5: dynamic filter enabled
SET datafusion.optimizer.enable_aggregate_dynamic_filter_pushdown =
true;
  SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view;
  SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view;
  SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view;
  SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view;
  SELECT MIN("EventDate"), MAX("EventDate") FROM hits_view;
  ```

  ### Before (baseline)

  | Iteration | dyn_off | dyn_on |
  |-----------|---------|--------|
  | 1 | 39ms | 44ms |
  | 2 | 39ms | 44ms |
  | 3 | 40ms | 44ms |
  | 4 | 39ms | 45ms |
  | 5 | 38ms | 45ms |
  | **Avg** | **39.0ms** | **44.4ms** |

We can see that that it's a little slow to execute with dynamic filters
on.

  ### After (this PR)

  | Iteration | dyn_off | dyn_on |
  |-----------|---------|--------|
  | 1 | 40ms | 33ms |
  | 2 | 41ms | 32ms |
  | 3 | 40ms | 32ms |
  | 4 | 42ms | 32ms |
  | 5 | 39ms | 31ms |
  | **Avg** | **40.4ms** | **32.0ms** |

  And we get approx 20% boost with the patch (in my local setup)

  </details>

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

---------

Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
 2026-02-16 15:18:09 +00:00
Jack Kleeman d47bd599ee Support parent dynamic filters for more join types (#20192) 
## Which issue does this PR close?

Solves the simplest possible subset of #16973

## Rationale for this change

Parent dynamic filters can be assessed for pushdown in much the same way
as parent static filters. The essence of it is that you can push a
filter into the preserved side if the filter only refers to preserved
side columns. This PR makes no attempt to cover other cases (eg non
preserved side filters or the self filters that hashjoin creates)

## What changes are included in this PR?

1. Allow a subset of parent dynamic filters for all join types except
full depending on lr_is_preserved and which columns they refer to
2. Sql logic tests for the above
3. New helper `ChildFilterDescription::all_unsupported` and
`ChildFilterDescription::from_child_with_allowed_columns`
4. `FilterColumnChecker` gets a sibling `FilterColumnIndexChecker` which
also needs to look at indices, not just column names, as the same name
can be on both sides of a join. We don't use this for other call sites
of FilterColumnChecker as we often use the checker before a expr remap
(so indices aren't yet reliable anyway)

## Are these changes tested?

Yes

## Are there any user-facing changes?

No
 2026-02-16 11:39:51 +00:00
Adrian Garcia Badaracco 53b0ffb93d fix: validate inter-file ordering in eq_properties() (#20329) 
## Summary

Discovered this bug while working on #19724.

TLDR: just because the files themselves are sorted doesn't mean the
partition streams are sorted.

- **`eq_properties()` in `FileScanConfig` blindly trusted
`output_ordering`** (set from Parquet `sorting_columns` metadata)
without verifying that files within a group are in the correct
inter-file order
- `EnforceSorting` then removed `SortExec` based on this unvalidated
ordering, producing **wrong results** when filesystem order didn't match
data order
- Added `validated_output_ordering()` that filters orderings using
`MinMaxStatistics::new_from_files()` + `is_sorted()` to verify
inter-file sort order before reporting them to the optimizer

## Changes

### `datafusion/datasource/src/file_scan_config.rs`
- Added `validated_output_ordering()` method on `FileScanConfig` that
validates each output ordering against actual file group statistics
- Changed `eq_properties()` to call `self.validated_output_ordering()`
instead of `self.output_ordering.clone()`

### `datafusion/sqllogictest/test_files/sort_pushdown.slt`
Added 8 new regression tests (Tests 4-11):

| Test | Scenario | Key assertion |
|------|----------|---------------|
| **4** | Reversed filesystem order (inferred ordering) | SortExec
retained — wrong inter-file order detected |
| **5** | Overlapping file ranges (inferred ordering) | SortExec
retained — overlapping ranges detected |
| **6** | `WITH ORDER` + reversed filesystem order | SortExec retained
despite explicit ordering |
| **7** | Correctly ordered multi-file group (positive) | SortExec
eliminated — validation passes |
| **8** | DESC ordering with wrong inter-file DESC order | SortExec
retained for DESC direction |
| **9** | Multi-column sort key (overlapping vs non-overlapping) |
Conservative rejection with overlapping stats; passes with clean
boundaries |
| **10** | Correctly ordered + `WITH ORDER` (positive) | SortExec
eliminated — both ordering and stats agree |
| **11** | Multiple partitions (one file per group) |
`SortPreservingMergeExec` merges; no per-partition sort needed |

## Test plan

- [x] `cargo test --test sqllogictests -- sort_pushdown` — all new +
existing tests pass
- [x] `cargo test -p datafusion-datasource` — 97 unit tests + 6 doc
tests pass
- [x] Existing Test 1 (single-file sort pushdown with `WITH ORDER`)
still eliminates SortExec (no regression)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-14 19:04:01 +00:00
Subham Singhal 98cc75361f Adds support for ANSI mode in negative function (#20189) 
## Which issue does this PR close?

https://github.com/apache/datafusion/issues/20034


## Rationale for this change

ANSI mode support for negative function

## What changes are included in this PR?

Added support for ANSI mode for negative function

## Are these changes tested?

yes through UT

## Are there any user-facing changes?

yes, adds ANSI support to existing function.

---------

Co-authored-by: Subham Singhal <subhamsinghal@Subhams-MacBook-Air.local>
 2026-02-14 18:22:04 +00:00
Neil Conway 3f38609277 perf: Optimize replace() fastpath by avoiding alloc (#20344) 
## Which issue does this PR close?

- Closes #20343.

## Rationale for this change

We already have a fastpath for when `from` and `to` are both single
ASCII characters, but this fastpath could be further optimized by
avoiding the `Vec<u8>` allocation.

## What changes are included in this PR?

Implement the described optimization.

## Are these changes tested?

Yes, no new tests or benchmarks warranted. This PR yields a 10-50%
performance improvement for the relevant microbenchmarks.

## Are there any user-facing changes?

No.
 2026-02-14 12:34:35 +00:00
dario curreri f471aaf275 feat(datafusion-cli): enhance CLI helper with default hint (#20310) 
# Pull Request Description

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

When the CLI prompt is empty, users have no indication of how to get
help or exit. Showing a subtle gray hint (e.g. `\? for help, \q to
quit`) on an empty line improves discoverability without changing any
existing behavior.

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

- Implement `Hinter::hint` on `CliHelper` so that when the current line
is empty, a default hint is shown (e.g. `\? for help, \q to quit`) in
gray.
- Add a `DEFAULT_HINT_SUGGESTION` constant and use it for the hint text.
- Expose the highlighter’s `Color` type and its methods as `pub(crate)`
and add `Color::gray()` for styling the hint.

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

No new tests were added; the change is a small UI hint.

Manual testing: start the CLI and confirm the hint appears on an empty
line and disappears when typing. Happy to add tests if the project
expects them for this behavior.

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

Yes. When the input line is empty, users now see a gray hint: `\? for
help, \q to quit`. No API or CLI flag changes. Documentation could
optionally mention this hint in the CLI docs if the project documents
REPL UX.

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-14 10:30:50 +00:00
Adrian Garcia Badaracco 7d217b1ec3 Reduce ExtractLeafExpressions optimizer overhead with fast pre-scan (#20341) 
## Summary

Follow-up to #20117 which added the `ExtractLeafExpressions` and
`PushDownLeafProjections` optimizer rules for get_field pushdown.

Benchmarking revealed that these rules added 5-31% overhead on *all*
queries (including those with no struct/get_field expressions) because
they unconditionally allocated column HashSets, extractors, and walked
every expression tree for every Filter/Sort/Limit/Aggregate/Join node.

This PR adds:
- **`has_extractable_expr()` pre-scan**: A lightweight check using
`Expr::exists()` that short-circuits before any expensive allocations
when no `MoveTowardsLeafNodes` expressions are present
- **Config option**
`datafusion.optimizer.enable_leaf_expression_pushdown` to disable the
rules entirely

### Benchmark Results (vs no-rules baseline)

| Benchmark | Before Fix | After Fix |
|---|---|---|
| physical_select_aggregates_from_200 | +31.1% | +3.7% |
| physical_many_self_joins | +12.9% | +2.2% |
| physical_join_consider_sort | +12.9% | +1.0% |
| physical_unnest_to_join | +12.5% | +1.4% |
| physical_select_one_from_700 | +12.2% | +2.6% |
| physical_theta_join_consider_sort | +8.7% | +0.2% |
| physical_plan_tpch_q18 | +9.3% | +1.4% |
| physical_plan_tpch_all | +4.8% | +2.1% |
| physical_plan_tpcds_all | +5.6% | +2.2% |

## Test plan

- [x] All 47 `extract_leaf_expressions` unit tests pass
- [x] Benchmarked with `cargo bench -p datafusion --bench sql_planner`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-13 22:54:44 +00:00
Neil Conway bdad206a3d chore: Make memchr a workspace dependency (#20345) 
## Which issue does this PR close?

N/A

## Rationale for this change

Suggested by @alamb.

## What changes are included in this PR?

Moving memchr to be a workspace dependency, instead of a subcrate
dependency.

## Are these changes tested?

Yes, no new tests needed.

## Are there any user-facing changes?

No.
 2026-02-13 20:17:50 +00:00
Sergey Zhukov 80485d7516 unify the prettier version adv(#20024) (#20311) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #https://github.com/apache/datafusion/issues/20024.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-13 17:23:17 +00:00
Neil Conway e9bcfb449c perf: Optimize compare_element_to_list (#20323) 
## Which issue does this PR close?

- Closes #20322 

## Rationale for this change

`compare_element_to_list` is a utility function used by several of the
array-related UDFs (e.g., array_position, array_positions, array_remove,
and array_replace).

The current implementation extracts a scalar from an array using
`arrow::compute::take()`. This is slow; we can just use `slice`
directly, which also avoids allocating an intermediate array of indices.

## What changes are included in this PR?

## Are these changes tested?

Yes; microbenchmarks indicate 15-50% performance improvement for
`array_remove`.

## Are there any user-facing changes?

No.
 2026-02-13 17:15:10 +00:00
Kumar Ujjawal f5a2ac3ec9 fix: percentile_cont interpolation causes NaN for f16 input (#20208) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #18945

## Rationale for this change

percentile_cont interpolation for Float16 could overflow f16
intermediates (e.g. when scaling the fractional component), producing
inf/NaN and incorrect results. This PR makes interpolation numerically
safe for f16.


<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

• Perform percentile interpolation in f64 and cast back to the input
float type (f16/f32/f64) to avoid f16 overflow.
• Add a regression unit test covering Float16 interpolation near the
maximum finite value.

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

Yes

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

Yes. percentile_cont on Float16 inputs no longer returns NaN due to
interpolation overflow and will produce correct
finite results for valid finite f16 data

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-13 17:06:28 +00:00
Neil Conway 4ad5c3dab7 perf: Optimize strpos() for ASCII-only inputs (#20295) 
The previous implementation had a fast path for ASCII-only inputs, but
it was still relatively slow. Switch to using memchr::memchr() to find
the first matching byte and then check the rest of the bytes by hand.
This improves performance for ASCII inputs by 2x-4x on the built-in
strpos benchmarks.

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #20294.

## Are these changes tested?

Yes, passes unit tests and SLT.

## Are there any user-facing changes?

No.
 2026-02-13 17:05:14 +00:00
Jonathan Chen 682da846b5 feat: Push limit into hash join (#20228) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Part of #18295.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?
Push limit down into hash join using limit pushdown optimizer. Use limit
pushdown optimizer to pass the limit value to Hash Join exec using
`with_fetch` and passing the `fetch` value to `LimitedBatch Coalescer`
to emit the batch once the limit is hit.

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?
SLT tests + unit tests

---------

Co-authored-by: Yongting You <2010youy01@gmail.com>
 2026-02-13 06:20:25 +00:00
Adrian Garcia Badaracco 9c6a35ffea feat: add ExtractLeafExpressions optimizer rule for get_field pushdown (#20117) 
## Summary

Adds a two-pass optimizer pipeline (`ExtractLeafExpressions` +
`PushDownLeafProjections`) that pushes cheap `MoveTowardsLeafNodes`
expressions (like `get_field` for struct field access) closer to data
sources, enabling source-level optimizations such as Parquet column
pruning for nested struct fields.

### Motivation

Previously, `get_field(s, 'label')` stayed in the top-level
`ProjectionExec`, forcing the scan to read the entire struct column `s`.
With this change, `get_field` is pushed all the way down to
`DataSourceExec`, allowing Parquet to read only the referenced
sub-columns.

### Example

```sql
SELECT id, s['label'] FROM t WHERE s['value'] > 150
```

**Before:** `get_field(s, 'label')` stayed in ProjectionExec, reading
full struct `s`

**After:** Both `get_field` expressions pushed to DataSourceExec:
```
DataSourceExec: projection=[get_field(s, value) as __datafusion_extracted_1, get_field(s, label) as __datafusion_extracted_2, id]
```

### How It Works

**Pass 1 — `ExtractLeafExpressions`** (top-down):
For non-projection nodes (Filter, Sort, Limit, Aggregate, Join),
extracts `MoveTowardsLeafNodes` sub-expressions into **extraction
projections** below the node with `__datafusion_extracted_N` aliases,
and adds **recovery projections** above to restore the original output
schema.

```text
-- Before:
Filter: user['status'] = 'active'
  TableScan: t [id, user]

-- After:
Projection: id, user                                             ← recovery projection
  Filter: __datafusion_extracted_1 = 'active'
    Projection: user['status'] AS __datafusion_extracted_1, id, user  ← extraction projection
      TableScan: t [id, user]
```

**Pass 2 — `PushDownLeafProjections`** (top-down):
Pushes extraction projections down through schema-preserving nodes
(Filter, Sort, Limit) and merges them into existing projections. Also
handles:
- **Mixed projections** containing `MoveTowardsLeafNodes`
sub-expressions — splits them into recovery + extraction, then pushes
the extraction down
- **Multi-input nodes** (Join, SubqueryAlias) — routes each extracted
expression to the correct input based on column references
- **SubqueryAlias** — remaps qualifiers from alias-space to input-space
before routing

After both passes, `OptimizeProjections` (which runs next) merges
consecutive projections and pushes `get_field` to the scan.

### Changes by file

- **`extract_leaf_expressions.rs`** (new, ~2800 lines): Both optimizer
rules plus extensive unit tests
- **`push_down_filter.rs`**: Teaches `PushDownFilter` to not push
filters through extraction projections (would undo the extraction by
rewriting `__datafusion_extracted_1 > 150` back to `get_field(s,'value')
> 150`). Adds 2 unit tests for this behavior.
- **`utils.rs`**: Adds `EXTRACTED_EXPR_PREFIX` constant and
`is_extracted_expr_projection()` detection helper
- **`optimizer.rs`**: Registers both new rules after
`CommonSubexprEliminate` and before `OptimizeProjections`
- **`expr.rs`**: Makes `Expr::Alias` delegate to the inner expression
for `placement()`, so aliases around `get_field` are correctly
classified as `MoveTowardsLeafNodes`
- **`test/mod.rs`**: Adds `test_table_scan_with_struct()` test helper
- **SLT files**: Updated explain plans reflecting extraction aliases in
logical plans and pushed-down `get_field` in physical plans

### Interaction with other optimizer rules

- **`PushDownFilter`**: Extraction projections are detected via
`is_extracted_expr_projection()` and filters are NOT pushed through them
- **`CommonSubexprEliminate`**: Runs before extraction; CSE aliases
(`__common_expr_N`) are preserved and correctly handled during merge
- **`OptimizeProjections`**: Runs after extraction; merges the recovery
+ extraction projections and pushes `get_field` to the scan

## Test plan

- [x] ~1400 lines of unit tests in `extract_leaf_expressions.rs`
covering: Filter, Sort, Limit, Aggregate, Join, SubqueryAlias, Union,
nested projections, deduplication, idempotency, mixed projections, and
multi-input routing
- [x] 2 new unit tests in `push_down_filter.rs` for
filter-through-extraction blocking
- [x] Updated sqllogictest expectations in `projection_pushdown.slt`,
`push_down_filter.slt`, `explain.slt`, `projection.slt`, `struct.slt`,
`unnest.slt`
- [x] All optimizer tests pass (`cargo test -p datafusion-optimizer`)

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-13 02:42:29 +00:00
xudong.w f48dc72220 Remove the statistics() api in execution plan (#20319) 
The API has been marked as deprecated for about 10 months, and we're
planning to do more
[changes](https://github.com/apache/datafusion/issues/20184) for
statistics. I think it's time to let it go.
 2026-02-13 01:09:47 +00:00
Kumar Ujjawal 51e5c98cf5 fix null handling for nanvl & implement fast path (#20205) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Part of  https://github.com/apache/datafusion-comet/issues/2986

## Rationale for this change

`nanvl` currently evaluates scalar inputs via
`make_scalar_function(nanvl, vec![])`, which converts scalar values into
size‑1 arrays before execution and then converts back. This adds
unnecessary overhead for constant folding / scalar
evaluation

Also fix bug where `null` was being returned if `y` was null, even if
`x` was not `nan`

- We treat nulls as normal values; we return `x` if and only if `x` is
not `nan`, otherwise return `y`

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

- Add match-based scalar fast path for `ColumnarValue::Scalar +
ColumnarValue::Scalar`
- Add Criterion benchmarks:
  - `nanvl/scalar_f64`
  - `nanvl/scalar_f32`

Benchmark      |       Before    |   After  |  Speedup
 ━━━━━━━━━━━━━━━━━━━━━━━
  nanvl/scalar_f64   | ~240.1 ns  | 50.104 ns    ~4.79x
  nanvl/scalar_f32   |~237.1 ns    | 49.284 ns    ~4.81x

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

Yes

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

No

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-13 00:56:53 +00:00
Oleks V a3d46510f1 chore: change width_bucket buckets parameter from i32 to i64 (#20330) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #20320 .

## Rationale for this change

Change `bucket` datatype to be same as Spark


https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L1864

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-13 00:54:08 +00:00
Divyansh Pratap Singh 2e202fd6fe docs: fix typos and improve wording in README (#20301) 
## Which issue does this PR close?

N/A

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->


## Rationale for this change

This PR fixes minor typos and improves wording consistency in README.md
to enhance clarity and maintain consistent formatting.


<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

- Fix minor typos
- Improve wording clarity
- Ensure consistent capitalization in feature descriptions

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

Not needed. These are documentation changes only.


<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

No. This only updates the README.

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-13 00:51:55 +00:00
Jack Kleeman af5f470465 Support pushing down empty projections into joins (#20191) 
## Which issue does this PR close?


- Closes #20190.

## Rationale for this change

We should push down empty projections into HashJoinExec

## What changes are included in this PR?

1. try_embed_projection should embed empty projections
2. build_batch_empty_build_side should support empty schemas

## Are these changes tested?

Yes

## Are there any user-facing changes?

No
 2026-02-12 20:35:51 +00:00
Neil Conway 2cca3fad33 chore: Remove "extern crate criterion" in benches (#20299) 
## Which issue does this PR close?

- Closes #20298

## Rationale for this change

This is no longer necessary with modern editions of Rust. In many cases,
we also did `#[macro_use]`, so removing this cleans up the namespace of
the file doing the `use`.

## What changes are included in this PR?

Remove "extern crate criterion" in benchmark code, and update "use"
statements as necessary. Along the way, remove a few unnecessary `extern
crate arrow` and similar from the benchmark code.

## Are these changes tested?

Yes, but no new tests added or needed.

## Are there any user-facing changes?

No.
 2026-02-12 20:34:45 +00:00
Acfboy e4e8598be4 fix: Add integer check for bitwise coercion (#20241) 
## Which issue does this PR close?
N/A

## Rationale for this change
In the original codebase, bitwise_coercion was implemented as follows:
```rust
if left_type == right_type {
        return Some(left_type.clone());
}
```

This causes any identical types—such as floats—to pass the check during
the logical planning stage. The error only surfaces much later when the
arrow kernel attempts execution. This appears to be a minor oversight by
the original author.

## What changes are included in this PR?

```diff
- if left_type == right_type {
+ if left_type == right_type && left_type.is_integer() {
         return Some(left_type.clone());
 }
```


## Are these changes tested?

Yes, a new unit test is added, and all existing tests passed.

## Are there any user-facing changes?
No.
 2026-02-12 20:33:42 +00:00
Andrew Lamb 9fd84e796a Introduce ProjectionExprs::unproject_exprs/project_exprs and improve docs (#20193) 
## Which issue does this PR close?

- Part of https://github.com/apache/datafusion/issues/20173

## Rationale for this change

I am going through how the various layers of expression pushdown and
schema rewrites work, and I spent a long time confused about what the
`sync_with_child` parameter on `update_expr` did -- like what was the
different between`update_expr(.., true)` vs `update_expr(..., false)` 😕

After some study I concluded it controls which way the rewrite is done
(either project the expressions to refer to the projection expressions,
or the opposite, 'unproject' an expression and substitute the projection
definitions back in)

## What changes are included in this PR?

1. Introduce ProjectionExprs::unproject_exprs/project_exprs 
2. rename `sync_with_child` to `unproject` 
3. Improve documentation about what is done

## Are these changes tested?
Yes by existing ci

## Are there any user-facing changes?

There are new APIs, but no changes to existing APIs
 2026-02-12 20:33:05 +00:00
Eren Avsarogullari 4d06c40d8d Improve ExternalSorter ResourcesExhausted Error Message (#20226) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #20225.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
When there is not enough memory to continue external sort, either
increasing the `memory limit` or decreasing
`sort_spill_reservation_bytes` need to be applied. It can be useful to
guide the user with clearer error message by highlighting required
configs for the consistency because expected settings are as follows:
```
SET datafusion.runtime.memory_limit = '10G'
SET datafusion.execution.sort_spill_reservation_bytes = 10485760
```
Current:
```
Not enough memory to continue external sort. Consider increasing the memory limit, 
or decreasing sort_spill_reservation_bytes
```
New:
```
Not enough memory to continue external sort. Consider increasing the memory limit config: 
'datafusion.runtime.memory_limit', or decreasing the config: 'datafusion.execution.sort_spill_reservation_bytes'.
```

## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
A new UT case has been added to cover this use-case and it has been
tested locally successfully.

**For updated snapshot files:**
Following integration tests have been successful by DataFusion Test
Pipeline:
```
test test_cli_top_memory_consumers::case_1 ... ok
test test_cli_top_memory_consumers::case_2 ... ok
test test_cli_top_memory_consumers::case_3 ... ok
```
**Ref:**
https://github.com/apache/datafusion/actions/runs/21811797863/job/62925363536?pr=20226

## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
Yes, improving legacy `ExternalSorter` `ResourcesExhausted` Error
Message.

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-12 20:27:29 +00:00
Neil Conway 73bce15f29 fix: Fix scalar broadcast for to_timestamp() (#20224) 
When to_timestamp() was invoked with a scalar Float64 and an array of
strings, the previous coding neglected to broadcast the scalar to the
array properly when producing the return value. That is, a query like
`SELECT to_timestamp(123.5, t.x) FROM t` would result in:

    Internal error: UDF to_timestamp returned a different number of rows
    than expected

## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #20223

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

Yes, added SLT.

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-12 20:26:55 +00:00
Neil Conway 0401a4732f fix: Avoid integer overflow in substr() (#20199) 
## Rationale for this change

Evaluating `SELECT SUBSTR('', 2, 9223372036854775807);` yields (in a
debug build):

```
thread 'main' (41414592) panicked at datafusion/functions/src/string/split_part.rs:236:47:
attempt to negate with overflow
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/ded5c06cf21d2b93bffd5d884aa6e96934ee4234/library/std/src/panicking.rs:698:5
   1: core::panicking::panic_fmt
             at /rustc/ded5c06cf21d2b93bffd5d884aa6e96934ee4234/library/core/src/panicking.rs:80:14
   2: core::panicking::panic_const::panic_const_neg_overflow
             at /rustc/ded5c06cf21d2b93bffd5d884aa6e96934ee4234/library/core/src/panicking.rs:180:17
   3: datafusion_functions::string::split_part::split_part_impl::{{closure}}
             at ./datafusion/functions/src/string/split_part.rs:236:47
   4: core::iter::traits::iterator::Iterator::try_for_each::call::{{closure}}
             at /Users/neilconway/.rustup/toolchains/1.92.0-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:2485:26
[...]
```

Found via fuzzing.

## Are these changes tested?

Yes, added a unit test.
 2026-02-12 20:25:12 +00:00
notashes 85cdf53fab perf: various optimizations to eliminate branch misprediction in hash_utils (#20168) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Part of #20152 

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

Compile time monomorphization helps bring `rehash` outside the hot loop
where it's not required.

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
Currently the PR adds a specialized `hash_dictionary_inner()` function
with const generic parameters that check for nulls in keys, values. It
also handles specific edge cases of just nulls in keys or values.

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
There are no additional tests yet. But I will add 'em as I continue. The
benchmark results seem promising.
here's `cargo bench --bench with_hashes -- dictionary` for

 <details>
 <summary>origin/main</summary>

```
Gnuplot not found, using plotters backend
Benchmarking dictionary_utf8_int32: single, no nulls
Benchmarking dictionary_utf8_int32: single, no nulls: Warming up for 3.0000 s
Benchmarking dictionary_utf8_int32: single, no nulls: Collecting 100 samples in estimated 5.0461 s (470k iterations)
Benchmarking dictionary_utf8_int32: single, no nulls: Analyzing
dictionary_utf8_int32: single, no nulls
                        time:   [10.668 µs 10.700 µs 10.734 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) low mild

Benchmarking dictionary_utf8_int32: single, nulls
Benchmarking dictionary_utf8_int32: single, nulls: Warming up for 3.0000 s
Benchmarking dictionary_utf8_int32: single, nulls: Collecting 100 samples in estimated 5.0428 s (409k iterations)
Benchmarking dictionary_utf8_int32: single, nulls: Analyzing
dictionary_utf8_int32: single, nulls
                        time:   [12.269 µs 12.293 µs 12.322 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

Benchmarking dictionary_utf8_int32: multiple, no nulls
Benchmarking dictionary_utf8_int32: multiple, no nulls: Warming up for 3.0000 s
Benchmarking dictionary_utf8_int32: multiple, no nulls: Collecting 100 samples in estimated 5.0864 s (162k iterations)
Benchmarking dictionary_utf8_int32: multiple, no nulls: Analyzing
dictionary_utf8_int32: multiple, no nulls
                        time:   [31.357 µs 31.426 µs 31.506 µs]
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  1 (1.00%) high severe

Benchmarking dictionary_utf8_int32: multiple, nulls
Benchmarking dictionary_utf8_int32: multiple, nulls: Warming up for 3.0000 s
Benchmarking dictionary_utf8_int32: multiple, nulls: Collecting 100 samples in estimated 5.0842 s (141k iterations)
Benchmarking dictionary_utf8_int32: multiple, nulls: Analyzing
dictionary_utf8_int32: multiple, nulls
                        time:   [36.060 µs 36.135 µs 36.220 µs]
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
  5 (5.00%) high severe
```
</details>

<details>
<summary>feat/brunch-prediction</summary>

```
Gnuplot not found, using plotters backend
Benchmarking dictionary_utf8_int32: single, no nulls
Benchmarking dictionary_utf8_int32: single, no nulls: Warming up for 3.0000 s
Benchmarking dictionary_utf8_int32: single, no nulls: Collecting 100 samples in estimated 5.0176 s (1.1M iterations)
Benchmarking dictionary_utf8_int32: single, no nulls: Analyzing
dictionary_utf8_int32: single, no nulls
                        time:   [4.7186 µs 4.7496 µs 4.7821 µs]
                        change: [−55.829% −55.537% −55.240%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking dictionary_utf8_int32: single, nulls
Benchmarking dictionary_utf8_int32: single, nulls: Warming up for 3.0000 s
Benchmarking dictionary_utf8_int32: single, nulls: Collecting 100 samples in estimated 5.0295 s (712k iterations)
Benchmarking dictionary_utf8_int32: single, nulls: Analyzing
dictionary_utf8_int32: single, nulls
                        time:   [6.9647 µs 7.0426 µs 7.1281 µs]
                        change: [−43.806% −43.445% −42.993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
  1 (1.00%) low severe
  4 (4.00%) low mild
  1 (1.00%) high mild
  10 (10.00%) high severe

Benchmarking dictionary_utf8_int32: multiple, no nulls
Benchmarking dictionary_utf8_int32: multiple, no nulls: Warming up for 3.0000 s
Benchmarking dictionary_utf8_int32: multiple, no nulls: Collecting 100 samples in estimated 5.0600 s (348k iterations)
Benchmarking dictionary_utf8_int32: multiple, no nulls: Analyzing
dictionary_utf8_int32: multiple, no nulls
                        time:   [13.365 µs 13.384 µs 13.404 µs]
                        change: [−57.610% −57.464% −57.313%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  2 (2.00%) low severe
  4 (4.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe

Benchmarking dictionary_utf8_int32: multiple, nulls
Benchmarking dictionary_utf8_int32: multiple, nulls: Warming up for 3.0000 s
Benchmarking dictionary_utf8_int32: multiple, nulls: Collecting 100 samples in estimated 5.0569 s (242k iterations)
Benchmarking dictionary_utf8_int32: multiple, nulls: Analyzing
dictionary_utf8_int32: multiple, nulls
                        time:   [20.785 µs 20.962 µs 21.173 µs]
                        change: [−42.370% −42.001% −41.579%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 18 outliers among 100 measurements (18.00%)
  1 (1.00%) low severe
  3 (3.00%) high mild
  14 (14.00%) high severe
```

 </details>

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

---------

Co-authored-by: Daniël Heres <danielheres@gmail.com>
Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
 2026-02-12 16:04:50 +00:00
Josh Elkind 5fccac12cc Add protoc support for ArrowScanExecNode (#20280) (#20284) 
## Which issue does this PR close?

- Closes #20280.

## Rationale for this change

Physical plans that read Arrow files (.arrow / IPC) could not be
serialized or deserialized via the proto layer. PhysicalPlanNode already
had scan nodes for Parquet, CSV, JSON, Avro, and in-memory sources, but
not for Arrow, so a DataSourceExec using ArrowSource was not
round-trippable. That blocked use cases like distributing plans that
scan Arrow files (e.g. Ballista). This change adds Arrow scan to the
proto layer so those plans can be serialized and deserialized like the
other file formats.

## What changes are included in this PR?

Proto: Added ArrowScanExecNode (with FileScanExecConf base_conf) and
arrow_scan = 38 to the PhysicalPlanNode oneof in datafusion.proto.

Generated code: Updated prost.rs and pbjson.rs to include
ArrowScanExecNode and the ArrowScan variant (manual edits; protoc was
not run).

To-proto: In try_from_data_source_exec, when the data source is a
FileScanConfig whose file source is ArrowSource, it is now serialized as
ArrowScanExecNode.

From-proto: Implemented try_into_arrow_scan_physical_plan to deserialize
ArrowScanExecNode into DataSourceExec with ArrowSource; missing
base_conf returns an explicit error (no .unwrap()).

Test: Added roundtrip_arrow_scan in roundtrip_physical_plan.rs to assert
Arrow scan plans round-trip correctly.

## Are these changes tested?

Yes. A new test roundtrip_arrow_scan builds a physical plan that scans
Arrow files, serializes it to bytes and deserializes it back, and
asserts the round-tripped plan matches the original. The full cargo test
-p datafusion-proto suite (150 tests: unit, integration, and doc tests)
passes, including all existing roundtrip and serialization tests.

## Are there any user-facing changes?

No. This only extends the existing physical-plan proto support to Arrow
scan. Callers that already serialize/deserialize physical plans (e.g.
for distributed execution) can now round-trip plans that read Arrow
files in addition to Parquet, CSV, JSON, and Avro, with no API or
behavioral changes for existing usage.
 2026-02-12 09:50:23 +00:00
Jonathan Chen aa9520e6e2 feat: Add partition_stats() for EmptyExec (#20203) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Part of #15873 .

## Rationale for this change
Add partition stats for `EmptyExec`. 
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?
Added integration test.
 2026-02-12 02:24:30 +00:00
notashes a544b8fc08 fix: disable dynamic filter pushdown for non min/max aggregates (#20279) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Partially closes #20267 

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
Currently whenever we get a query with `min` or `max` we default to
always pushing down the dynamic filter (when it's enabled). However if
the query contains other aggregate functions such as `sum`, `avg` they
require the full batch of rows. And because of the pruned rows we
receive incorrect outputs for the query.

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

return the `init_dynamic_filter()` early if it contains non min/max
aggregates.

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

Tested locally for the same query mentioned in the issue with
`hits_partitioned` and got the correct output. Will add the tests!

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->

---------

Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
 2026-02-12 00:51:19 +00:00
notashes ecf3b502cf Add StructArray and RunArray benchmark tests to with_hashes (#20182) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #20181 

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
Issue #20152 shows some areas of optimization for `RunArray` and
`StructArray` hashing. But the existing `with_hashes` benchmark tests
don't include coverage for these!


## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
Added benchmarks to `with_hashes.rs`:
  - **StructArray**: 4-column struct (bool, int32, int64, string)
  - **RunArray**: Int32 run-encoded array
  - Both include single/multiple columns and with/without nulls

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

No additional tests added, but the benchmarks both compile and run. 

<details>
<summary>a sample run:</summary>

```
❯ cargo bench --features=parquet --bench with_hashes -- array
   Compiling datafusion-common v52.1.0 (/Users/notashes/dev/datafusion/datafusion/common)
    Finished `bench` profile [optimized] target(s) in 34.49s
     Running benches/with_hashes.rs (target/release/deps/with_hashes-2f180744d22084f3)
Gnuplot not found, using plotters backend
struct_array: single, no nulls
                        time:   [38.389 µs 38.437 µs 38.485 µs]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  2 (2.00%) high mild

struct_array: single, nulls
                        time:   [46.108 µs 46.197 µs 46.291 µs]
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

struct_array: multiple, no nulls
                        time:   [114.64 µs 114.79 µs 114.93 µs]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  1 (1.00%) high mild

struct_array: multiple, nulls
                        time:   [138.29 µs 138.62 µs 139.07 µs]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low severe
  4 (4.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe

run_array_int32: single, no nulls
                        time:   [1.8777 µs 1.9098 µs 1.9457 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

run_array_int32: single, nulls
                        time:   [2.0110 µs 2.0417 µs 2.0751 µs]
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe

run_array_int32: multiple, no nulls
                        time:   [5.0511 µs 5.0603 µs 5.0693 µs]
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild

run_array_int32: multiple, nulls
                        time:   [5.6052 µs 5.6201 µs 5.6353 µs]
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
  ```


</details>

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
 2026-02-11 19:06:15 +00:00
lyne 4e2c0f1fea perf: improve performance of array_union/array_intersect with batched row conversion (#20243) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

The current implementation of `array_union` and `array_intersect`
performs `RowConverter::convert_columns()` on a per-row basis, which
introduces avoidable overhead due to repeated conversions and
intermediate allocations.

This PR improves performance by:
1. converting all list values to rows in a batch
2. reusing hash sets across iterations
3. removing the `sorted().dedup()` pattern in favor of hash-based set
operations

## What changes are included in this PR?

Refactored the internal set operation implementation to use batch row
conversion and a single-pass construction of result arrays.

### Benchmarks
```
group                               before                                  optimized
-----                               ------                                  ---------
array_intersect/high_overlap/10     2.99  1442.0±99.94µs        ? ?/sec     1.00   481.6±21.45µs        ? ?/sec
array_intersect/high_overlap/100    1.90      9.5±0.63ms        ? ?/sec     1.00      5.0±0.09ms        ? ?/sec
array_intersect/high_overlap/50     2.01      5.3±0.41ms        ? ?/sec     1.00      2.6±0.05ms        ? ?/sec
array_intersect/low_overlap/10      3.47  1288.1±72.39µs        ? ?/sec     1.00   371.4±14.08µs        ? ?/sec
array_intersect/low_overlap/100     2.35      9.2±0.43ms        ? ?/sec     1.00      3.9±0.08ms        ? ?/sec
array_intersect/low_overlap/50      2.45      5.1±0.41ms        ? ?/sec     1.00      2.1±0.07ms        ? ?/sec
array_union/high_overlap/10         4.01  1593.1±292.17µs        ? ?/sec    1.00   396.9±13.43µs        ? ?/sec
array_union/high_overlap/100        2.54      9.8±0.18ms        ? ?/sec     1.00      3.9±0.11ms        ? ?/sec
array_union/high_overlap/50         2.65      5.4±0.10ms        ? ?/sec     1.00      2.0±0.07ms        ? ?/sec
array_union/low_overlap/10          3.74  1622.7±96.50µs        ? ?/sec     1.00   434.3±17.87µs        ? ?/sec
array_union/low_overlap/100         2.39     10.3±0.92ms        ? ?/sec     1.00      4.3±0.11ms        ? ?/sec
array_union/low_overlap/50          2.63      5.8±0.27ms        ? ?/sec     1.00      2.2±0.11ms        ? ?/sec
```

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

Yes. Existing SQL logic tests updated to reflect new output order.

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
4. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

Yes. The output order may differ from the previous implementation.

Previously, results were implicitly sorted due to the use of
`sorted().dedup()`. The new implementation preserves the order of first
appearance within each list.

This is a user-visible behavioral change, but it is consistent with
typical SQL set operation semantics, which do not guarantee a specific
output order.

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-11 16:14:47 +00:00
Sergey Zhukov 17416bf92f Cleanup example metadata parsing utilities(#20251) (#20252) 
## Which issue does this PR close?

<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->

- Closes #https://github.com/apache/datafusion/issues/20251.

## Rationale for this change

<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?

<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->

## Are these changes tested?

<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code

If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->

## Are there any user-facing changes?

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
 2026-02-11 16:13:54 +00:00
dependabot[bot] 6b6f64d7fb chore(deps): bump the arrow-parquet group with 7 updates (#20256) 
Bumps the arrow-parquet group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [arrow](https://github.com/apache/arrow-rs) | `57.2.0` | `57.3.0` |
| [arrow-buffer](https://github.com/apache/arrow-rs) | `57.2.0` |
`57.3.0` |
| [arrow-flight](https://github.com/apache/arrow-rs) | `57.2.0` |
`57.3.0` |
| [arrow-ipc](https://github.com/apache/arrow-rs) | `57.2.0` | `57.3.0`
|
| [arrow-ord](https://github.com/apache/arrow-rs) | `57.2.0` | `57.3.0`
|
| [arrow-schema](https://github.com/apache/arrow-rs) | `57.2.0` |
`57.3.0` |
| [parquet](https://github.com/apache/arrow-rs) | `57.2.0` | `57.3.0` |

Updates `arrow` from 57.2.0 to 57.3.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/releases">arrow's
releases</a>.</em></p>
<blockquote>
<h2>arrow 57.3.0</h2>
<!-- raw HTML omitted -->
<h1>Changelog</h1>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">arrow's
changelog</a>.</em></p>
<blockquote>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/arrow-rs/commit/750500594839258f809a8248bce92b244bbb40cc"><code>7505005</code></a>
[57_maintenance] Update version to 57.3.0, add changelog (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9333">#9333</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/6bbfb9932b5268c600bcf3dc201e79acd20a551e"><code>6bbfb99</code></a>
[maintenance_57] Fix string array equality when the values buffer is the
same...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/505eb8ec606907af2144ca664555ec60d1d55a26"><code>505eb8e</code></a>
[57_maintenance] Revert &quot;Seal Array trait (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)&quot;,
mark <code>Array</code> as <code>unsafe</code> ...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/74cf9142cc36dac1514cd0bcc71ed6974155689e"><code>74cf914</code></a>
[57_maintenance] Mark <code>BufferBuilder::new_from_buffer</code> as
unsafe (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9292">#9292</a>)
(<a
href="https://redirect.github.com/apache/arrow-rs/issues/9312">#9312</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/25cc1ac28ccdf2fffb844380d2dde1a1966b5ae4"><code>25cc1ac</code></a>
[57_maintenance] fix: ensure <code>BufferBuilder::truncate</code>
doesn't overset length...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/9fc2fbb72e4a7e6dd1c339a69ff3a0f4ca5ec0c1"><code>9fc2fbb</code></a>
[57_maintenance[Parquet] Provide only encrypted column stats in
plaintext foo...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/3df3157b5b14b9f99869cec4cd9feedc0a6736c1"><code>3df3157</code></a>
[57_maintenance] [regression] Error with adaptive predicate pushdown:
&quot;Invali...</li>
<li>See full diff in <a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `arrow-buffer` from 57.2.0 to 57.3.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/releases">arrow-buffer's
releases</a>.</em></p>
<blockquote>
<h2>arrow 57.3.0</h2>
<!-- raw HTML omitted -->
<h1>Changelog</h1>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">arrow-buffer's
changelog</a>.</em></p>
<blockquote>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/arrow-rs/commit/750500594839258f809a8248bce92b244bbb40cc"><code>7505005</code></a>
[57_maintenance] Update version to 57.3.0, add changelog (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9333">#9333</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/6bbfb9932b5268c600bcf3dc201e79acd20a551e"><code>6bbfb99</code></a>
[maintenance_57] Fix string array equality when the values buffer is the
same...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/505eb8ec606907af2144ca664555ec60d1d55a26"><code>505eb8e</code></a>
[57_maintenance] Revert &quot;Seal Array trait (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)&quot;,
mark <code>Array</code> as <code>unsafe</code> ...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/74cf9142cc36dac1514cd0bcc71ed6974155689e"><code>74cf914</code></a>
[57_maintenance] Mark <code>BufferBuilder::new_from_buffer</code> as
unsafe (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9292">#9292</a>)
(<a
href="https://redirect.github.com/apache/arrow-rs/issues/9312">#9312</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/25cc1ac28ccdf2fffb844380d2dde1a1966b5ae4"><code>25cc1ac</code></a>
[57_maintenance] fix: ensure <code>BufferBuilder::truncate</code>
doesn't overset length...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/9fc2fbb72e4a7e6dd1c339a69ff3a0f4ca5ec0c1"><code>9fc2fbb</code></a>
[57_maintenance[Parquet] Provide only encrypted column stats in
plaintext foo...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/3df3157b5b14b9f99869cec4cd9feedc0a6736c1"><code>3df3157</code></a>
[57_maintenance] [regression] Error with adaptive predicate pushdown:
&quot;Invali...</li>
<li>See full diff in <a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `arrow-flight` from 57.2.0 to 57.3.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/releases">arrow-flight's
releases</a>.</em></p>
<blockquote>
<h2>arrow 57.3.0</h2>
<!-- raw HTML omitted -->
<h1>Changelog</h1>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">arrow-flight's
changelog</a>.</em></p>
<blockquote>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/arrow-rs/commit/750500594839258f809a8248bce92b244bbb40cc"><code>7505005</code></a>
[57_maintenance] Update version to 57.3.0, add changelog (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9333">#9333</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/6bbfb9932b5268c600bcf3dc201e79acd20a551e"><code>6bbfb99</code></a>
[maintenance_57] Fix string array equality when the values buffer is the
same...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/505eb8ec606907af2144ca664555ec60d1d55a26"><code>505eb8e</code></a>
[57_maintenance] Revert &quot;Seal Array trait (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)&quot;,
mark <code>Array</code> as <code>unsafe</code> ...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/74cf9142cc36dac1514cd0bcc71ed6974155689e"><code>74cf914</code></a>
[57_maintenance] Mark <code>BufferBuilder::new_from_buffer</code> as
unsafe (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9292">#9292</a>)
(<a
href="https://redirect.github.com/apache/arrow-rs/issues/9312">#9312</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/25cc1ac28ccdf2fffb844380d2dde1a1966b5ae4"><code>25cc1ac</code></a>
[57_maintenance] fix: ensure <code>BufferBuilder::truncate</code>
doesn't overset length...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/9fc2fbb72e4a7e6dd1c339a69ff3a0f4ca5ec0c1"><code>9fc2fbb</code></a>
[57_maintenance[Parquet] Provide only encrypted column stats in
plaintext foo...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/3df3157b5b14b9f99869cec4cd9feedc0a6736c1"><code>3df3157</code></a>
[57_maintenance] [regression] Error with adaptive predicate pushdown:
&quot;Invali...</li>
<li>See full diff in <a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `arrow-ipc` from 57.2.0 to 57.3.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/releases">arrow-ipc's
releases</a>.</em></p>
<blockquote>
<h2>arrow 57.3.0</h2>
<!-- raw HTML omitted -->
<h1>Changelog</h1>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">arrow-ipc's
changelog</a>.</em></p>
<blockquote>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/arrow-rs/commit/750500594839258f809a8248bce92b244bbb40cc"><code>7505005</code></a>
[57_maintenance] Update version to 57.3.0, add changelog (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9333">#9333</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/6bbfb9932b5268c600bcf3dc201e79acd20a551e"><code>6bbfb99</code></a>
[maintenance_57] Fix string array equality when the values buffer is the
same...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/505eb8ec606907af2144ca664555ec60d1d55a26"><code>505eb8e</code></a>
[57_maintenance] Revert &quot;Seal Array trait (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)&quot;,
mark <code>Array</code> as <code>unsafe</code> ...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/74cf9142cc36dac1514cd0bcc71ed6974155689e"><code>74cf914</code></a>
[57_maintenance] Mark <code>BufferBuilder::new_from_buffer</code> as
unsafe (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9292">#9292</a>)
(<a
href="https://redirect.github.com/apache/arrow-rs/issues/9312">#9312</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/25cc1ac28ccdf2fffb844380d2dde1a1966b5ae4"><code>25cc1ac</code></a>
[57_maintenance] fix: ensure <code>BufferBuilder::truncate</code>
doesn't overset length...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/9fc2fbb72e4a7e6dd1c339a69ff3a0f4ca5ec0c1"><code>9fc2fbb</code></a>
[57_maintenance[Parquet] Provide only encrypted column stats in
plaintext foo...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/3df3157b5b14b9f99869cec4cd9feedc0a6736c1"><code>3df3157</code></a>
[57_maintenance] [regression] Error with adaptive predicate pushdown:
&quot;Invali...</li>
<li>See full diff in <a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `arrow-ord` from 57.2.0 to 57.3.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/releases">arrow-ord's
releases</a>.</em></p>
<blockquote>
<h2>arrow 57.3.0</h2>
<!-- raw HTML omitted -->
<h1>Changelog</h1>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">arrow-ord's
changelog</a>.</em></p>
<blockquote>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/arrow-rs/commit/750500594839258f809a8248bce92b244bbb40cc"><code>7505005</code></a>
[57_maintenance] Update version to 57.3.0, add changelog (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9333">#9333</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/6bbfb9932b5268c600bcf3dc201e79acd20a551e"><code>6bbfb99</code></a>
[maintenance_57] Fix string array equality when the values buffer is the
same...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/505eb8ec606907af2144ca664555ec60d1d55a26"><code>505eb8e</code></a>
[57_maintenance] Revert &quot;Seal Array trait (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)&quot;,
mark <code>Array</code> as <code>unsafe</code> ...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/74cf9142cc36dac1514cd0bcc71ed6974155689e"><code>74cf914</code></a>
[57_maintenance] Mark <code>BufferBuilder::new_from_buffer</code> as
unsafe (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9292">#9292</a>)
(<a
href="https://redirect.github.com/apache/arrow-rs/issues/9312">#9312</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/25cc1ac28ccdf2fffb844380d2dde1a1966b5ae4"><code>25cc1ac</code></a>
[57_maintenance] fix: ensure <code>BufferBuilder::truncate</code>
doesn't overset length...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/9fc2fbb72e4a7e6dd1c339a69ff3a0f4ca5ec0c1"><code>9fc2fbb</code></a>
[57_maintenance[Parquet] Provide only encrypted column stats in
plaintext foo...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/3df3157b5b14b9f99869cec4cd9feedc0a6736c1"><code>3df3157</code></a>
[57_maintenance] [regression] Error with adaptive predicate pushdown:
&quot;Invali...</li>
<li>See full diff in <a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `arrow-schema` from 57.2.0 to 57.3.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/releases">arrow-schema's
releases</a>.</em></p>
<blockquote>
<h2>arrow 57.3.0</h2>
<!-- raw HTML omitted -->
<h1>Changelog</h1>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">arrow-schema's
changelog</a>.</em></p>
<blockquote>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/arrow-rs/commit/750500594839258f809a8248bce92b244bbb40cc"><code>7505005</code></a>
[57_maintenance] Update version to 57.3.0, add changelog (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9333">#9333</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/6bbfb9932b5268c600bcf3dc201e79acd20a551e"><code>6bbfb99</code></a>
[maintenance_57] Fix string array equality when the values buffer is the
same...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/505eb8ec606907af2144ca664555ec60d1d55a26"><code>505eb8e</code></a>
[57_maintenance] Revert &quot;Seal Array trait (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)&quot;,
mark <code>Array</code> as <code>unsafe</code> ...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/74cf9142cc36dac1514cd0bcc71ed6974155689e"><code>74cf914</code></a>
[57_maintenance] Mark <code>BufferBuilder::new_from_buffer</code> as
unsafe (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9292">#9292</a>)
(<a
href="https://redirect.github.com/apache/arrow-rs/issues/9312">#9312</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/25cc1ac28ccdf2fffb844380d2dde1a1966b5ae4"><code>25cc1ac</code></a>
[57_maintenance] fix: ensure <code>BufferBuilder::truncate</code>
doesn't overset length...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/9fc2fbb72e4a7e6dd1c339a69ff3a0f4ca5ec0c1"><code>9fc2fbb</code></a>
[57_maintenance[Parquet] Provide only encrypted column stats in
plaintext foo...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/3df3157b5b14b9f99869cec4cd9feedc0a6736c1"><code>3df3157</code></a>
[57_maintenance] [regression] Error with adaptive predicate pushdown:
&quot;Invali...</li>
<li>See full diff in <a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `parquet` from 57.2.0 to 57.3.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/releases">parquet's
releases</a>.</em></p>
<blockquote>
<h2>arrow 57.3.0</h2>
<!-- raw HTML omitted -->
<h1>Changelog</h1>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md">parquet's
changelog</a>.</em></p>
<blockquote>
<h2><a href="https://github.com/apache/arrow-rs/tree/57.3.0">57.3.0</a>
(2026-02-02)</h2>
<p><a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">Full
Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Revert &quot;Seal Array trait&quot;, mark <code>Array</code> as
<code>unsafe</code> <a
href="https://redirect.github.com/apache/arrow-rs/pull/9313">#9313</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/gabotechs">gabotechs</a>)</li>
<li>Mark <code>BufferBuilder::new_from_buffer</code> as unsafe <a
href="https://redirect.github.com/apache/arrow-rs/pull/9312">#9312</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Fix string array equality when the values buffer is the same and
only the offsets to access it differ <a
href="https://redirect.github.com/apache/arrow-rs/pull/9330">#9330</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/jhorstmann">jhorstmann</a>)</li>
<li>Ensure <code>BufferBuilder::truncate</code> doesn't overset length
<a
href="https://redirect.github.com/apache/arrow-rs/pull/9311">#9311</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/Jefffrey">Jefffrey</a>)</li>
<li>[parquet] Provide only encrypted column stats in plaintext footer <a
href="https://redirect.github.com/apache/arrow-rs/pull/9310">#9310</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/rok">rok</a>, <a
href="https://github.com/adamreeve">adamreeve</a>)</li>
<li>[regression] Error with adaptive predicate pushdown: &quot;Invalid
offset …&quot; <a
href="https://redirect.github.com/apache/arrow-rs/pull/9309">#9309</a>
(<a href="https://github.com/alamb">alamb</a>, <a
href="https://github.com/erratic-pattern">erratic-pattern</a>, <a
href="https://github.com/sdf-jkl">sdf-jkl</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/arrow-rs/commit/750500594839258f809a8248bce92b244bbb40cc"><code>7505005</code></a>
[57_maintenance] Update version to 57.3.0, add changelog (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9333">#9333</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/6bbfb9932b5268c600bcf3dc201e79acd20a551e"><code>6bbfb99</code></a>
[maintenance_57] Fix string array equality when the values buffer is the
same...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/505eb8ec606907af2144ca664555ec60d1d55a26"><code>505eb8e</code></a>
[57_maintenance] Revert &quot;Seal Array trait (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9092">#9092</a>)&quot;,
mark <code>Array</code> as <code>unsafe</code> ...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/74cf9142cc36dac1514cd0bcc71ed6974155689e"><code>74cf914</code></a>
[57_maintenance] Mark <code>BufferBuilder::new_from_buffer</code> as
unsafe (<a
href="https://redirect.github.com/apache/arrow-rs/issues/9292">#9292</a>)
(<a
href="https://redirect.github.com/apache/arrow-rs/issues/9312">#9312</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/25cc1ac28ccdf2fffb844380d2dde1a1966b5ae4"><code>25cc1ac</code></a>
[57_maintenance] fix: ensure <code>BufferBuilder::truncate</code>
doesn't overset length...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/9fc2fbb72e4a7e6dd1c339a69ff3a0f4ca5ec0c1"><code>9fc2fbb</code></a>
[57_maintenance[Parquet] Provide only encrypted column stats in
plaintext foo...</li>
<li><a
href="https://github.com/apache/arrow-rs/commit/3df3157b5b14b9f99869cec4cd9feedc0a6736c1"><code>3df3157</code></a>
[57_maintenance] [regression] Error with adaptive predicate pushdown:
&quot;Invali...</li>
<li>See full diff in <a
href="https://github.com/apache/arrow-rs/compare/57.2.0...57.3.0">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-11 14:51:54 +00:00
Adrian Garcia Badaracco f9d37cf769 Clean up expression placement UDF usage in tests (#20272) 
Followup to #20238 bringing in the shared test UDFs from #20239
 2026-02-11 12:54:47 +00:00
Jack Kleeman 0f74dbfda1 FilterExec should remap indices of parent dynamic filters (#20286) 
## Which issue does this PR close?

- Closes #20285.

## Rationale for this change

Parent dynamic filters should be remapped from input to output schema,
otherwise the indices can be wrong

## What changes are included in this PR?

- Use with_child to remap parent filters
- Add unit test that catches this issue

## Are these changes tested?

Yes

## Are there any user-facing changes?

No
 2026-02-11 12:22:51 +00:00
Marko Milenković 49776a65e4 chore: Add confirmation before tarball is released (#20207) 
## Which issue does this PR close?

- Closes #.

## Rationale for this change

Add confirmation dialogue to confirm tarball release 

## What changes are included in this PR?

Update `tarball-release.sh` to ask `y/N` confirmation before it proceeds
to release upload

## Are these changes tested?


## Are there any user-facing changes?

No
 2026-02-11 09:15:40 +00:00
Qi Zhu 69d0f44cc4 Support JSON arrays reader/parse for datafusion (#19924) 
## Which issue does this PR close?

Closes #19920

## Rationale for this change

DataFusion currently only supports line-delimited JSON (NDJSON) format.
Many data sources provide JSON in array format `[{...}, {...}]`, which
cannot be parsed by the existing implementation.

## What changes are included in this PR?

- Add `newline_delimited` option to `JsonOptions` (default `true` for
backward compatibility)
- Implement streaming JSON array to NDJSON conversion via
`JsonArrayToNdjsonReader`
- Support both file-based and stream-based (e.g., S3) reading with
memory-efficient streaming
- Add `ChannelReader` for async-to-sync byte transfer in object store
streaming scenarios
- Add protobuf serialization support for the new option
- Rename `NdJsonReadOptions` to `JsonReadOptions` (with deprecation
alias)
- SQL support via `OPTIONS ('format.newline_delimited' 'false')`

### Architecture
```text
JSON Array File (e.g., 33GB)
        │
        ▼ read chunks via ChannelReader (for streams) or BufReader (for files)
┌───────────────────┐
│ JsonArrayToNdjson │  ← streaming character substitution:
│      Reader       │    '[' skip, ',' → '\n', ']' stop
└───────────────────┘
        │
        ▼ outputs NDJSON format
┌───────────────────┐
│   Arrow Reader    │  ← batch parsing
└───────────────────┘
        │
        ▼ RecordBatch
```

### Memory Efficiency

| Approach | Memory for 33GB file | Parse count |
|----------|---------------------|-------------|
| Load entire file + serde_json | ~100GB+ | 3x |
| Streaming with JsonArrayToNdjsonReader | ~32MB | 1x |

## Are these changes tested?

Yes:
- Unit tests for `JsonArrayToNdjsonReader` (nested objects, escaped
strings, empty arrays, buffer boundaries)
- Unit tests for `ChannelReader`
- Integration tests for `JsonOpener` (file-based, stream-based, large
files, cancellation)
- Schema inference tests (normal, empty, nested struct, list types)
- End-to-end query tests with SQL
- SQLLogicTest for SQL validation

## Are there any user-facing changes?

Yes. Users can now read JSON array format files:

**Via SQL:**
```sql
CREATE EXTERNAL TABLE my_table
STORED AS JSON
OPTIONS ('format.newline_delimited' 'false')
LOCATION 'path/to/array.json';
```

**Via API:**
```rust
let options = JsonReadOptions::default().newline_delimited(false);
ctx.register_json("my_table", "path/to/array.json", options).await?;
```

**Note:** `NdJsonReadOptions` is deprecated in favor of
`JsonReadOptions`.

**Limitation:** JSON array format does not support range-based file
scanning (`repartition_file_scans`). Users will see a clear error
message if this is attempted.
 2026-02-11 04:10:53 +00:00
dependabot[bot] dff1cad385 chore(deps): bump async-compression from 0.4.37 to 0.4.39 (#20259) 
Bumps
[async-compression](https://github.com/Nullus157/async-compression) from
0.4.37 to 0.4.39.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/Nullus157/async-compression/commit/b1da57fecf43f0bb84d7910bc36792a47dcc645b"><code>b1da57f</code></a>
chore(async-compression): release v0.4.39 (<a
href="https://redirect.github.com/Nullus157/async-compression/issues/447">#447</a>)</li>
<li><a
href="https://github.com/Nullus157/async-compression/commit/f95a4b9203abc356d8a372f9d4ce8ad4661cce00"><code>f95a4b9</code></a>
Revert &quot;Fix decoding of empty response streams.&quot; (<a
href="https://redirect.github.com/Nullus157/async-compression/issues/446">#446</a>)</li>
<li><a
href="https://github.com/Nullus157/async-compression/commit/b4e46d8d68dead0c125eb08248540f10708b3f9b"><code>b4e46d8</code></a>
chore(async-compression): release v0.4.38 (<a
href="https://redirect.github.com/Nullus157/async-compression/issues/443">#443</a>)</li>
<li><a
href="https://github.com/Nullus157/async-compression/commit/e67f0b123af45c35a9d00e365650534a136254a8"><code>e67f0b1</code></a>
Fix decoding of empty response streams. (<a
href="https://redirect.github.com/Nullus157/async-compression/issues/444">#444</a>)</li>
<li><a
href="https://github.com/Nullus157/async-compression/commit/f5e532e2989752f6d35f8375ddeeb196c8506713"><code>f5e532e</code></a>
chore(deps): update proptest-derive requirement from 0.7 to 0.8 (<a
href="https://redirect.github.com/Nullus157/async-compression/issues/442">#442</a>)</li>
<li>See full diff in <a
href="https://github.com/Nullus157/async-compression/compare/async-compression-v0.4.37...async-compression-v0.4.39">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=async-compression&package-manager=cargo&previous-version=0.4.37&new-version=0.4.39)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 18:17:03 +00:00
dependabot[bot] b790ed313e chore(deps): bump aws-config from 1.8.12 to 1.8.13 (#20263) 
Bumps [aws-config](https://github.com/smithy-lang/smithy-rs) from 1.8.12
to 1.8.13.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/smithy-lang/smithy-rs/commits">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=aws-config&package-manager=cargo&previous-version=1.8.12&new-version=1.8.13)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 13:40:39 +00:00
dependabot[bot] 9e86de2865 chore(deps): bump regex-syntax from 0.8.8 to 0.8.9 (#20264) 
Bumps [regex-syntax](https://github.com/rust-lang/regex) from 0.8.8 to
0.8.9.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/rust-lang/regex/commit/a3433f691863d80300dfd6a52e332cb5a568e895"><code>a3433f6</code></a>
regex-syntax-0.8.9</li>
<li><a
href="https://github.com/rust-lang/regex/commit/0c07fae444adf0802d84455e689f1143d2dd7790"><code>0c07fae</code></a>
regex-lite-0.1.9</li>
<li><a
href="https://github.com/rust-lang/regex/commit/6a810068f030c023a12c93ccae49bc5fd907c4f6"><code>6a81006</code></a>
cargo: exclude development scripts and fuzzing data</li>
<li><a
href="https://github.com/rust-lang/regex/commit/4733e28ba4f281f643ce93e4089eccbb9a9d5a5a"><code>4733e28</code></a>
automata: fix <code>onepass::DFA::try_search_slots</code> panic when too
many slots are ...</li>
<li><a
href="https://github.com/rust-lang/regex/commit/5ea3eb1e95f0338e283f5f0b4681f0891a1cd836"><code>5ea3eb1</code></a>
1.12.2</li>
<li><a
href="https://github.com/rust-lang/regex/commit/ab0b07171b82d1d4fdc8359505d12b2e818514d4"><code>ab0b071</code></a>
regex-automata-0.4.13</li>
<li>See full diff in <a
href="https://github.com/rust-lang/regex/compare/regex-syntax-0.8.8...regex-syntax-0.8.9">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=regex-syntax&package-manager=cargo&previous-version=0.8.8&new-version=0.8.9)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 13:39:13 +00:00
dependabot[bot] 86e39afcc1 chore(deps): bump criterion from 0.8.1 to 0.8.2 (#20258) 
Bumps [criterion](https://github.com/criterion-rs/criterion.rs) from
0.8.1 to 0.8.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/criterion-rs/criterion.rs/releases">criterion's
releases</a>.</em></p>
<blockquote>
<h2>criterion-plot-v0.8.2</h2>
<h3>Other</h3>
<ul>
<li>Update Readme</li>
</ul>
<h2>criterion-v0.8.2</h2>
<h3>Fixed</h3>
<ul>
<li>don't build alloca on unsupported targets</li>
</ul>
<h3>Other</h3>
<ul>
<li><em>(deps)</em> bump crate-ci/typos from 1.40.0 to 1.43.0</li>
<li>Fix panic with uniform iteration durations in benchmarks</li>
<li>Update Readme</li>
<li>Exclude development scripts from published package</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/criterion-rs/criterion.rs/blob/master/CHANGELOG.md">criterion's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/criterion-rs/criterion.rs/compare/criterion-v0.8.1...criterion-v0.8.2">0.8.2</a>
- 2026-02-04</h2>
<h3>Fixed</h3>
<ul>
<li>don't build alloca on unsupported targets</li>
</ul>
<h3>Other</h3>
<ul>
<li><em>(deps)</em> bump crate-ci/typos from 1.40.0 to 1.43.0</li>
<li>Fix panic with uniform iteration durations in benchmarks</li>
<li>Update Readme</li>
<li>Exclude development scripts from published package</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/criterion-rs/criterion.rs/commit/7f0d745532e3c7b2e11bbf9de9b911f91790d3b1"><code>7f0d745</code></a>
chore: release v0.8.2</li>
<li><a
href="https://github.com/criterion-rs/criterion.rs/commit/4a467ce964052ae9bd9266c0706b470b817613e0"><code>4a467ce</code></a>
chore(deps): bump crate-ci/typos from 1.40.0 to 1.43.0</li>
<li><a
href="https://github.com/criterion-rs/criterion.rs/commit/b277a751453cf9ce0595e41bddf819210a6d6e47"><code>b277a75</code></a>
Fix panic with uniform iteration durations in benchmarks</li>
<li><a
href="https://github.com/criterion-rs/criterion.rs/commit/828af1450d648c599a92a077b75e292747761d99"><code>828af14</code></a>
fix: don't build alloca on unsupported targets</li>
<li><a
href="https://github.com/criterion-rs/criterion.rs/commit/b01316b76e42028f3b1cf3731f643bea7f354f39"><code>b01316b</code></a>
Update Readme</li>
<li><a
href="https://github.com/criterion-rs/criterion.rs/commit/4c02a3b4e560fe1f296c0ed1e9b53e3154a3cac6"><code>4c02a3b</code></a>
Exclude development scripts from published package</li>
<li>See full diff in <a
href="https://github.com/criterion-rs/criterion.rs/compare/criterion-v0.8.1...criterion-v0.8.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=criterion&package-manager=cargo&previous-version=0.8.1&new-version=0.8.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 13:38:11 +00:00
dependabot[bot] a41e543326 chore(deps): bump regex from 1.12.2 to 1.12.3 (#20260) 
Bumps [regex](https://github.com/rust-lang/regex) from 1.12.2 to 1.12.3.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-lang/regex/blob/master/CHANGELOG.md">regex's
changelog</a>.</em></p>
<blockquote>
<h1>1.12.3 (2025-02-03)</h1>
<p>This release excludes some unnecessary things from the archive
published to
crates.io. Specifically, fuzzing data and various shell scripts are now
excluded. If you run into problems, please file an issue.</p>
<p>Improvements:</p>
<ul>
<li><a
href="https://redirect.github.com/rust-lang/regex/pull/1319">#1319</a>:
Switch from a Cargo <code>exclude</code> list to an <code>include</code>
list, and exclude some
unnecessary stuff.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/rust-lang/regex/commit/b028e4f40eac8959d05e82abf8404906b1c565c0"><code>b028e4f</code></a>
1.12.3</li>
<li><a
href="https://github.com/rust-lang/regex/commit/5e195de266e203441b2c8001d6ebefab1161a59e"><code>5e195de</code></a>
regex-automata-0.4.14</li>
<li><a
href="https://github.com/rust-lang/regex/commit/a3433f691863d80300dfd6a52e332cb5a568e895"><code>a3433f6</code></a>
regex-syntax-0.8.9</li>
<li><a
href="https://github.com/rust-lang/regex/commit/0c07fae444adf0802d84455e689f1143d2dd7790"><code>0c07fae</code></a>
regex-lite-0.1.9</li>
<li><a
href="https://github.com/rust-lang/regex/commit/6a810068f030c023a12c93ccae49bc5fd907c4f6"><code>6a81006</code></a>
cargo: exclude development scripts and fuzzing data</li>
<li><a
href="https://github.com/rust-lang/regex/commit/4733e28ba4f281f643ce93e4089eccbb9a9d5a5a"><code>4733e28</code></a>
automata: fix <code>onepass::DFA::try_search_slots</code> panic when too
many slots are ...</li>
<li>See full diff in <a
href="https://github.com/rust-lang/regex/compare/1.12.2...1.12.3">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=regex&package-manager=cargo&previous-version=1.12.2&new-version=1.12.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 13:37:24 +00:00
dependabot[bot] 35caa19b6d chore(deps): bump tempfile from 3.24.0 to 3.25.0 (#20262) 
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.24.0 to
3.25.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md">tempfile's
changelog</a>.</em></p>
<blockquote>
<h2>3.25.0</h2>
<ul>
<li>Allow <code>getrandom</code> 0.4.x while retaining support for
<code>getrandom</code> 0.3.x.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/Stebalien/tempfile/commits">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=tempfile&package-manager=cargo&previous-version=3.24.0&new-version=3.25.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 11:39:25 +00:00
dependabot[bot] cbe878ccdd chore(deps): bump clap from 4.5.56 to 4.5.57 (#20265) 
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.56 to 4.5.57.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.5.57</h2>
<h2>[4.5.57] - 2026-02-03</h2>
<h3>Fixes</h3>
<ul>
<li>Regression from 4.5.55 where having an argument with
<code>.value_terminator(&quot;--&quot;)</code> caused problems with an
argument with <code>.last(true)</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.5.57] - 2026-02-03</h2>
<h3>Fixes</h3>
<ul>
<li>Regression from 4.5.55 where having an argument with
<code>.value_terminator(&quot;--&quot;)</code> caused problems with an
argument with <code>.last(true)</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/clap-rs/clap/commit/69c0ddbbfb56db1bccbb5954b62bb89a567a3c8d"><code>69c0ddb</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/8206bba73fd6c5d567cb95949fd1c3c6c48e4e20"><code>8206bba</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/c109d67ea493823727411f60f354edb3d83117ee"><code>c109d67</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/6104">#6104</a>
from epage/hide</li>
<li><a
href="https://github.com/clap-rs/clap/commit/9d7f2128f77023941b53b7cfc311120a2ead75a2"><code>9d7f212</code></a>
fix(complete): Hide dot files on dynamic completer</li>
<li><a
href="https://github.com/clap-rs/clap/commit/77b3fdbbea64ae0b0b3a51309bcbb861360de8d1"><code>77b3fdb</code></a>
test(complete): Show dot file behavior</li>
<li><a
href="https://github.com/clap-rs/clap/commit/f89b9b8d1b818a2eb3863745be48725ace2d8f12"><code>f89b9b8</code></a>
test(derive): Make stable across upgrade</li>
<li><a
href="https://github.com/clap-rs/clap/commit/58eb8a937ac6ca4a59614dc26deedb6cfe16c424"><code>58eb8a9</code></a>
chore: Release</li>
<li><a
href="https://github.com/clap-rs/clap/commit/10a2a7559b0663143d56c850c0c40ed31620cb5b"><code>10a2a75</code></a>
docs: Update changelog</li>
<li><a
href="https://github.com/clap-rs/clap/commit/a42eebf56bf20d587347abb03105f95c98bfda51"><code>a42eebf</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/6103">#6103</a>
from epage/mut_subcommands</li>
<li><a
href="https://github.com/clap-rs/clap/commit/5335f54d73eef9276c13313661fcfffb720c87cf"><code>5335f54</code></a>
feat: Add Command::mut_subcommands</li>
<li>Additional commits viewable in <a
href="https://github.com/clap-rs/clap/compare/clap_complete-v4.5.56...clap_complete-v4.5.57">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=clap&package-manager=cargo&previous-version=4.5.56&new-version=4.5.57)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 11:38:42 +00:00
dependabot[bot] 7eae96bf1d chore(deps): bump sysinfo from 0.38.0 to 0.38.1 (#20261) 
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.38.0
to 0.38.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/GuillaumeGomez/sysinfo/blob/main/CHANGELOG.md">sysinfo's
changelog</a>.</em></p>
<blockquote>
<h1>0.38.1</h1>
<ul>
<li>macOS: Put back support for versions below macOS 12.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/0d91dfb0e34cb7414541ecd574fc2e39ac0ce057"><code>0d91dfb</code></a>
Merge pull request <a
href="https://redirect.github.com/GuillaumeGomez/sysinfo/issues/1622">#1622</a>
from GuillaumeGomez/update</li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/74ae5e97a2e2a42438eb59d47d5d38af120e37f6"><code>74ae5e9</code></a>
Update crate version to <code>0.38.1</code></li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/19cea2ea35aae35c56104fb8ca858a53d05afded"><code>19cea2e</code></a>
Update CHANGELOG for <code>0.38.1</code></li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/45472ee669c3d744c2f673250038a98d3d115e42"><code>45472ee</code></a>
Merge pull request <a
href="https://redirect.github.com/GuillaumeGomez/sysinfo/issues/1621">#1621</a>
from GuillaumeGomez/older-mac-versions</li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/385e819ff44c359312ba2f4e448e92134b480b9e"><code>385e819</code></a>
Enforce version restrictions on <code>objc2-core-foundation</code> and
on <code>objc2-io-kit</code>...</li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/4b07f3eb841ce572acdeddd9e29e285919250ffa"><code>4b07f3e</code></a>
Revert &quot;Update <code>objc2-core-foundation</code> and
<code>objc2-io-kit</code>&quot;</li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/6d11ee1213dde5cc58da1e9fee3152eef10d0761"><code>6d11ee1</code></a>
Merge pull request <a
href="https://redirect.github.com/GuillaumeGomez/sysinfo/issues/1619">#1619</a>
from jterry75/pdh_to_wide</li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/92fd93b600b50c72353bba2e4169bece619bd4ab"><code>92fd93b</code></a>
Move PDH references to W fn's</li>
<li><a
href="https://github.com/GuillaumeGomez/sysinfo/commit/da79fbc7548a5eb36bf237596042a0f030aac393"><code>da79fbc</code></a>
Merge pull request <a
href="https://redirect.github.com/GuillaumeGomez/sysinfo/issues/1618">#1618</a>
from GuillaumeGomez/changelog</li>
<li>See full diff in <a
href="https://github.com/GuillaumeGomez/sysinfo/compare/v0.38.0...v0.38.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sysinfo&package-manager=cargo&previous-version=0.38.0&new-version=0.38.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 11:38:09 +00:00
dependabot[bot] ff15500676 chore(deps): bump setuptools from 80.10.2 to 82.0.0 in /docs (#20255) 
Bumps [setuptools](https://github.com/pypa/setuptools) from 80.10.2 to
82.0.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pypa/setuptools/blob/main/NEWS.rst">setuptools's
changelog</a>.</em></p>
<blockquote>
<h1>v82.0.0</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li><code>pkg_resources</code> has been removed from Setuptools. Most
common uses of <code>pkg_resources</code> have been superseded by the
<code>importlib.resources
&lt;https://docs.python.org/3/library/importlib.resources.html&gt;</code>_
and <code>importlib.metadata
&lt;https://docs.python.org/3/library/importlib.metadata.html&gt;</code>_
projects. Projects and environments relying on
<code>pkg_resources</code> for namespace packages or other behavior
should depend on older versions of <code>setuptools</code>. (<a
href="https://redirect.github.com/pypa/setuptools/issues/3085">#3085</a>)</li>
</ul>
<h1>v81.0.0</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Removed support for the --dry-run parameter to setup.py. This one
feature by its nature threads through lots of core and ancillary
functionality, adding complexity and friction. Removal of this parameter
will help decouple the compiler functionality from distutils and thus
the eventual full integration of distutils. These changes do affect some
class and function signatures, so any derivative functionality may
require some compatibility shims to support their expected interface.
Please report any issues to the Setuptools project for investigation.
(<a
href="https://redirect.github.com/pypa/setuptools/issues/4872">#4872</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pypa/setuptools/commit/03f3615362c4eb19c770b71be5bd58e38f235528"><code>03f3615</code></a>
Bump version: 81.0.0 → 82.0.0</li>
<li><a
href="https://github.com/pypa/setuptools/commit/530d11498af526c4210d8eeaa1ed6c63f44a390f"><code>530d114</code></a>
Merge pull request <a
href="https://redirect.github.com/pypa/setuptools/issues/5007">#5007</a>
from pypa/feature/remove-more-pkg_resources</li>
<li><a
href="https://github.com/pypa/setuptools/commit/11efe9f552290bf536515d458aa85752a0606aa8"><code>11efe9f</code></a>
Merge branch 'maint/75.3'</li>
<li><a
href="https://github.com/pypa/setuptools/commit/118f129dd0fb319058bd05f382c50188fd60a60e"><code>118f129</code></a>
Bump version: 75.3.3 → 75.3.4</li>
<li><a
href="https://github.com/pypa/setuptools/commit/90561ffde1220a590b7644745f48b5837b1a130d"><code>90561ff</code></a>
Merge pull request <a
href="https://redirect.github.com/pypa/setuptools/issues/5150">#5150</a>
from UladzimirTrehubenka/backport_cve_47273</li>
<li><a
href="https://github.com/pypa/setuptools/commit/4595034db8aab4ea33035a47a068b04fd8aa00cc"><code>4595034</code></a>
Add news fragment.</li>
<li><a
href="https://github.com/pypa/setuptools/commit/fc008006fc072af02eb7e0b601172c67eba395e3"><code>fc00800</code></a>
Merge pull request <a
href="https://redirect.github.com/pypa/setuptools/issues/5171">#5171</a>
from cclauss/ruff-v0.15.0</li>
<li><a
href="https://github.com/pypa/setuptools/commit/127e561362a2b4e560faabe9e979ed848106b62d"><code>127e561</code></a>
Remove tests reliant on pkg_resources, rather than xfailing them.</li>
<li><a
href="https://github.com/pypa/setuptools/commit/64bc21e10b5d749b1b75fa334caedb67cc7414c4"><code>64bc21e</code></a>
Reference the superseding libraries.</li>
<li><a
href="https://github.com/pypa/setuptools/commit/cf1ff459ea997b615a75d99304f6c9aa1fc94c06"><code>cf1ff45</code></a>
Merge branch 'main' into debt/pbr-without-pkg_resources</li>
<li>Additional commits viewable in <a
href="https://github.com/pypa/setuptools/compare/v80.10.2...v82.0.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=setuptools&package-manager=pip&previous-version=80.10.2&new-version=82.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 11:37:46 +00:00
dependabot[bot] eca6bfe133 chore(deps): bump taiki-e/install-action from 2.67.18 to 2.67.27 (#20254) 
Bumps
[taiki-e/install-action](https://github.com/taiki-e/install-action) from
2.67.18 to 2.67.27.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/releases">taiki-e/install-action's
releases</a>.</em></p>
<blockquote>
<h2>2.67.27</h2>
<ul>
<li>
<p>Update <code>sccache@latest</code> to 0.14.0.</p>
</li>
<li>
<p>Update <code>typos@latest</code> to 1.43.4.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.2.8.</p>
</li>
</ul>
<h2>2.67.26</h2>
<ul>
<li>
<p>Update <code>tombi@latest</code> to 0.7.28.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.2.7.</p>
</li>
</ul>
<h2>2.67.25</h2>
<ul>
<li>Update <code>cargo-hack@latest</code> to 0.6.42.</li>
</ul>
<h2>2.67.24</h2>
<ul>
<li>
<p>Update <code>cargo-llvm-cov@latest</code> to 0.8.4.</p>
</li>
<li>
<p>Update <code>typos@latest</code> to 1.43.3.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.23.8.</p>
</li>
</ul>
<h2>2.67.23</h2>
<ul>
<li>
<p>Update <code>prek@latest</code> to 0.3.2.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.2.5.</p>
</li>
<li>
<p>Update <code>typos@latest</code> to 1.43.2.</p>
</li>
<li>
<p>Update <code>release-plz@latest</code> to 0.3.155.</p>
</li>
</ul>
<h2>2.67.22</h2>
<ul>
<li>
<p>Update <code>uv@latest</code> to 0.10.0.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.23.7.</p>
</li>
</ul>
<h2>2.67.21</h2>
<ul>
<li>
<p>Update <code>cargo-llvm-cov@latest</code> to 0.8.3.</p>
</li>
<li>
<p>Update <code>trivy@latest</code> to 0.69.1.</p>
</li>
<li>
<p>Update <code>tombi@latest</code> to 0.7.27.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.2.4.</p>
</li>
</ul>
<h2>2.67.20</h2>
<ul>
<li>
<p>Update <code>wasmtime@latest</code> to 41.0.3.</p>
</li>
<li>
<p>Update <code>wash@latest</code> to 0.43.0.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md">taiki-e/install-action's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<p>All notable changes to this project will be documented in this
file.</p>
<p>This project adheres to <a href="https://semver.org">Semantic
Versioning</a>.</p>
<!-- raw HTML omitted -->
<h2>[Unreleased]</h2>
<h2>[2.67.27] - 2026-02-09</h2>
<ul>
<li>
<p>Update <code>sccache@latest</code> to 0.14.0.</p>
</li>
<li>
<p>Update <code>typos@latest</code> to 1.43.4.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.2.8.</p>
</li>
</ul>
<h2>[2.67.26] - 2026-02-09</h2>
<ul>
<li>
<p>Update <code>tombi@latest</code> to 0.7.28.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.2.7.</p>
</li>
</ul>
<h2>[2.67.25] - 2026-02-07</h2>
<ul>
<li>Update <code>cargo-hack@latest</code> to 0.6.42.</li>
</ul>
<h2>[2.67.24] - 2026-02-06</h2>
<ul>
<li>
<p>Update <code>cargo-llvm-cov@latest</code> to 0.8.4.</p>
</li>
<li>
<p>Update <code>typos@latest</code> to 1.43.3.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.23.8.</p>
</li>
</ul>
<h2>[2.67.23] - 2026-02-06</h2>
<ul>
<li>
<p>Update <code>prek@latest</code> to 0.3.2.</p>
</li>
<li>
<p>Update <code>mise@latest</code> to 2026.2.5.</p>
</li>
<li>
<p>Update <code>typos@latest</code> to 1.43.2.</p>
</li>
<li>
<p>Update <code>release-plz@latest</code> to 0.3.155.</p>
</li>
</ul>
<h2>[2.67.22] - 2026-02-06</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/taiki-e/install-action/commit/1e67dedb5e3c590e1c9d9272ace46ef689da250d"><code>1e67ded</code></a>
Release 2.67.27</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/b0cd8d375157b5578bcbd493453903ce77e8033e"><code>b0cd8d3</code></a>
Update <code>sccache@latest</code> to 0.14.0</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/7b52b108a03df51c3ed18df07eee256826458729"><code>7b52b10</code></a>
Update <code>typos@latest</code> to 1.43.4</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/292e521d7cd0961a51f49d78132f997940df0402"><code>292e521</code></a>
tools: Update tidy.sh</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/c9244c19ac8e9989c91d2f8b565c7c8302af0fa8"><code>c9244c1</code></a>
manifest-schema: Disable std prelude</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/672b40bb62b60077928da748150fadc988cfb587"><code>672b40b</code></a>
manifest-schema: Apply clippy to doctest</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/783ca381c0c346662fe3b41ebcfc2d1b43f4ed98"><code>783ca38</code></a>
Update <code>mise@latest</code> to 2026.2.8</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/509565405a8a987e73cf742e26b26dcc72c4b01a"><code>5095654</code></a>
Release 2.67.26</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/c6468a01fd7537b0f4b952a5878828171b8f29e8"><code>c6468a0</code></a>
Update changelog</li>
<li><a
href="https://github.com/taiki-e/install-action/commit/43a18eebd63b96f70f38c92ac232584ec3998c24"><code>43a18ee</code></a>
Update <code>tombi@latest</code> to 0.7.28</li>
<li>Additional commits viewable in <a
href="https://github.com/taiki-e/install-action/compare/650c5ca14212efbbf3e580844b04bdccf68dac31...1e67dedb5e3c590e1c9d9272ace46ef689da250d">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=taiki-e/install-action&package-manager=github_actions&previous-version=2.67.18&new-version=2.67.27)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-02-10 11:37:00 +00:00