datafusion

mirror of https://github.com/langchain-ai/datafusion.git synced 2026-07-18 13:15:59 -04:00

Files

T

Neil Conway 585bbf35d3 perf: Optimize array_has_any() with scalar arg (#20385 )

## Which issue does this PR close?

- Closes #20384.
- See #18181 for related context.

## Rationale for this change

When `array_has_any` is passed a scalar for either of its arguments, we
can use a much faster algorithm: rather than doing O(N*M) comparisons
for each row of the columnar arg, we can build a hash table on the
scalar argument and probe it instead.

## What changes are included in this PR?

* Add benchmark to cover the one-scalar-arg case
* Implement optimization as described above

Note that we fallback to a linear scan when the scalar arg is smaller
than a threshold (<= 8 elements), because benchmarks suggested probing a
HashSet is not profitable for very small arrays.

## Are these changes tested?

Yes. Tests pass and benchmarked.

## Are there any user-facing changes?

No.

---------

Co-authored-by: Martin Grigorov <martin-g@users.noreply.github.com>
Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>

2026-02-24 20:59:08 +00:00

_static

minor: Add favicon (#20000 )

2026-01-26 05:19:56 +00:00

_templates

feat(docs): enable navbar (#18324 )

2025-10-31 02:14:57 +00:00

contributor-guide

Runs-on for more actions (#20274 )

2026-02-23 10:04:37 +00:00

library-user-guide

Cache PlanProperties, add fast-path for with_new_children (#19792 )