4 Commits

Author SHA1 Message Date
Ethan Urbanski 1724f89548 chore: python datafusion 52 upgrade (#4226)
# Description
Upgrades the Python DataFusion path to 52.x and makes the integration
lane blocking in CI

# Related Issue(s)
<!---
For example:

- closes #106
--->

# Documentation

<!---
Share links to useful documentation
--->

---------

Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>
2026-02-24 01:19:58 +01:00
Ethan Urbanski 5121983552 fix(python): guard DataFusion FFI export on datafusion major version (#4142)
# Description

- Add a runtime version check in __datafusion_table_provider__ to
prevent FFI ABI mismatch segfaults
- Block capsule export when installed datafusion major != 52
- Provide actionable error text with QueryBuilder workaround

Changes:

- lib.rs: add REQUIRED_DATAFUSION_PY_MAJOR, datafusion_python_version(),
guard at method start
- test_datafusion.py: add incompatible version and not installed tests

Note: This guard is a temporary safety net to prevent segfaults until
DataFusion 52 Python wheels are available on PyPI. Once wheels land,
users can install datafusion==52.* and use SessionContext registration
normally.

# Related Issue(s)
- #4135 
<!---
For example:

- closes #106
--->

# Documentation

<!---
Share links to useful documentation
--->

Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>
2026-01-28 23:14:57 +01:00
Ethan Urbanski f40d60e62a fix(datafusion): migrate delta scan plumbing to DF52 APIs
Remove SchemaAdapter usage and switch scans to DF52 TableSchema + updated FileScanConfigBuilder.

Update ColumnStatistics literals and align stats expectations with DF52 precision.

Adjust ops session handling and python datafusion glue for DF52.

Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>
2026-01-26 07:03:37 -08:00
Ion Koutsouris 2ad40c6d6c fix: wrap table provider with block_on in scan for python (#4072)
# Description
- closes https://github.com/delta-io/delta-rs/issues/4063

Signed-off-by: Ion Koutsouris
Signed-off-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com>
2026-01-12 22:45:52 +01:00