Commit Graph

39 Commits

Author SHA1 Message Date
Adrian Garcia Badaracco 1ee782f783 Migrate Python usage to uv workspace (#20414)
I was having trouble getting benchmarks to gen data.

## Summary
- Replace three independent `requirements.txt` files with a uv workspace
(`benchmarks`, `dev`, `docs` projects)
- Single `uv.lock` lockfile for reproducible dependency resolution
- Simplify `bench.sh` by removing all ad-hoc venv/pip logic in favor of
`uv run`

## Test plan
- [ ] `uv sync` resolves all deps from repo root
- [ ] `uv run --project benchmarks python3 benchmarks/compare.py` works
- [ ] `uv run --project docs sphinx-build docs/source docs/build` builds
docs
- [ ] Run a benchmark from `bench.sh` that uses Python (e.g., h2o data
gen or compare flow)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 16:29:56 +00:00
Nuno Faria 92f60ad513 docs: Automatically update DataFusion version in docs (#20001)
## Which issue does this PR close?

- N/A.

## Rationale for this change

I was looking at the download page
(https://datafusion.apache.org/download.html) when I noticed the version
there was not updated in a while. Likewise, there are other places in
the docs where the version is not updated.

This PR changes the `update_datafusion_versions.py` script to
automatically update those files, in addition to the Cargo files.

## What changes are included in this PR?

- Updated `update_datafusion_versions.py` to check some doc files.
- Updated the doc files to the current major version.

## Are these changes tested?

Yes, script runs correctly.

## Are there any user-facing changes?

No.

---------

Co-authored-by: Martin Grigorov <martin-g@users.noreply.github.com>
2026-01-29 03:02:37 +00:00
Andrew Lamb d1eea0755a Fix broken links in the documentation (#19964)
## Which issue does this PR close?
- Similarly to https://github.com/apache/datafusion/pull/19958

## Rationale for this change
I found some broken links in our docs using codex
]
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->

## What changes are included in this PR?
Update broken links (will comment inline) 

## Are these changes tested?

By CI

## Are there any user-facing changes?

Better docs / fewer broken links

---------

Co-authored-by: Martin Grigorov <martin-g@users.noreply.github.com>
2026-01-25 22:02:46 +00:00
Edmondo Porcu f4e519f9df Move min and max to user defined aggregate function, remove AggregateFunction / AggregateFunctionDefinition::BuiltIn (#11013)
* Moving min and max to new API and removing from protobuf

* Using input_type rather than data_type

* Adding type coercion

* Fixed doctests

* Implementing feedback from code review

* Implementing feedback from code review

* Fixed wrong name

* Fixing name
2024-08-03 19:18:10 +08:00
Andrew Lamb 0b2eb50c0f Move configuration information out of example usage page (#11300) 2024-07-11 07:58:53 -07:00
张林伟 2f02c43b62 Fix typos in datafusion-examples/datafusion-cli/docs (#11259)
* Add typos check in ci workflow

* typos.toml

* exclude more

* typos autofix

* remove typos ci
2024-07-08 06:00:11 -04:00
Andrew Lamb 03848c52f0 Move basic SQL query examples to user guide (#11217) 2024-07-03 14:43:36 -04:00
hsiang-c ccf395f40c (Doc) Enable rt-multi-thread feature for sample code (#10770) 2024-06-03 14:46:45 -04:00
tison 5deefd5a34 docs: add download page (#10271)
* docs: add download page

Signed-off-by: tison <wander4096@gmail.com>

* Apply suggestions from code review

Co-authored-by: Phillip LeBlanc <phillip@leblanc.tech>

* Apply suggestions from code review

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: Phillip LeBlanc <phillip@leblanc.tech>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2024-04-30 09:27:10 -04:00
comphead 5d44685794 Doc: Modify docs to fix old naming (#10199)
* fix docs on datafusion names

* fix code

* fmt

* Update dev/release/README.md

Co-authored-by: Andy Grove <andygrove73@gmail.com>

* Update dev/release/README.md

Co-authored-by: Andy Grove <andygrove73@gmail.com>

* Update dev/release/README.md

Co-authored-by: Andy Grove <andygrove73@gmail.com>

---------

Co-authored-by: Andy Grove <andygrove73@gmail.com>
2024-04-23 12:12:57 -07:00
张林伟 465c89f7f1 Update github repo links (#10167)
* Update github repo link

* Format markdown

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2024-04-22 11:11:31 -06:00
comphead 0b5bfe2cfe Add test example for backtraces (#10143) 2024-04-22 08:38:21 -07:00
colommar feb9100432 fix#9501 (#10028) 2024-04-11 14:55:25 -04:00
comphead edaf235828 doc: Updated known users list and usage dependency description (#9718)
* minor: update known users and usage description
2024-03-21 08:12:18 -07:00
Bruce Ritchie c0a21b28c7 Update example-usage.md to remove reference to simd and rust nightly. (#9677)
* Fix to_timestamp benchmark

* Remove reference to simd and nightly build as simd is no longer an available feature in DataFusion and building with nightly may not be a good recommendation when getting started.
2024-03-18 14:23:16 -04:00
Chih Wang ae88235617 Fix sphinx warnings (#9142)
* Fix 'Configuration Settings' link in write_options.md

* Fix multiple reference definition in scalar_functions.md

* Fix typo: 'contcat_ws' to 'concat_ws' in scalar_functions.md

* Add missing section 'array_contains' in scalar_functions.md

* Fix reference warnings in operators.md

* Remove unsupported syntax 'csv' from code blocks of adding-udfs.md

* Fix syntax tag in example-usage.md

* Change syntax tag of code blocks that contain multiple syntaxes of
cli.md

* Remove 'sql' syntax tags from code blocks

* Remove leading ❯ in sql code blocks
2024-02-09 16:01:26 -05:00
comphead c843226d84 doc: Add example how to include latest datafusion (#9076)
* doc: Add example how to include latest datafusion

* rm unused block

* Update docs/source/user-guide/example-usage.md

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2024-01-31 13:51:58 -08:00
Raphael Taylor-Davies acf0f78987 Upgrade to object_store 0.9.0 and arrow 50.0.0 (#8758)
* Prepare object_store 0.9.0

* Update test

* Update to arrow 50.0.0

* Update sqllogictest

* Update sqllogictests

* Format

* Use nullif

* Use released version of arrow-rs

* Update README to remove references to SIMD

* unpatch datafusion-cli

* Adjust memory sizes in tests

* Restore test without explicit region

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2024-01-14 16:42:51 +00:00
Chih Wang 5e9dd44edd Minor: Change all file links to GitHub in document (#7768) 2023-10-08 15:17:41 +08:00
Andrew Lamb b1587c1149 Document crate feature flags (#7713)
* Document crate feature flags

* prettier
2023-10-03 06:01:09 -04:00
comphead 9b45ed992a Adding backtrace documentation (#7628) 2023-09-23 06:03:15 -04:00
comphead c2b2606207 Minor: Update user guide (#6692) 2023-06-16 13:16:21 +08:00
Andrew Lamb 8336a5823c Improve getting started guide, add note about compatible arrow versions (#6472)
* Improve main readme

* update links

* Update docs

* prettier

* prettier

* Update docs/source/user-guide/example-usage.md

* Add mention to parquet and add examples
2023-05-30 15:11:05 -04:00
Kirill Zaborsky cbdf4cda45 Bump snmalloc version in the example usage doc (#6344)
Follow snmalloc upgrade in 9.0.0
2023-05-12 10:48:35 -04:00
Yijie Shen a55d805e55 minor: fix doctest that runs accidentally (#5989)
* minor: fix doctest that runs accidentally

* enable doctest in ci for docs/**

* separate yaml for doctest
2023-04-13 20:12:20 +08:00
Andrew Lamb 4c7833ebfd [DOCS]: consolidate doc site content simplify navbar (#5962)
* [DOCS]: consolidate doc site content simplify navbar

* prettier

* Update docs/source/user-guide/faq.md

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* Update versions to latest

* remove reundant example

* update duckdb link and polars description

* update velox link

* prettier

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2023-04-12 16:22:49 -04:00
Jeffrey 4d07360e6f Support catalog.schema.table.column in SQL SELECT and WHERE (#5343)
* Support catalog.schema.table.column in SQL SELECT and WHERE

* Merge branch 'main' into support_catalog_schema_in_ident

* Update column new() docstring

* Add tests for dfschema search

* Introduce DFField::new_unqualified

* Introduce new_unqualified methods for simpler syntax

* Fix merge

* New ident() expr function

* Refactor OwnedTableReference to be a type alies of TableReference<'static>

* Update comments

* Comments

* Update docstrings

* From OwnedTableReference to TableReference impl
2023-03-14 10:50:10 -04:00
Chih Wang fd75ba461d Make SQL query consistent with API syntax expression in code examples (#5303)
* Update code example

Make SQL query consistent with API syntax expression.

* Update example in core/src/lib.rs
2023-02-16 13:44:36 -05:00
Andrew Lamb 664edea4ec Minor: Move test data into datafusion/core/tests/data (#4855)
* Minor: Move test data into tests/data
2023-01-09 12:10:52 +08:00
Martin Grigorov 049690412c Use limit() function instead of show_limit() in the first example (#4369)
Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>

Signed-off-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
2022-11-27 06:19:48 -05:00
Andrew Lamb 85e8fe1343 Clarify initial example with respect to capitalization (#3815) 2022-10-12 19:42:19 -04:00
nbr d5e6736dd2 Improve doc on lowercase treatment of columns on SQL (#3385)
* Improve doc

Adds specific note on lowercase treatment of columns on SQL

This example is the first thing newcomers see when they start with datafusion (literally the second page on the docs) so it should be clearer.

* Update select.md

add note on the use of default lower-case for SQL query

* prettier this

* improve example file link

* add capitalized example
2022-10-12 16:42:20 -04:00
Andy Grove 8ee31cc69f Prepare for 11.0.0 release (#3171)
* minor docs updates

* bump crate versions

* Update docs/source/user-guide/sql/ddl.md

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* CHANGELOG

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-08-16 09:22:49 -06:00
Andy Grove 5d52b32a7d Prepare for datafusion 8.0.0 , ballista 0.7.0 release (#2490)
* bump versions

* changelog 7.0.0 to 8.0.0

* update versions in docs

* revert changelog

* re-generate changelogs

* revert ballista changelog

* update version missed by script and update script

* regenerate ballista changelog with correct version
2022-05-12 17:13:05 -06:00
mingmwang 2d6addd4c4 Refactor SessionContext, BallistaContext to support multi-tenancy configurations - Part 3 (#2091)
* Refactor SessionContext, BallistaContext to support multi-tenancy configurations - Part 3

resolve review comments

* fix fmt

* fix check
2022-03-28 08:38:57 -04:00
mingmwang 4994eda81c Refactor ExecutionContext and related conf to support multi-tenancy configurations - Part 1 (#1987)
Co-authored-by: Wang <mingmwang@lm-shc-16503062.corp.ebay.com>
2022-03-16 17:50:38 +08:00
Andrew Lamb 2a5a43195c Update documentation example for change in API (#1812)
* Update documentation example for change in API

* Update docs/source/user-guide/example-usage.md

* Update docs/source/user-guide/example-usage.md

Co-authored-by: xudong.w <wxd963996380@gmail.com>

Co-authored-by: xudong.w <wxd963996380@gmail.com>
2022-02-16 14:35:13 -05:00
Andrew Lamb de3eb2ab47 Remove unused use statements from examples (#1032) 2021-09-21 18:01:57 -07:00
QP Hou dcd34c6679 consolidate datafusion docs with sphinx (#993)
* consolidate datafusion docs with sphinx
* added python doc
* combined, cli, user-guide and specification docs into a single datafusion doc

Co-authored-by: Jiayu Liu <Jimexist@users.noreply.github.com>
2021-09-14 22:16:31 -07:00