## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #https://github.com/apache/datafusion/issues/19294.
## Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
---------
Co-authored-by: Sergey Zhukov <szhukov@aligntech.com>
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #https://github.com/apache/datafusion/issues/19141.
## Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
---------
Co-authored-by: Sergey Zhukov <szhukov@aligntech.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #.
## Rationale for this change
A small fix-up to run
```sh
cargo run --example relation_planner
```
instead of
```sh
cargo run --example relation_planner -- all
```
If no arguments are specified, run all examples. It conforms to the
expected behaviour of cargo examples.
Extracted from to
https://github.com/apache/datafusion/pull/19311#discussion_r2645925190
## What changes are included in this PR?
## Are these changes tested?
Manual tests.
## Are there any user-facing changes?
No
## Which issue does this PR close?
Related to #16756
## Rationale for this change
The existing `sql_dialect.rs` example demonstrates `COPY ... STORED AS
...`, which is actually already fully supported by the standard
`DFParser`.
This PR replaces it with the example from #16756: `CREATE EXTERNAL
CATALOG ... STORED AS ... LOCATION ...` with automatic table discovery.
## What changes are included in this PR?
The first commit updates `dialect.rs` to show that `DFParser` already
handles `COPY ... STORED AS`, making it clear this syntax doesn't need
customization.
Example output from `cargo run --example sql_ops -- dialect`:
```
Query: COPY source_table TO 'file.fasta' STORED AS FASTA
--- Parsing without extension ---
Standard DFParser: Parsed as Statement::CopyTo: COPY source_table TO file.fasta STORED AS FASTA
--- Parsing with extension ---
Custom MyParser: Parsed as MyStatement::MyCopyTo: COPY source_table TO 'file.fasta' STORED AS FASTA
```
The second commit adds a new `custom_sql_parser.rs` example that
implements `CREATE EXTERNAL CATALOG my_catalog STORED AS <format>
LOCATION '<url>'` with automatic table discovery from object storage. It
also removes the old `dialect.rs` example.
## Are these changes tested?
Yes, the new example is runnable with `cargo run --example sql_ops --
custom_sql_parser` and demonstrates the full flow from parsing custom
DDL through registering the catalog to querying discovered tables.
Example output:
```
=== Part 1: Standard DataFusion Parser ===
Parsing: CREATE EXTERNAL CATALOG parquet_testing
STORED AS parquet
LOCATION 'local://workspace/parquet-testing/data'
OPTIONS (
'schema_name' = 'staged_data',
'format.pruning' = 'true'
)
Error: SQL error: ParserError("Expected: TABLE, found: CATALOG at Line: 1, Column: 17")
=== Part 2: Custom Parser ===
Parsing: CREATE EXTERNAL CATALOG parquet_testing
STORED AS parquet
LOCATION 'local://workspace/parquet-testing/data'
OPTIONS (
'schema_name' = 'staged_data',
'format.pruning' = 'true'
)
Target Catalog: parquet_testing
Data Location: local://workspace/parquet-testing/data
Resolved Schema: staged_data
Registered 69 tables into schema: staged_data
Executing: SELECT id, bool_col, tinyint_col FROM parquet_testing.staged_data.alltypes_plain LIMIT 5
+----+----------+-------------+
| id | bool_col | tinyint_col |
+----+----------+-------------+
| 4 | true | 0 |
| 5 | false | 1 |
| 6 | true | 0 |
| 7 | false | 1 |
| 2 | true | 0 |
+----+----------+-------------+
```
## Are there any user-facing changes?
Documentation only. I replaced the `sql_dialect.rs` example with
`custom_sql_parser.rs` and updated the README. No API changes.
## Which issue does this PR close?
Closes https://github.com/apache/datafusion/issues/15804
## Rationale for this change
Now that we are on MSRV 1.88 we can use rust edition 2024, which brings
let chains and other nice features. It also improves `unsafe` checking.
In order to introduce these changes in slower way instead of one massive
PR that is too difficult to manage we are updating a few crates at a
time.
## What changes are included in this PR?
Updates these crates to 2024.
- datafusion-benchmarks
- datafusion-ffi
- datafusion-sqllogictest
- datafusion-examples
## Are these changes tested?
Existing unit tests. There are no functional code changes.
## Are there any user-facing changes?
None.
## Note
It is recommended to review with the ignore whitespace setting:
https://github.com/apache/datafusion/pull/19361/files?w=1
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #https://github.com/apache/datafusion/issues/19126.
## Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
---------
Co-authored-by: Sergey Zhukov <szhukov@aligntech.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes #https://github.com/apache/datafusion/issues/18512.
## Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
---------
Co-authored-by: Sergey Zhukov <szhukov@aligntech.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- part of #https://github.com/apache/datafusion/issues/18142.
## Rationale for this change
This PR is for consolidating all the `sql_ops` examples (analysis,
dialect, frontend, query) into a single example binary. We are agreed on
the pattern and we can apply it to the remaining examples
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
---------
Co-authored-by: Sergey Zhukov <szhukov@aligntech.com>
Sergey Zhukov