karuppuchamysuresh
9de192af29
## Which issue does this PR close? - Closes #18314 ## Rationale for this change The documentation in `data_types.md` was outdated and showed `Utf8` as the default mapping for character types (CHAR, VARCHAR, TEXT, STRING), but the current implementation defaults to `Utf8View`. This caused confusion for users reading the documentation as it didn't match the actual behavior. Additionally, the "Supported Arrow Types" section at the end was redundant since `arrow_typeof` now supports all Arrow types, making the comprehensive list unnecessary. ## What changes are included in this PR? 1. **Updated Character Types table**: Changed the Arrow DataType column from `Utf8` to `Utf8View` for CHAR, VARCHAR, TEXT, and STRING types 2. **Added configuration note**: Documented the `datafusion.sql_parser.map_string_types_to_utf8view` setting that allows users to switch back to `Utf8` if needed 3. **Removed outdated section**: Deleted the "Supported Arrow Types" section (39 lines) as it's no longer necessary ## Are these changes tested? This is a documentation-only change. The documentation accurately reflects the current behavior of DataFusion: - The default mapping to `Utf8View` is the current implementation behavior - The `datafusion.sql_parser.map_string_types_to_utf8view` configuration option exists and works as documented ## Are there any user-facing changes? Yes, documentation changes only. Users will now see accurate information about: - The correct default Arrow type mappings for character types - How to configure the string type mapping behavior if they need the old `Utf8` behavior --------- Co-authored-by: Claude (claude-sonnet-4.5) <noreply@anthropic.com>
DataFusion Documentation
This folder contains the source content of the User Guide and Contributor Guide. These are both published to https://datafusion.apache.org/ as part of the release process.
Dependencies
It's recommended to install build dependencies and build the documentation inside a Python virtualenv.
python3 -m venv venv
pip install -r requirements.txt
If using uv the script can be run like so without needing to create a virtual environment:
uv run --with-requirements requirements.txt bash build.sh
The docs build regenerates the workspace dependency graph via
docs/scripts/generate_dependency_graph.sh, so ensure cargo, cargo-depgraph
(cargo install cargo-depgraph --version ^1.6 --locked), and Graphviz dot
(brew install graphviz or sudo apt-get install -y graphviz) are available.
Build & Preview
Run the provided script to build the HTML pages.
# If using venv, ensure you have activated it
./build.sh
The HTML will be generated into a build directory. Open build/html/index.html
in your preferred browser, e.g.
Preview the site on Linux by running this command.
# On macOS
open build/html/index.html
# On Linux with Firefox
firefox build/html/index.html
Making Changes
To make changes to the docs, simply make a Pull Request with your proposed changes as normal. When the PR is merged the docs will be automatically updated.
Release Process
This documentation is hosted at https://datafusion.apache.org/
When the PR is merged to the main branch of the DataFusion
repository, a github workflow which:
- Builds the html content
- Pushes the html content to the
asf-sitebranch in this repository.
The Apache Software Foundation provides https://datafusion.apache.org/, which serves content based on the configuration in .asf.yaml, which specifies the target as https://datafusion.apache.org/.