mirror of
https://github.com/langchain-ai/datafusion.git
synced 2026-07-21 03:05:28 -04:00
Krisztián Szűcs
b26b2cf17c
Also resolves: - ARROW-11213: [Packaging][Python] Dockerize wheel building on windows - ARROW-11215: [CI] Use named volumes by default for caching in docker-compose - ARROW-11231: [Python][Packaging] Enable mimalloc in wheels Main features: - dockerize windows builds - use vcpkg as the dependency source where we can explicitly pin a working version without worrying about the drag of build environment over time Potential follow-up: - use vcpkg on macos as well cc @kou @xhochy ## Manylinux testing Should be straightforward (I'm going to shared pre-built images for quicker testing). ## Windows testing Only windows host is able to run windows containers. I'm virtualizing windows on macOS (should work on linux as well) using virtualbox: ```bash git clone https://github.com/StefanScherer/windows-docker-machine cd windows-docker-machine # grant more resources in the vagrantfile under the virtualbox section (I use 8 cores and 16GiB of ram) vagrant up --provider virtualbox 2019-box docker context ls docker context use 2019-box docker image ls ``` Now docker should use the windows docker daemon. ```bash pip install -e dev/archery[docker] # the volumes defined in the docker compose file should be commented out and pass arrow's source explicitly # this is required because compose is being executed on a unix host and the directory is mounted through a virtual machine PYTHON=3.6 archery docker run --no-pull --using-docker-cli -v C:$(pwd):C:/arrow python-wheel-windows-vs2017 cmd.exe # then within the container execute a build or do other interactive things arrow\ci\scripts\python_wheel_windown_build.bat ``` Closes #9096 from kszucs/wheel-vcpkg Lead-authored-by: Krisztián Szűcs <szucs.krisztian@gmail.com> Co-authored-by: Sutou Kouhei <kou@clear-code.com> Co-authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>