The additional benefit of running these tests in parallel is that more
racey/timing related test failures are cropping up for me.
Signed-off-by: R. Tyler Croy <rtyler@brokenco.de>
FREEZE!
Looks like we were _really_ close to full multi-threaded support before,
and just needed to sprinkle a little macro magic on the pyclass
definition.
See [pyo3 docs](https://pyo3.rs/v0.23.0/class/thread-safety)
Fixes#3594
Signed-off-by: R. Tyler Croy <rtyler@brokenco.de>
This change introduces an internal Mutex inside of RawDeltaTable which
allows the PyO3 bindings to share the Python object between threads at
the Python layer.
PyO3 will raise a `RuntimeError: Already borrowed` for any
function call which takes a mutable reference to `self`. Introducing the
internal Mutex ensures that all function signatures can operate with
just self-references safely.
The Rust-level Mutex is a simple passthrough for most operations which
do not need to modify the underlying state. The critical sections which
typically need to acquire and mutate with a lock are after I/O bound
operations are completed as far as I can tell, so I don't anticipate
deadlock or performance issues.
There is still some cleanup of errors that needs to happen to make the
code here more ergonomic when blending DeltaError with PoisonError from
the lock, as such right now there's a lot of ugly error mapping.
Fixes#2958
Signed-off-by: R. Tyler Croy <rtyler@brokenco.de>
Sponsored-by: Neuralink Corp.
Robert Pack