This repository was archived by the owner on Mar 6, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Mar 6, 2026. It is now read-only.
pyarrow compatibility issue when instantiating a client #890
Description
Creating an SDK client using v2.24.1 in Databricks runtime v8.3 (Spark v3.1.1 and Python v3.8.8 and pyarrow v1.0.1) generates the following error: module 'pyarrow' has no attribute 'decimal256'.
pip version = 20.2.4
I tried:
- Upgraded to Databricks runtime v9.0 which brings Python down to v3.8.1 and pyarrow up to v4. Same error as above and below.
- Reverted back to Databricks runtime v8.3 (see specs above). Then, I downgraded BigQuery SDK to v2.24.0. This resolved the error.
Activity
plamut commentedon Aug 23, 2021
@dpmcclurg Which
pyarrowversion do you have installed? What's the output of the following?As of BigQuery client version v2.24.1 released last week,
pyarrow >= 3.0.0is required.Has the
pyarrowdependency been upgraded? I would check that first, and upgrade if necessary.tswast commentedon Sep 1, 2021
This issue has also been reported in Beam and TFX.
Closing as a duplicate of #933