-
Notifications
You must be signed in to change notification settings - Fork 714
bug: DuckDB backend returns pyarrow.RecordBatchReader instead of pyarrow.Table, breaking .execute() #11632
Copy link
Copy link
Closed
Labels
bugIncorrect behavior inside of ibisIncorrect behavior inside of ibis
Description
What happened?
When using the DuckDB backend in Ibis with recent versions of DuckDB (≥1.4.0), calling .execute() on a table expression can fail because the backend returns a pyarrow.RecordBatchReader.
Ibis currently assumes the result is a pyarrow.Table and accesses attributes like .column_names, which RecordBatchReader does not have. This leads to:
AttributeError: 'pyarrow.lib.RecordBatchReader' object has no attribute 'column_names'How to reproduce
- Install latest (will reproduce the issue with DuckDB ≥1.4.0)
pip install "ibis-framework[duckdb]" duckdb pyarrow- Run the following example
import ibis
# Connect via Ibis DuckDB backend
con = ibis.duckdb.connect(database=":memory:")
# Create an in-memory table
data = ibis.memtable({"col1": [1, 2], "col2": [3, 4]})
con.create_table("t", data)
# Load the table back
t = con.table("t")
# Attempt to execute
result = t.execute() # Fails with AttributeError when DuckDB >=1.4.0What version of ibis are you using?
ibis-framework==10.8.0
What backend(s) are you using, if any?
duckdb==1.4.0 (regression not seen with <1.4.0)
Relevant log output
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/opt/miniconda3/envs/test-ibis-env/lib/python3.11/site-packages/ibis/expr/types/core.py", line 424, in execute
return self._find_backend(use_default=True).execute(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/opt/miniconda3/envs/test-ibis-env/lib/python3.11/site-packages/ibis/backends/duckdb/__init__.py", line 1415, in execute
for name, col in zip(table.column_names, table.columns)
^^^^^^^^^^^^^^^^^^
AttributeError: 'pyarrow.lib.RecordBatchReader' object has no attribute 'column_names'Code of Conduct
- I agree to follow this project's Code of Conduct
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugIncorrect behavior inside of ibisIncorrect behavior inside of ibis
Type
Projects
Status
done