Summary
The Spanner DBAPI layer (spanner_dbapi) does not pass the timeout parameter when calling _SnapshotBase.execute_sql(), causing all queries to use the gRPC default timeout of 3600 seconds. This prevents DBAPI consumers (SQLAlchemy, Django, raw DBAPI) from controlling per-statement gRPC deadlines.
Background
_SnapshotBase.execute_sql() accepts a timeout parameter that controls the gRPC deadline for the ExecuteStreamingSql RPC. When not provided, it defaults to gapic_v1.method.DEFAULT, which resolves to default_timeout=3600.0 in the gRPC transport layer.
The DBAPI calls execute_sql() in three code paths, none of which pass timeout=:
- Snapshot reads —
cursor._handle_DQL_with_snapshot() calls snapshot.execute_sql(sql, params, param_types, request_options=...) without timeout=
- Transaction reads/writes —
connection.run_statement() calls transaction.execute_sql(sql, params, param_types=..., request_options=...) without timeout=
- Autocommit DML —
cursor._do_execute_update_in_autocommit() calls transaction.execute_sql(sql, params=..., param_types=..., last_statement=True) without timeout=
Since timeout defaults to gapic_v1.method.DEFAULT, which resolves to default_timeout=3600.0 in the gRPC transport layer (services/spanner/transports/base.py), all DBAPI queries have a 3600-second gRPC deadline regardless of the caller's intent.
Timeline
Other execution parameters (request_options, request_priority, transaction_tag, request_tag) were each wired through incrementally via the same pattern: a Connection property plus pass-through in cursor methods. The timeout parameter was not included in any of these additions.
Proposed Change
Add a timeout property to Connection, following the same pattern used for staleness, read_only, and request_priority. Pass timeout= in the three cursor/connection methods that call execute_sql().
Files changed
connection.py — Add self._timeout = None to __init__, add timeout property/setter, pass timeout= in run_statement()cursor.py — Pass timeout= in _handle_DQL_with_snapshot() and _do_execute_update_in_autocommit()
Usage
from google.cloud.spanner_dbapi import connect
conn = connect(instance_id, database_id, project=project)
conn.timeout = 60 # 60-second gRPC deadline for subsequent statements
cursor = conn.cursor()
cursor.execute("SELECT * FROM my_table")This also enables framework integration. The companion change in python-spanner-sqlalchemy can wire this through execution_options:
engine.execution_options(timeout=60)
Related
Summary
The Spanner DBAPI layer (
spanner_dbapi) does not pass thetimeoutparameter when calling_SnapshotBase.execute_sql(), causing all queries to use the gRPC default timeout of 3600 seconds. This prevents DBAPI consumers (SQLAlchemy, Django, raw DBAPI) from controlling per-statement gRPC deadlines.Background
_SnapshotBase.execute_sql()accepts atimeoutparameter that controls the gRPC deadline for theExecuteStreamingSqlRPC. When not provided, it defaults togapic_v1.method.DEFAULT, which resolves todefault_timeout=3600.0in the gRPC transport layer.The DBAPI calls
execute_sql()in three code paths, none of which passtimeout=:cursor._handle_DQL_with_snapshot()callssnapshot.execute_sql(sql, params, param_types, request_options=...)withouttimeout=connection.run_statement()callstransaction.execute_sql(sql, params, param_types=..., request_options=...)withouttimeout=cursor._do_execute_update_in_autocommit()callstransaction.execute_sql(sql, params=..., param_types=..., last_statement=True)withouttimeout=Since
timeoutdefaults togapic_v1.method.DEFAULT, which resolves todefault_timeout=3600.0in the gRPC transport layer (services/spanner/transports/base.py), all DBAPI queries have a 3600-second gRPC deadline regardless of the caller's intent.Timeline
9b7fcd6(PR #6536)timeout=added to_SnapshotBase.execute_sql()2493fa1(PR googleapis/python-spanner#160)execute_sql()withouttimeout=d59d502(PR googleapis/python-spanner#168)run_statement()added — callsexecute_sql()withouttimeout=1a7c9d2(PR googleapis/python-spanner#278)timeout=expanded toread(),partition_read(),partition_query()cd3b950(PR googleapis/python-spanner#475)_handle_DQL_with_snapshot()extracted — still notimeout=ab768e4(PR googleapis/python-spanner#838)request_optionsadded to_handle_DQL_with_snapshot()—timeout=not addedee9662f(PR googleapis/python-spanner#1262)request_tag/transaction_tagadded —timeout=not addedOther execution parameters (
request_options,request_priority,transaction_tag,request_tag) were each wired through incrementally via the same pattern: aConnectionproperty plus pass-through in cursor methods. Thetimeoutparameter was not included in any of these additions.Proposed Change
Add a
timeoutproperty toConnection, following the same pattern used forstaleness,read_only, andrequest_priority. Passtimeout=in the three cursor/connection methods that callexecute_sql().Files changed
connection.py— Addself._timeout = Noneto__init__, addtimeoutproperty/setter, passtimeout=inrun_statement()cursor.py— Passtimeout=in_handle_DQL_with_snapshot()and_do_execute_update_in_autocommit()Usage
This also enables framework integration. The companion change in
python-spanner-sqlalchemycan wire this throughexecution_options:Related
timeout=onexecute_sql(): PR chore: add samples for transaction timeout configuration python-spanner#1380 (spanner_set_statement_timeoutsample), PR feat: add retry and timeout for batch dml python-spanner#1107 (spanner_set_custom_timeout_and_retrysample)