Skip to content

Commit b72df7b

Browse files
authored
Increase scheduler loop sleep in dag.test for executors (#58739)
If we are using a real executor, those often make some requests or describe calls to an external service to track the progress of the running tasks. Doing that every 1s is way too aggressive, especially given that these are just tests.
1 parent da9bb78 commit b72df7b

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

  • task-sdk/src/airflow/sdk/definitions

task-sdk/src/airflow/sdk/definitions/dag.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1316,7 +1316,13 @@ def test(
13161316
ids_unrunnable = {x for x in all_tis if x.state not in FINISHED_STATES} - scheduled_tis
13171317
if not scheduled_tis and ids_unrunnable:
13181318
log.warning("No tasks to run. unrunnable tasks: %s", ids_unrunnable)
1319-
time.sleep(1)
1319+
if use_executor:
1320+
# If we are using a real executor, those often make some request or describe call to
1321+
# an external service to track the progress of the running tasks. Doing that every 1s
1322+
# is way too aggressive, especially given that these are just tests.
1323+
time.sleep(30)
1324+
else:
1325+
time.sleep(1)
13201326

13211327
for ti in scheduled_tis:
13221328
task = self.task_dict[ti.task_id]

0 commit comments

Comments
 (0)