Skip to content

Fix nested non-Dag with-statement exiting the Dag context in version inflation checker - #68795

Merged
jedcunningham merged 4 commits into
apache:mainfrom
kalluripradeep:fix/dag-version-checker-nested-with-exits-dag-context
Aug 21, 2026
Merged

jedcunningham merged 4 commits into
apache:mainfrom
kalluripradeep:fix/dag-version-checker-nested-with-exits-dag-context

Conversation

@kalluripradeep

@kalluripradeep kalluripradeep commented Jun 21, 2026 •

Copy link
Copy Markdown
Contributor

Closes #68796

What's the problem?

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context() unconditionally at the end of every with-statement. When a non-DAG with-statement (e.g. with open(...) as f) was nested inside a DAG with-block, it would reset is_in_dag_context=False. Any tasks constructed after the nested with — but still inside the DAG block — were invisible to the checker and never flagged for runtime-varying values even when they used datetime.now(), random(), etc.

with DAG('my_dag') as dag:
    with open('file.txt') as f:   # <-- this incorrectly exited DAG context
        data = f.read()
    # BUG: datetime.now() not flagged — checker thinks we're outside DAG
    t1 = BashOperator(task_id='t', bash_command=str(datetime.now()))

Root cause

# visit_With (before fix) — unconditional, wrong
self.dag_detector.exit_dag_context()

Fix

Guard the call with if is_with_dag_context so only the with-statement that actually entered the DAG context exits it.

if is_with_dag_context:
    self.dag_detector.exit_dag_context()

Changes

  • airflow-core/src/airflow/utils/dag_version_inflation_checker.py — one-line guard
  • airflow-core/tests/unit/utils/test_dag_version_inflation_checker.py — 4 regression tests

…ersion inflation checker

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context()
unconditionally at the end of every with-statement. When a non-DAG
with-statement (e.g. `with open(...) as f`) was nested inside a DAG
with-block, it would set is_in_dag_context=False, causing any tasks
constructed AFTER the nested with — but still inside the DAG block —
to be invisible to the checker. Those tasks were never flagged for
runtime-varying values even when they used datetime.now(), random(), etc.

Fix: guard the exit_dag_context() call with `if is_with_dag_context`
so only the with-statement that actually entered the DAG context exits it.

Regression tests added for:
- nested non-DAG with followed by a task (primary case)
- multiple successive nested non-DAG withs
- task constructed inside a nested non-DAG with
- task outside the DAG block not over-flagged after the fix
@potiuk potiuk added the ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green. label Jun 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Aug 12, 2026
@github-actions github-actions Bot closed this Aug 19, 2026
@jedcunningham jedcunningham reopened this Aug 20, 2026
@eladkal eladkal removed the stale Stale PRs per the .github/workflows/stale.yml policy file label Aug 20, 2026
@eladkal eladkal added this to the Airflow 3.3.2 milestone Aug 20, 2026
@kalluripradeep kalluripradeep changed the title fix: nested non-DAG with-statement prematurely exits DAG context in version inflation checker Fix nested non-Dag with-statement exiting the Dag context in version inflation checker Aug 20, 2026
@jedcunningham
jedcunningham merged commit c1fcbef into apache:main Aug 21, 2026
155 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-3-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 21, 2026
… in version inflation checker (apache#68795)

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context()
unconditionally at the end of every with-statement. When a non-DAG
with-statement (e.g. `with open(...) as f`) was nested inside a DAG
with-block, it would set is_in_dag_context=False, causing any tasks
constructed AFTER the nested with — but still inside the DAG block —
to be invisible to the checker. Those tasks were never flagged for
runtime-varying values even when they used datetime.now(), random(), etc.

Fix: guard the exit_dag_context() call with `if is_with_dag_context`
so only the with-statement that actually entered the DAG context exits it.

Regression tests added for:
- nested non-DAG with followed by a task (primary case)
- multiple successive nested non-DAG withs
- task constructed inside a nested non-DAG with
- task outside the DAG block not over-flagged after the fix
(cherry picked from commit c1fcbef)

Co-authored-by: Pradeep Kalluri <128097794+kalluripradeep@users.noreply.github.com>
aws-airflow-bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 21, 2026
… in version inflation checker (apache#68795)

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context()
unconditionally at the end of every with-statement. When a non-DAG
with-statement (e.g. `with open(...) as f`) was nested inside a DAG
with-block, it would set is_in_dag_context=False, causing any tasks
constructed AFTER the nested with — but still inside the DAG block —
to be invisible to the checker. Those tasks were never flagged for
runtime-varying values even when they used datetime.now(), random(), etc.

Fix: guard the exit_dag_context() call with `if is_with_dag_context`
so only the with-statement that actually entered the DAG context exits it.

Regression tests added for:
- nested non-DAG with followed by a task (primary case)
- multiple successive nested non-DAG withs
- task constructed inside a nested non-DAG with
- task outside the DAG block not over-flagged after the fix
(cherry picked from commit c1fcbef)

Co-authored-by: Pradeep Kalluri <128097794+kalluripradeep@users.noreply.github.com>
jedcunningham pushed a commit that referenced this pull request Aug 25, 2026
… in version inflation checker (#68795) (#71956)

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context()
unconditionally at the end of every with-statement. When a non-DAG
with-statement (e.g. `with open(...) as f`) was nested inside a DAG
with-block, it would set is_in_dag_context=False, causing any tasks
constructed AFTER the nested with — but still inside the DAG block —
to be invisible to the checker. Those tasks were never flagged for
runtime-varying values even when they used datetime.now(), random(), etc.

Fix: guard the exit_dag_context() call with `if is_with_dag_context`
so only the with-statement that actually entered the DAG context exits it.

Regression tests added for:
- nested non-DAG with followed by a task (primary case)
- multiple successive nested non-DAG withs
- task constructed inside a nested non-DAG with
- task outside the DAG block not over-flagged after the fix
(cherry picked from commit c1fcbef)

Co-authored-by: Pradeep Kalluri <128097794+kalluripradeep@users.noreply.github.com>
vatsrahul1001 pushed a commit that referenced this pull request Sep 7, 2026
… in version inflation checker (#68795) (#71956)

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context()
unconditionally at the end of every with-statement. When a non-DAG
with-statement (e.g. `with open(...) as f`) was nested inside a DAG
with-block, it would set is_in_dag_context=False, causing any tasks
constructed AFTER the nested with — but still inside the DAG block —
to be invisible to the checker. Those tasks were never flagged for
runtime-varying values even when they used datetime.now(), random(), etc.

Fix: guard the exit_dag_context() call with `if is_with_dag_context`
so only the with-statement that actually entered the DAG context exits it.

Regression tests added for:
- nested non-DAG with followed by a task (primary case)
- multiple successive nested non-DAG withs
- task constructed inside a nested non-DAG with
- task outside the DAG block not over-flagged after the fix
(cherry picked from commit c1fcbef)

Co-authored-by: Pradeep Kalluri <128097794+kalluripradeep@users.noreply.github.com>
imrichardwu pushed a commit to imrichardwu/airflow that referenced this pull request Sep 11, 2026
…inflation checker (apache#68795)

AirflowRuntimeVaryingValueChecker.visit_With() called exit_dag_context()
unconditionally at the end of every with-statement. When a non-DAG
with-statement (e.g. `with open(...) as f`) was nested inside a DAG
with-block, it would set is_in_dag_context=False, causing any tasks
constructed AFTER the nested with — but still inside the DAG block —
to be invisible to the checker. Those tasks were never flagged for
runtime-varying values even when they used datetime.now(), random(), etc.

Fix: guard the exit_dag_context() call with `if is_with_dag_context`
so only the with-statement that actually entered the DAG context exits it.

Regression tests added for:
- nested non-DAG with followed by a task (primary case)
- multiple successive nested non-DAG withs
- task constructed inside a nested non-DAG with
- task outside the DAG block not over-flagged after the fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green. type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AirflowRuntimeVaryingValueChecker misses runtime-varying values in tasks after nested non-DAG with-statements

4 participants