Repository navigation
fix(worker): rollback DB session on failed log context population - #2574
Open
sentry[bot] wants to merge 1 commit into
Open
sentry[bot] wants to merge 1 commit into
sentry[bot] wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2574 +/- ##
==========================================
- Coverage 91.64% 91.64% -0.01%
==========================================
Files 1341 1341
Lines 53339 53347 +8
Branches 1649 1649
==========================================
+ Hits 48885 48891 +6
- Misses 4133 4135 +2
Partials 321 321
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
Contributor
Author
|
The Sentry GitHub App installation for this repository is missing permissions it needs to keep iterating on this pull request to get CI passing. Review and accept the updated permissions to let Seer continue: https://github.com/organizations/codecov/settings/installations/86101127/permissions/update |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In
BaseCodecovTask.run, before executingrun_impl,LogContext.populate_from_sqlalchemyqueries the database for repository and owner metadata to enrich task logs. If the database connection drops during this query (e.g.psycopg2.OperationalError: server closed the connection unexpectedly), the exception was caught and logged withlog.exception("Failed to populate log context"), but thedb_sessionwas left with an invalidated transaction.Because the thread-scoped session is reused across the task, the task's first database query in
run_impl(such asCommitUpdate'scommits.first()) would subsequently fail withsqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back.Changes
apps/worker/helpers/log_context.py, add a guardeddbsession.rollback()within theexceptblock ofpopulate_from_sqlalchemyso any invalidated transaction state is cleared beforerun_implruns.query.first()returningNonebefore tuple unpacking when a repository or owner record is missing, avoiding unnecessaryTypeErrors during log population.Legal Boilerplate
Look, I get it. The entity doing business as "Codecov" is owned by Harness, Inc. In 2026 Harness acquired Codecov and as a result Harness is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Harness can use, modify, copy, and redistribute my contributions, under Harness's choice of terms.
Fixes WORKER-ZAW
@sentry <feedback>: Autofix iterates on these changes@sentry stop iterating: Autofix stops iterating on this runThis PR was automatically generated by Sentry. You can adjust this setting at any time.
Note
Low Risk
Narrow change to error handling in optional log enrichment; rollback on failure is the intended fix for poisoned sessions.
Overview
Fixes worker tasks failing with
PendingRollbackErrorwhen optional log-context DB lookups error out but the shared SQLAlchemy session is still used for real work.In
populate_from_sqlalchemy, failed queries (e.g. dropped connections) now trigger a guardeddbsession.rollback()so the thread-scoped session is usable again beforerun_impl. Repository/owner lookups also skip tuple unpacking whenquery.first()returnsNone, avoiding extraTypeErrors during enrichment.Reviewed by Cursor Bugbot for commit ae7d646. Bugbot is set up for automated code reviews on this repo. Configure here.