Repository navigation
fix(worker): rollback session before marking commit error in process_upload - #2560
sentry[bot] wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2560 +/- ##
==========================================
- Coverage 91.65% 91.64% -0.01%
==========================================
Files 1341 1341
Lines 53342 53348 +6
Branches 1649 1649
==========================================
+ Hits 48888 48891 +3
- Misses 4133 4136 +3
Partials 321 321
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
|
The Sentry GitHub App installation for this repository is missing permissions it needs to keep iterating on this pull request to get CI passing. Review and accept the updated permissions to let Seer continue: https://github.com/organizations/codecov/settings/installations/86101127/permissions/update |
Problem
When an error occurs during upload processing that invalidates the database connection (for example, a PostgreSQL
OperationalError: server closed the connection unexpectedly), theexcept Exception:block inprocess_uploadimmediately attempted to setcommit.state = "error"and calldb_session.commit(). Because the invalidated transaction was not rolled back first, SQLAlchemy raised aPendingRollbackError: Can't reconnect until invalid transaction is rolled back., masking the root cause and preventing the commit state update from being recorded.Solution
In
apps/worker/services/processing/processing.py:log.exception("Could not properly process commit")so the primary error context is never obscured.db_session.rollback()prior to any subsequent database operations to clear invalid transaction state and allow reconnecting.commit.state = "error"assignment anddb_session.commit()in a guardedtry/except SQLAlchemyError:block so secondary persistence failures are logged and rolled back without displacing the original exception.Legal Boilerplate
Look, I get it. The entity doing business as "Codecov" is owned by Harness, Inc. In 2026 Harness acquired Codecov and as a result Harness is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Harness can use, modify, copy, and redistribute my contributions, under Harness's choice of terms.
Fixes WORKER-ZAP
@sentry <feedback>: Autofix iterates on these changes@sentry stop iterating: Autofix stops iterating on this runThis PR was automatically generated by Sentry. You can adjust this setting at any time.
Note
Medium Risk
Changes error-path DB session handling in upload processing; behavior on success is unchanged, but failed uploads may now reliably record
errorstate when the connection recovers after rollback.Overview
Hardens upload processing failure handling so a dropped or invalidated DB connection no longer hides the real error or blocks marking the commit as failed.
On any non-Celery exception in
process_upload, the worker now logs the original failure first,rollback()s the session before retrying DB work, then attemptscommit.state = "error"inside aSQLAlchemyErrorguard that logs and rolls back if persistence fails. The original exception is still re-raised for retries and monitoring.Reviewed by Cursor Bugbot for commit 5825f59. Bugbot is set up for automated code reviews on this repo. Configure here.