Repository navigation
[v3-3-test] Fix db migrate failure to Airflow 2.9.2 under PyMySQL driver (#70235) - #72613
Merged
Merged
Conversation
…ver (#70235) * Fix db migrate failure to Airflow 2.9.2 under PyMySQL driver Upgrading through revision 686269002441 raised pymysql.err.ProgrammingError because the migration used MySQL's PREPARE/EXECUTE/DEALLOCATE PREPARE dynamic SQL to conditionally drop unique constraints, which PyMySQL's driver does not support (see PyMySQL/PyMySQL#202). Replaced with a plain lookup of existing constraints from information_schema.TABLE_CONSTRAINTS followed by conditional ALTER TABLE ... DROP INDEX statements, which works with any MySQL driver. Verified against a real MySQL/PyMySQL backend: downgrading to bff083ad727d and re-running db migrate now completes without error. closes: #43690 * Add newsfragment for MySQL PyMySQL migration fix * Fix offline migration for 2.9.2 ORM-consistency revision The PyMySQL online-mode fix in this migration queries information_schema.TABLE_CONSTRAINTS via conn.execute() to decide which legacy unique constraints to drop. In Alembic's offline (--sql) mode there is no live connection, so conn.execute() returns None and iterating over it raises TypeError, breaking the CI "Test offline migration" step. Guard the query-then-conditionally-drop logic behind context.is_offline_mode(), falling back to the original dynamic prepare/execute/deallocate SQL text for the offline branch, since that text is only ever emitted as script output there rather than executed directly. * Address review feedback on MySQL PyMySQL migration fix Bind the table name as a query parameter in the information_schema lookup instead of interpolating it into the SQL string, since it's a value comparison rather than an identifier — matches Airflow's usual parameter-binding convention even though every call site currently passes a hardcoded literal. Also correct the docstring: PyMySQL can run a bare PREPARE statement fine, what it actually can't do is send more than one statement per cursor.execute() call, because it never enables CLIENT.MULTI_STATEMENTS. * Fixed the PR comments (cherry picked from commit 08e428c) Co-authored-by: Aaryan Mahajan <aaryanmhjn@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrading through revision 686269002441 raised
pymysql.err.ProgrammingError because the migration used MySQL's
PREPARE/EXECUTE/DEALLOCATE PREPARE dynamic SQL to conditionally drop
unique constraints, which PyMySQL's driver does not support
(see PyMySQL/PyMySQL#202). Replaced with a
plain lookup of existing constraints from
information_schema.TABLE_CONSTRAINTS followed by conditional
ALTER TABLE ... DROP INDEX statements, which works with any MySQL
driver.
Verified against a real MySQL/PyMySQL backend: downgrading to
bff083ad727d and re-running db migrate now completes without error.
closes: #43690
Add newsfragment for MySQL PyMySQL migration fix
Fix offline migration for 2.9.2 ORM-consistency revision
The PyMySQL online-mode fix in this migration queries
information_schema.TABLE_CONSTRAINTS via conn.execute() to decide which
legacy unique constraints to drop. In Alembic's offline (--sql) mode
there is no live connection, so conn.execute() returns None and
iterating over it raises TypeError, breaking the CI "Test offline
migration" step. Guard the query-then-conditionally-drop logic behind
context.is_offline_mode(), falling back to the original dynamic
prepare/execute/deallocate SQL text for the offline branch, since that
text is only ever emitted as script output there rather than executed
directly.
Bind the table name as a query parameter in the information_schema
lookup instead of interpolating it into the SQL string, since it's a
value comparison rather than an identifier — matches Airflow's usual
parameter-binding convention even though every call site currently
passes a hardcoded literal. Also correct the docstring: PyMySQL can run
a bare PREPARE statement fine, what it actually can't do is send more
than one statement per cursor.execute() call, because it never enables
CLIENT.MULTI_STATEMENTS.
(cherry picked from commit 08e428c)
Co-authored-by: Aaryan Mahajan aaryanmhjn@gmail.com