Repository navigation
Fix dict literal inference against generic dict hints - #5453
Open
Divyansh151005 wants to merge 1 commit into
Open
Divyansh151005 wants to merge 1 commit into
Divyansh151005 wants to merge 1 commit into
Conversation
Two related bugs in hint decomposition for dict displays and comprehensions: - A `dict[K, V]` union member whose key and value are both type variables decomposed to `(None, None)`, which `infer_with_decomposed_hint` treats as "not a dict", so the member was skipped and inference fell back to an erroring concrete member (meta#4913). `decompose_dict` now only rejects hints that can't be dicts, or uninformative ones without placeholder vars. - In the snapshot branch of `infer_with_decomposed_hint`, decomposition unified the hint's quantified vars with fresh unwrap vars, discarding their bounds and restrictions, so a bounded TypeVar could be solved outside its bound without an error and overloads didn't fall through (meta#4937). Decomposition is now isolated with a snapshot/restore, as `decompose_hint` already does. The isolation also fixes the known over-narrowing in `test_any_bounded_typevar_empty_list_repeated_append`.
Contributor
|
This pull request has been imported. If you are a Meta employee, you can view this in D128354790. (Because this pull request was imported automatically, there will not be any future comments.) |
|
Diff from mypy_primer, showing the effect of this PR on open source code: ============================================================
SUMMARY
============================================================
Total: +49 new errors, -45 fixed errors
By preset: +1/-1 (default), +7/-26 (strict)
Projects with changes (31):
pip: +1 -0
tornado: +1 -0
apprise: +2 -0
cki-lib: +1 -1
openlibrary: +3 -0
rotki: +0 -2
schema_salad: +8 -1
jax: +0 -4
scikit-learn: +12 -4
spark: +0 -1
scikit-build-core: +1 -1
freqtrade: +0 -1
colour: +2 -2
schemathesis: +0 -1
streamlit: +0 -6
vision: +0 -2
zulip: +0 -1
ibis: +1 -0
zope.interface: +1 -0
static-frame: +0 -1
pandas: +0 -5
dd-trace-py: +0 -1
core: +6 -1
poetry: +1 -0
spack: +2 -0
speedrun.com_global_scoreboard_webapp: +1 -3
xarray: +2 -3
scrapy: +0 -2
rich: +1 -0
cvxpy: +1 -0
django-modern-rest: +2 -2
============================================================
FULL DIFF DETAILS
------------------------------------------------------------
pip (https://github.com/pypa/pip)
+ ERROR src/pip/_vendor/rich/pretty.py:973:12-1005:6: The type of `data` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
tornado (https://github.com/tornadoweb/tornado)
+ ERROR tornado/test/web_test.py:4058:13-4061:14: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
apprise (https://github.com/caronc/apprise)
+ ERROR apprise/plugins/aprs.py:711:17-717:18: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR apprise/utils/logic.py:108:17-45: The type of `logic` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
cki-lib (https://gitlab.com/cki-project/cki-lib)
- ERROR cki_lib/metrics/server.py:23:12-24: Returned type `Iterator[Any] | Iterator[Unknown]` is not assignable to declared return type `None` [bad-return]
+ ERROR cki_lib/metrics/server.py:23:12-24: Returned type `Iterator[Unknown]` is not assignable to declared return type `None` [bad-return]
openlibrary (https://github.com/internetarchive/openlibrary)
+ ERROR openlibrary/plugins/upstream/utils.py:742:26-745:10: The type of `language_names` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR openlibrary/solr/updater/work.py:591:25-596:10: The type of `work_cover_id` is unknown; it is inferred as an implicit `Any` [unknown-variable-type]
+ ERROR openlibrary/solr/updater/work.py:592:13-595:14: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
rotki (https://github.com/rotki/rotki)
- ERROR rotkehlchen/tests/api/test_user_authenticate.py:211:11-13: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR rotkehlchen/tests/api/test_user_authenticate.py:214:37-40: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
schema_salad (https://github.com/common-workflow-language/schema_salad)
+ ERROR src/schema_salad/dlang_codegen.py:148:55-57: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
+ ERROR src/schema_salad/dlang_codegen.py:150:16-25: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR src/schema_salad/dlang_codegen.py:150:16-25: Implicit conversion of `str | Any` to `bool` is not allowed [implicit-bool]
+ ERROR src/schema_salad/dlang_codegen.py:150:16-25: Implicit conversion of `str | Unknown` to `bool` is not allowed [implicit-bool]
+ ERROR src/schema_salad/dlang_codegen.py:151:21-32: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR src/schema_salad/dlang_codegen.py:151:21-32: The type of `lines` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR src/schema_salad/dlang_codegen.py:153:17-101: The type of `lines` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR src/schema_salad/dlang_codegen.py:153:23-30: The type of attribute `replace` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR src/schema_salad/dlang_codegen.py:153:53-60: The type of attribute `replace` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
jax (https://github.com/google/jax)
- ERROR jax/_src/array.py:446:12-14: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR jax/_src/lax/lax.py:4205:12-14: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR jax/_src/lax/lax.py:5320:12-14: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR jax/_src/pallas/mosaic_gpu/primitives.py:1236:9-11: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
scikit-learn (https://github.com/scikit-learn/scikit-learn)
+ ERROR sklearn/compose/_column_transformer.py:451:32-70: The type of `transformers` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR sklearn/compose/_column_transformer.py:1174:32-1177:18: The type of `transformers` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR sklearn/compose/_column_transformer.py:1180:32-1182:18: The type of `transformers` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR sklearn/compose/_column_transformer.py:1225:28-86: The type of `transformers` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR sklearn/tests/test_common.py:267:26-273:2: The type of `column_name_estimators` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR sklearn/tests/test_common.py:268:5-272:6: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR sklearn/tests/test_common.py:358:25-367:2: The type of `SET_OUTPUT_ESTIMATORS` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR sklearn/tests/test_common.py:359:5-366:6: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR sklearn/tests/test_discriminant_analysis.py:384:32-54: The type of this argument is an explicit `Any` [explicit-any-use]
+ ERROR sklearn/tests/test_discriminant_analysis.py:384:32-54: The type of this argument is unknown [unknown-argument-type]
+ ERROR sklearn/tests/test_discriminant_analysis.py:384:47-53: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR sklearn/tests/test_discriminant_analysis.py:387:32-54: The type of this argument is an explicit `Any` [explicit-any-use]
+ ERROR sklearn/tests/test_discriminant_analysis.py:387:32-54: The type of this argument is unknown [unknown-argument-type]
+ ERROR sklearn/tests/test_discriminant_analysis.py:387:47-53: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR sklearn/utils/fixes.py:74:13-15: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR sklearn/utils/fixes.py:250:12-14: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
spark (https://github.com/apache/spark)
- ERROR python/pyspark/eval_handlers/tests/test_udtf_eval_type_handlers.py:274:16-18: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
scikit-build-core (https://github.com/scikit-build/scikit-build-core)
- ERROR tests/test_dynamic_metadata_unit.py:146:9-74: Result of call expression is of type `list[dict[str, str]] | list[str]` and is not used; assign to `_` if this is intentional [unused-call-result]
+ ERROR tests/test_dynamic_metadata_unit.py:146:9-74: Result of call expression is of type `list[str]` and is not used; assign to `_` if this is intentional [unused-call-result]
freqtrade (https://github.com/freqtrade/freqtrade)
- ERROR freqtrade/rpc/api_server/api_backtest.py:237:76-78: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
colour (https://github.com/colour-science/colour)
- ERROR colour/io/ctl.py:441:8-15: Implicit conversion of `Sequence[str] | list[str]` to `bool` is not allowed [implicit-bool]
+ ERROR colour/io/ctl.py:441:8-15: Implicit conversion of `Sequence[str]` to `bool` is not allowed [implicit-bool]
- ERROR colour/io/ctl.py:566:8-15: Implicit conversion of `Sequence[str] | list[str]` to `bool` is not allowed [implicit-bool]
+ ERROR colour/io/ctl.py:566:8-15: Implicit conversion of `Sequence[str]` to `bool` is not allowed [implicit-bool]
schemathesis (https://github.com/schemathesis/schemathesis)
- ERROR src/schemathesis/specs/openapi/coverage/_schema.py:1560:20-22: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
streamlit (https://github.com/streamlit/streamlit)
- ERROR lib/streamlit/elements/lib/utils.py:300:25-27: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR lib/streamlit/elements/lib/utils.py:306:9-44: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR lib/streamlit/elements/lib/utils.py:309:9-33: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR lib/streamlit/elements/lib/utils.py:313:9-50: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR lib/streamlit/elements/lib/utils.py:315:62-77: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR lib/tests/streamlit/dataframe_util_test.py:1237:14-16: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
vision (https://github.com/pytorch/vision)
- ERROR torchvision/ops/misc.py:114:22-24: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR torchvision/ops/misc.py:290:18-20: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
zulip (https://github.com/zulip/zulip)
- ERROR zerver/lib/email_notifications.py:104:22-24: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
ibis (https://github.com/ibis-project/ibis)
+ ERROR ibis/backends/clickhouse/tests/test_operators.py:259:28-30: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
zope.interface (https://github.com/zopefoundation/zope.interface)
+ ERROR src/zope/interface/common/tests/test_collections.py:122:23-25: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
static-frame (https://github.com/static-frame/static-frame)
- ERROR static_frame/test/unit/test_store.py:266:50-64: Unused `# type: ignore` comment [unused-type-ignore]
pandas (https://github.com/pandas-dev/pandas)
- ERROR pandas/tests/groupby/test_numeric_only.py:251:14-16: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR pandas/tests/resample/test_resample_api.py:1092:14-16: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR pandas/tests/reshape/concat/test_datetimes.py:142:18-20: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR pandas/tests/reshape/concat/test_datetimes.py:164:18-20: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR pandas/tests/tools/test_to_timedelta.py:711:14-16: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
dd-trace-py (https://github.com/DataDog/dd-trace-py)
- ERROR ddtrace/llmobs/_llmobs.py:3335:25-27: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
core (https://github.com/home-assistant/core)
+ ERROR homeassistant/components/pilight/__init__.py:132:16-134:10: The type of `data` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR homeassistant/components/pilight/__init__.py:137:37-46: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR homeassistant/components/pilight/__init__.py:137:37-46: The type of this argument is an explicit `Any` [explicit-any-use]
+ ERROR homeassistant/components/pilight/__init__.py:137:37-46: The type of this argument is unknown [unknown-argument-type]
+ ERROR homeassistant/components/pilight/__init__.py:138:34-38: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR homeassistant/components/uk_transport/sensor.py:139:26-141:10: The type of `request_params` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR homeassistant/components/uk_transport/sensor.py:143:44-65: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
poetry (https://github.com/python-poetry/poetry)
+ ERROR tests/repositories/link_sources/test_base.py:47:13-60:14: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
spack (https://github.com/spack/spack)
+ ERROR lib/spack/spack/modules/common.py:643:28-69: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR lib/spack/spack/package_base.py:1275:40-42: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
speedrun.com_global_scoreboard_webapp (https://github.com/Avasam/speedrun.com_global_scoreboard_webapp)
- ERROR backend/services/utils.py:186:27-29: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR backend/services/utils.py:189:12-48: Implicit conversion of `Literal[0] | dict[str, int | str] | dict[Unknown, Unknown] | None` to `bool` is not allowed [implicit-bool]
+ ERROR backend/services/utils.py:189:12-48: Implicit conversion of `Literal[0] | dict[str, int | str] | None` to `bool` is not allowed [implicit-bool]
- ERROR backend/services/utils.py:191:13-31: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
xarray (https://github.com/pydata/xarray)
+ ERROR xarray/core/datatree.py:1412:32-1415:10: The type of `flat_data_and_coords` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
+ ERROR xarray/tests/backends/test_file_manager.py:382:77-79: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR xarray/tests/test_coding_times.py:2233:33-35: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR xarray/tests/test_coding_times.py:2234:35-46: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
- ERROR xarray/tests/test_groupby.py:403:14-16: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
scrapy (https://github.com/scrapy/scrapy)
- ERROR tests/test_downloaderslotssettings.py:190:16-18: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
- ERROR tests/test_downloaderslotssettings.py:191:28-37: The type of this argument is partially unknown; it contains an implicit `Any` [partially-unknown-type]
rich (https://github.com/Textualize/rich)
+ ERROR rich/pretty.py:973:12-1005:6: The type of `data` is partially unknown; it contains an implicit `Any` [partially-unknown-type]
cvxpy (https://github.com/cvxpy/cvxpy)
+ ERROR cvxpy/transforms/indicator.py:46:53-55: Cannot infer type of empty container; it will be treated as containing `Any` [implicit-any-empty-container]
django-modern-rest (https://github.com/wemake-services/django-modern-rest)
- ERROR dmr/openapi/generators/parameter.py:183:20-54: Implicit conversion of `bool | Any | None` to `bool` is not allowed [implicit-bool]
+ ERROR dmr/openapi/generators/parameter.py:183:20-54: Implicit conversion of `Any | None` to `bool` is not allowed [implicit-bool]
- ERROR dmr/openapi/generators/parameter.py:188:20-53: Implicit conversion of `bool | Any | None` to `bool` is not allowed [implicit-bool]
+ ERROR dmr/openapi/generators/parameter.py:188:20-53: Implicit conversion of `Any | None` to `bool` is not allowed [implicit-bool] |
Contributor
Author
|
Triage of the mypy_primer diff:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Dict literals checked against a generic
dict[...]hint had two related bugs. Both live in hint decomposition for dict displays and comprehensions.#4913: a
dict[K, V]union member is skipped. When both the key and the value of a hint are type variables,decompose_dictresolves neither, and the decompose closures inexpr.rsturned(None, None)intoNone.infer_with_decomposed_hintreadsNoneas "this member is not a dict", sodict[K, S1]indict[str, list[int]] | dict[K, S1]was never tried. Inference fell back todict[str, list[int]], which matched with errors, so the call reported bogusbad-assignmenterrors and leftKandS1unsolved. The bound onKis incidental: anydict[A, B]member with two type variables hits this.decompose_dictnow returnsNoneonly when the hint can't be a dict, or when it is a dict that contributes no key hint, no value hint, and no placeholder vars to pin. Uninformative var-free members such asobjectare therefore still skipped in favor of informative siblings, and the two duplicated closures collapse into the helper.#4937: a bounded TypeVar is solved outside its bound. In the snapshot branch of
infer_with_decomposed_hint, decomposition runsdict[@k, @v] <: dict[@K, @V]. That unifies the hint's quantified vars with freshUnwrapvars, and unification keeps the non-quantified side's state.Klost its bound or restriction, and inference then pinned it to e.g.str(orint | Nonein the issue) without abad-specializationerror. The speculative check also passed, so overloads didn't fall through.decompose_hintalready isolates decomposition with a snapshot and restore ("Decomposing a hint should not have any side effects"). The snapshot branch now does the same before inferring, so pinning the vars goes through the normal bound and restriction checks. Pre-typed arguments and list literals were already checked correctly. Only dict displays and comprehensions reached this path with a merged var.The two fixes ship together because fixing #4913 alone makes more dict hints reach the #4937 path. For example,
dict[str, list[int]] | dict[KI, S1]withKI: intwould go from a (correct) error to silently solvingKI = str.Fixes #4913
Fixes #4937
Test Plan
New tests in
pyrefly/lib/test/dict.rs:test_dict_literal_union_hint_with_generic_dict_membercovers the Dict-literal arguments don't try all members of a union ofdict[...]types #4913 example, both union orders, and the dict-comprehension path.test_dict_literal_overload_falls_through_on_typevar_boundcovers the Bound TypeVar solved to a type outside its bound #4937 example.b.map({1: 10, 0: 20, None: None})is nowBox[Any], matching mypy, pyright and ty. The bound-satisfying call is stillBox[int].test_dict_literal_generic_hint_checks_typevar_boundchecks that bounded vars in key position, value position, and inside a union are validated.test_dict_literal_union_hint_generic_member_bound_violationchecks that a union whose generic member violates its bound still falls back to the concrete member's error.On
main(source changes stashed, new tests kept):With this change:
Full library suite (
cargo test -p pyrefly --lib -- --skip commands::coverage --skip lsp_interaction): 9173 passed, 1 failed. The one failure wastest_any_bounded_typevar_empty_list_repeated_append, abug =test that encoded the over-narrowing of an empty list passed tokeep[T: list[Any]](xs.append(2)rejected,xstypedlist[Literal[1]] | list[Any]). The isolation fix resolves it in the way its comment describes ("Both appends should succeed, and xs should retain type list[Any]"), so I dropped thebugmarker and the two error expectations. Rerunning thecontextual,dict,generic*andoverloadmodules gives 483 passed, 0 failed.cargo clippy -p pyrefly --libis clean.