Unable to specify a default value for a generic parameter

Simplified Example:

from typing import TypeVar

_T = TypeVar('_T')
def foo(a: _T = 42) -> _T:  # E: Incompatible types in assignment (expression has type "int", variable has type "_T")
    return a

Real World example is something closer to:

_T = TypeVar('_T')
def noop_parser(x: str) -> str:
    return x

def foo(value: str, parser: Callable[[str], _T] = noop_parser) -> _T:
    return parser(value)

I think the problem here is that in general, e.g. if there are other parameters also using _T in their type, the default value won't work. Since this is the only parameter, you could say that this is overly restrictive, and if there are no parameters, the return type should just be determined by that default value (in your simplified example, you'd want it to return int). But I'm not so keen on doing that, since it breaks as soon as another generic parameter is used.

To make this work, you can use @overload, e.g.

@overload
def foo() -> int: ...
@overload
def foo(a: _T) -> _T: ...
def foo(a = 42):  # Implementation, assuming this isn't a stub
    return a

That's what we do in a few places in typeshed too (without the implementation though), e.g. check out Mapping.get() in typing.pyi.

I'm running in to a similar issue when trying to iron out a bug in the configparser stubs. Reducing RawConfigParser to the relevant parts gives:

_section = Mapping[str, str]

class RawConfigParser:
    def __init__(self, dict_type: Mapping[str, str] = ...) -> None: ...

    def defaults(self) -> _section: ...

This is incorrect at the moment, because dict_type should (as the name indicates) be a type. Furthermore, defaults() returns an instance of dict_type, so this seems like a good application of generics. The following works as expected when a dict_type is passed (i.e. reveal_type(RawConfigParser(dict_type=dict).default()) gives builtins.dict*[Any, Any]):

_T1 = TypeVar('_T1', bound=Mapping)

class RawConfigParser(Generic[_T1]):
    def __init__(self, dict_type: Type[_T1] = ...) -> None: ...

    def defaults(self) -> _T1: ...

but when you don't pass a dict_type, mypy can't infer what type it should be using and asks for a type annotation where it is being used. This is less than ideal for the default instantiation of the class (which will be what the overwhelming majority of people are using). So I tried adding the appropriate default to the definition:

_T1 = TypeVar('_T1', bound=Mapping)

class RawConfigParser(_parser, Generic[_T1]):
    def __init__(self, dict_type: Type[_T1] = OrderedDict) -> None: ...

    def defaults(self) -> _T1: ...

but then I hit the error that OP was seeing:

stdlib/3/configparser.pyi:51: error: Incompatible default for argument "dict_type" (default has type Type[OrderedDict[Any, Any]], argument has type Type[_T1])

This feels like something that isn't currently supported, but I'm not sure if I'm missing a way that I could do this...

@OddBloke

@OddBloke
This default is, strictly speaking, not safe. This would require a lower bound for _T1 in order to be safe.

@ilevkivskyi

@ilevkivskyi I don't follow, I'm afraid; could you expand on that a little, please?

Consider this code:

class UserMapping(Mapping):
    ...
RawConfigParser[UserMapping]()

In this case OrderedDict will be "assigned" to dict_type, which will be Type[UserMapping], which is not a supertype of Type[OrderedDict]. In order for the default value to be safe it is necessary to have some guarantee that _T1 will be always a supertype of OrderedDict (this is called a lower bound), but this is not supported, only upper bounds are allowed for type variables.

@ilevkivskyi

@ilevkivskyi Thanks, that makes sense. Do you think what I'm trying to do is unrepresentable as things stand?

I didn't think enough about this, but my general rule is not to be "obsessed" with precise types, sometimes Any is OK. If a certain problem appears repeatedly, then maybe we need to improve something (also now we have plugin system). In this particular case my naive guess would be to play with overloads, as Guido suggested (note that __init__ can also be overloaded).

Allowing default values for generic parameters might also enable using type constraints to ensure that generics are only used in specified ways. This is a bit esoteric, but here is an example of how you might use the same implementation for a dict and a set while requiring that consumers of the dict type always provide a value to insert() while consumers of the set type never do.

class Never(Enum):
    """An uninhabited type"""

class Only(Enum):
    """A type with one member (kind of like NoneType)"""
    ONE = 1

NOT_PASSED = Only(1)

class MyGenericMap(Generic[K, V]):

    def insert(key, value=cast(Never, NOT_PASSED)):
        # type: (K, V) -> None
        ...
       if value is NOT_PASSED:
           ....

class MySet(MyGenericMap[K, Never]):
    pass

class MyDict(MyGenericMap[K, V]):
    pass

my_set = MySet[str]()
my_set.insert('hi')           # this is fine, default value matches concrete type
my_set.insert('hi', 'hello')  # type error, because 'hello' is not type never

my_dict = MyDict[str, int]()
my_dict.insert('hi')          # type error because value is type str, not Never
my_dict.insert('hi', 22)      # this is fine, because the default value is not used

Huh, this just came up in our code.

This request already appeared five times, so I am raising priority to high.

I think my preferred solution for this would be to support lower bounds for type variables. It is not too hard to implement (but still a large addition), and at the same time it may provide more expressiveness in other situations.

It seems like there are some workarounds available for class methods and normal function but does someone have a workaround for class attributes? E.g.

config_parse_string: ConfigParseCallback[str]

class ConfigSetting(Generic[T]):
    parse: ConfigParseCallback[T] = config_parse_string

I'm looking at this and it seems like one potential solution would be updating TypeChecker.check_default_args to replace the types of generic arguments with the types of their initializers - i.e def f[T](var: T = "asdf") -> T: ... is actually checked as def f(var: str = "asdf") -> str, but I'm not sure how to infer str and not Literal["asdf"] from the context of a type checker.

@mcmanustfj

@mcmanustfj That is not a solution because we do not want the default’s type to constrain call sites where the default is not used. In your example, we need to accept f(42) returning int. See my comment above for details of what should be accepted and rejected.

@andersk

@andersk My thought was to only change the function type checking and not the invocation type checking, but I could definitely be misunderstanding how check_default_args works.

My test patch has the following change:

    def check_default_args(self, item: FuncItem, body_is_trivial: bool) -> None:
        if any(
            arg.initializer is not None and type(arg.type_annotation) is UnboundType
            for arg in item.arguments
        ):
            args_without_generic_defaults = [
                Argument(
                    variable=arg.variable,
                    type_annotation=arg.type_annotation,
                    initializer=None
                    if type(arg.type_annotation) is UnboundType
                    else arg.initializer,
                    kind=arg.kind,
                    pos_only=arg.pos_only,
                )
                for arg in item.arguments
            ]

            item_without_generic_defaults: FuncItem = FuncItem(
                arguments=args_without_generic_defaults,
                body=item.body,
                typ=item.type,
                type_args=item.type_args,
            )

            self.check_default_args(item_without_generic_defaults, body_is_trivial)
            return
    ... # rest of check_default_args

Right now,

def test_func(var: T = "asdf") -> T:
    return var

if __name__ == "__main__":
    x = test_func()
    reveal_type(x)
    reveal_type(test_func())
    reveal_type(test_func("asdf"))
    reveal_type(test_func(True))
    reveal_type(test_func(1))

has the opposite problem and doesn't give the default invocation a type at all,

example.py:31: note: Revealed type is "Any"
example.py:32: note: Revealed type is "Never"
example.py:33: note: Revealed type is "builtins.str"
example.py:34: note: Revealed type is "builtins.bool"
example.py:35: note: Revealed type is "builtins.int"

Not sure where to go from here, unfortunately.

@mcmanustfj

@mcmanustfj Just conceptually, changing only the function type checking as you describe is still not a solution, because it would fail to reject this wrong definition:

def f[T](var: T = "asdf") -> T:
    # we should not be able to assume var: str here
    return var.lower()

f(42)

and fail to reject this wrong invocation:

def f[T](var: T = "asdf") -> T:
    return var

# when the default is used, we should not let T be anything but str
ret: int = f()

The correct solution needs to involve type-checking the default value at each call site if the default is used there. That is, we need to treat

def f[T](var: T = "asdf") -> T:
    return var

f()
f(arg)

as the syntactic sugar it is for something like

f_var_default = "asdf"
def f[T](var: T) -> T:
    return var

f(f_var_default)
f(arg)

(with perhaps an additional definition-site check that there exists some T that unifies the types of f_var_default and var, but that’s not required for soundness, as the consequence if there doesn’t would be that a call relying on the default is safely prohibited at the call site).

Add additional try based on Guido's 2017 suggestion

Hi there! mypy shows a type error in my code which Google's AI vaguely said is a mypy limitation (without citing any source), digging further I've found this github issue which I think is the right one. So, 3 questions:

1 - Is my error a manifestation of the issue discussed here? (i.e, am I posting in the right place?)

# test.py

def csv2list[T](text:str, sep:str=",", itemtype:type[T]=str) ->  list[T]:
    return [itemtype(_.strip()) for _ in (text.split(sep) if text else [])]

print(csv2list("a, b, c"))
print(csv2list("1, 2, 3", itemtype=int))

$ python3.12 test.py
['a', 'b', 'c']
[1, 2, 3]
$ mypy test.py
test.py:3: error: Incompatible default for argument "itemtype" (default has type "type[str]", argument has type "type[T]")  assignment]
test.py:4: error: Too many arguments for "object"  [call-arg]
Found 2 errors in 1 file (checked 1 source file)
$ mypy --version
mypy 1.17.1 (compiled: yes)

2 - Is this the correct way of annotating my function, and the error is a known bug/limitation in mypy?

3 - Are there any workarounds besides @overload? Would using pre-3.12 syntax of T = typing.TypeVar("T") help?

Re 1, 2: Yes. You can tell it's a mypy limitation because it works in pyright, which you can try out here: https://pyright-play.net/

Though you should annotate itemtype as Callable[[str], T] instead of type[T] because you don't just require a type -- you require a constructor that takes a str as an argument. (This doesn't help you with the mypy limitation though.)

So, I would write it like this.

3: Pre-3.12 syntax will not help. I think what I would do with your code is disentangling the two functionalities of the function: have one function that just does the splitting and stripping; and have another that does conversion if needed.

@tmke8

@tmke8 , thank you so much! I actually began with Callable, then trimmed down to type[] trying to please mypy.

Regarding your suggestion of splitting the function, it would simply push the bug down to the "conversion" function, no? csv2list() would get rid of the problematic itemtype and return list[str], becoming a non-generic function. But if I understood the bug correctly, the new function would have the same signature regarding itemtype, triggering the (buggy) behavior.

I would probably write it like this:

def csv2list(text:str, sep:str=",") -> list[str]:
    return [_.strip() for _ in (text.split(sep) if text else [])]

print(csv2list("a, b, c"))
print(map(int, csv2list("1, 2, 3")))

cache: Fix type: ignore issues.

bumped into the same issue with following snippet:

def func[T](callback: Callable[[], T] = lambda: None) -> T:
    return callback()

mypy raises two errors:

error: Incompatible default for parameter "callback" (default has type "Callable[[], None]", parameter has type "Callable[[], T]")  [assignment]
error: Incompatible return value type (got "None", expected "T")  [return-value]

not sure though if second error (return-value) is a consequence of failed binding of default value and generic parameter.

Navigation Menu

Repository navigation

Unable to specify a default value for a generic parameter #3737

39 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Unable to specify a default value for a generic parameter #3737

Description

Activity

gvanrossum commented on Jul 19, 2017

OddBloke commented on Aug 16, 2017

ilevkivskyi commented on Aug 16, 2017

OddBloke commented on Aug 16, 2017

ilevkivskyi commented on Aug 16, 2017

OddBloke commented on Aug 16, 2017

ilevkivskyi commented on Aug 16, 2017

nematsakis commented on Jan 21, 2018

gvanrossum commented on Jun 25, 2018

ilevkivskyi commented on Oct 19, 2018

39 remaining items

septatrix commented on Nov 20, 2024

mcmanustfj commented on Dec 5, 2024

andersk commented on Dec 5, 2024

mcmanustfj commented on Dec 10, 2024

andersk commented on Dec 10, 2024

MestreLion commented on Aug 13, 2025

tmke8 commented on Aug 13, 2025

MestreLion commented on Aug 13, 2025

tmke8 commented on Aug 13, 2025

RomanValov commented on Sep 11, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions