Repository navigation
Introspection capabilities are incomplete for C objects #32
Description
For example, we have hacks that give us signatures of callables in some cases, but we cannot introspect signatures of vectorcall/fastcall functions. We should have a formal introspection protocol to reliably obtain the signature of any callable.
Similarly, other introspection hacks in the inspect module should be formalised and implemented reliably as part of the C api.
Activity
vstinner commentedon Jun 8, 2023
Can Argument Clinic be seen as a practical solution for the specific case of function signatures? It generates a
__text_signature__attribute which is parsed byinspect.signature(). Right now, Argument Clinic is not usable outside CPython. It's sad since it makes the usage of fastcall / vectorcall simple.davidhewitt commentedon Jul 30, 2023
PyO3 has internal functionality in its Rust "procedural macros" which works very similarly to argument clinic, we generate a
__text_signature__too.I would be very keen to have some support for annotations for "builtin" functions and methods defined via the C-API, I think the inability for PyO3 to add annotations (without us implementing custom callables, which we're not keen to do yet) is one of the biggest technical limitations in the C API which affects PyO3 users. The workaround is for them to maintain separate
.pyitypedef files, but that's never satisfying to tell them 😄davidhewitt commentedon Jul 30, 2023
(Making
__text_signature__grammar permit type annotations might go a long way towards lifting this limitation without needing to do more complex changes at the C-API level.)iritkatriel commentedon Jul 30, 2023
I would be interested in solving this issue properly by making all callables reliably introspectible. I'm not sure though whether we should do it by extending
__text_signature__or by making all callables expose a c type equivalent to the inspect modules's Signature class. Maybe through__signature__.encukou commentedon Aug 1, 2023
__signature__should be aninspect.Signatureobject, meaning we'd need to importinspecton attribute access, which doesn't seem desirable.AFAIK, that's exactly why we have
__text_signature__: C exposes a string, and creating the expensive user-friendlySignatureobjects is left to Python code.iritkatriel commentedon Aug 1, 2023
I was thinking of making Signature a builtin (so it can be used without the inspect module). As long as we stick with the
__text__signature__arrangement, introspection is not available to C code.erlend-aasland commentedon Aug 1, 2023
That may be a good idea. I guess we'd have to make other parts of the
inspectmodule builtins as well (for exampleinspect.Parameter).hauntsaninja commentedon Aug 1, 2023
Note that most Python type checkers are static, so they're not going to import the code and access attributes to determine function signatures. This means you will still need
pyifiles (although better tooling might mean it becomes easier and easier to generate these).Also note there are several kinds of signatures common in C callables that
inspect.Signaturecan't really represent (such as things you would type with@overload), more discussion in https://discuss.python.org/t/signatures-a-call-to-action/23580davidhewitt commentedon Aug 2, 2023
Yes, I think in PyO3's case the ideal outcome is that at build time the pyi can be generated and shipped in the wheel.
skirpichev commentedon Feb 27, 2024
On some attribute access. Is that much worse than the current solution with parsing the
__text_signature__in theinspect.signature()? BTW, importing pure-python modules now used for integer arithmetic.This approach mixes support for optional typing hints and more basic support for introspection signatures in extension modules. Generally, there are no type annotations in the stdlib. Thus, generated here (by AC?) stub files will be useless for type checkers...