Repository navigation
Translate PostgreSQL static inline function bodies into Rust - #2392
Draft
eeeebbbbrrrr wants to merge 39 commits into
Draft
eeeebbbbrrrr wants to merge 39 commits into
eeeebbbbrrrr wants to merge 39 commits into
Conversation
Contributor
Author
|
Inline functions that remain native calls, and why. These are possible follow-ups, not part of this PR.
Verification beyond the fixture tests: a throwaway test, not part of this PR, invoked each of the 1,664 translated macros across PG15 to PG19 once with arguments of its binding's types, and all of them compiled. That check found the enum-result and record-result failures fixed in the second commit. |
Contributor
Author
|
Measured cost of the guard this removes: a temporary |
eeeebbbbrrrr
marked this pull request as draft
October 5, 2026 02:54
eeeebbbbrrrr
force-pushed
the
macro_rules-inline-bodies
branch
from
October 27, 2026 21:12
7fa3bc0 to
d8afa31
Compare
eeeebbbbrrrr
force-pushed
the
macro_rules-inline-bodies
branch
from
October 28, 2026 18:03
d8afa31 to
fbd8fbc
Compare
eeeebbbbrrrr
force-pushed
the
macro_rules-inline-bodies
branch
2 times, most recently
from
October 29, 2026 08:13
a2c9917 to
b17f0ea
Compare
eeeebbbbrrrr
force-pushed
the
macro_rules-inline-bodies
branch
from
October 29, 2026 17:05
b17f0ea to
8a8d30f
Compare
Generated macros for static inline functions always called the original function through its guarded native binding, so PG16+ scalar conversions such as Int32GetDatum paid for sigsetjmp and a backend-thread check on every call. When a value-returning definition's body uses no C macros, the generator now analyzes it as a private statement root that declares each parameter from its argument, and emits it like a translated macro body whose return yields the value. Arguments are still evaluated once and converted by C assignment rules, calls inside the body keep their guarded bindings, and a body the analyzer or renderer refuses falls back to the native call. Bodies are tokenized one active statement at a time and refused when any token was ever defined as a macro, because a macro replaced or undefined after the definition would change their meaning. Like the C macros these functions replaced, a translated body that performs no pointer, global, or call operation no longer needs unsafe. A translated body's documentation shell stays untyped until translated bodies have a type witness.
…calls A static inline function body may now use macros. The frontend keeps the tokens the preprocessor read for the body, and the compiler expands them in the final environment only when each macro, and every macro it reaches, has no definition after its use and is still defined at the end of the headers. Void functions without an early return are translated, while records still cannot pass into or out of a translation by value. A generated call to a translated function now expands that translation instead of calling the guarded binding, except among functions that reach themselves. Translated functions also request support for their own result and parameter types, which an enum result needed. A documentation shell whose witness would name a translated call stays untyped until translations have witnesses.
A translated static inline function declared each parameter as a local initialized in its own `let` statement, so Rust dropped an argument's temporaries before the body ran. A pointer into such a temporary, such as `CString::new(..).unwrap().as_ptr()`, then dangled, although the same call through the native adapter was valid. The translation now evaluates all arguments left to right in one `match` scrutinee and initializes the parameters from the bound values. Their temporaries live until the body finishes, as in a call, and every argument is evaluated before the body runs, as in C.
A generated call to a translated inline function passed each argument as a `@compiled` descriptor whose size slot was a `compile_error!`. A callee's `sizeof` of a local expands its parameters' safety checks, which expanded that size slot, so every invocation of the caller failed to compile. Compiled descriptors now carry a fifth slot holding the producing macro's own dead checks of its caller operands, and the safety check expands only that slot. The checks stay active, are always well typed, and no longer depend on whether the size alternative could be rendered, which also covers delegated macro arguments.
Body retention dropped a directive's tokens only on the physical line of its `#`. A block comment spanning lines, or a backslash-continued directive closing a skipped group, left the rest of the directive in the translated body and changed its result. The frontend now finds each directive's logical end after line splices and comments, for active and skipped directives alike. It refuses a body with a `#` that it cannot place at the start of a line.
The proof that a body's macros expand unchanged at the end of the translation unit assumed the active definition was the last recorded `#define`. `#pragma pop_macro` restores an earlier definition without a record, so a body could be translated with the wrong macro value. The proof now requires the active definition to be the last recorded definition, by location, and each expansion in the body to have used that definition.
Expansion records and skipped ranges identify a position by file and offset, and libclang reports skipped ranges for a file's first entry only. A template header included twice could therefore be translated with another inclusion's macros or conditional branches. The frontend now refuses bodies from a file that was included more than once without a recognized include guard, or that has a macro definition outside its first entry.
The fixtures did not exercise the inline-body proof's refusal for a macro that is reached through another macro and redefined later, or for names inside an invocation's arguments. New native cases cover both, including an enum constant that later becomes a macro.
When a translated inline function could not be rendered, its macro fell back to calling the original function, but demand planning only saw the translated body. The final support could then omit the generated wrapper that the call used, and an identity-only adapter for an address could be called as if it had a wrapper. Planning now covers each translated function's call root, and the final support is planned from the analyses actually emitted. Typed lowering no longer resolves a call to an identity-only adapter.
A translated inline function that failed to render, for example because bindgen and Clang disagree about an enum constant, fell back to a native call without recording why. The emission now keeps that reason as `fallback`, which the CLI's JSON output and pgrx-bindgen's macro report serialize. pgrx-bindgen's test fixture constructs the struct, so it gains the new field.
libclang spells punctuation and literals from the source buffer, so a token after a backslash-newline keeps the splice. PostgreSQL 18 and 19 put the opening brace of HeapTupleHeaderXminInvalid and HeapTupleHeaderIsHeapOnly after a splice, so their bodies were refused, and a spliced `#` or `%:` closing a skipped group let the next line's tokens leak into the body. Body spellings are now compared after the crate's splice removal, which follows Clang. Bodies containing a trigraph or a bare carriage return, which the line handling does not model, are not retained.
A header that the preprocessor entered more than once could still have its bodies translated when Clang considered it include-guarded, for example a template wrapped in an `#ifndef` it never defines. Later entries then replaced the expansion records that prove a body's macros unchanged, and a definition in a later entry was read with the first entry's skipped ranges, so translations silently differed from C. Expansion records from a later entry of a file are now ignored, and a definition outside its file's first entry keeps only the native call. The include counting and guard heuristics are removed.
A macro undefined when a body was parsed and later restored by `#pragma pop_macro` looked unchanged, because libclang records neither the undefinition nor the pragma. A body that reached such a name through another macro was translated with the restored definition and silently differed from C. The text of every file in the unit is now searched for pop pragmas, and a body that reaches a name one restores keeps the native call. When a pop's operand cannot be read, no body that uses a macro is translated.
Clang records only top-level macro expansions. A function-like macro named in an argument and expanded when the result was rescanned has no record, so once it was undefined the proof treated it as a plain name. A same-named function was then called where C had expanded the macro. A body name without an expansion record that was ever defined as a macro now keeps the native call.
A body that names macros is expanded after the headers by replaying its tokens with a space between each pair. When a macro it reaches stringified body tokens, the replay produced a differently spaced string, so `sizeof` or indexing of the literal silently differed from C. Such a body now keeps the native call, as a body that reaches token pasting already did.
…cts them A body that both multiplies and adds checks every value against the composed floating-point profile, which the default -ffp-contract=on rejects. A translated function with its own floating values was still emitted, so none of its calls compiled, and a floating argument to an integer parameter was rejected although a call converts it. Such a body now keeps the native call, and arguments are converted with the value profile that native call arguments use.
A translated function evaluates its arguments in its match scrutinee, outside generated `unsafe`, so its body never holds caller tokens. Its size witnesses still expanded dead checks of the caller's operands, which rejected an operand whose temporary still borrowed a local, such as `v.drain(..)`, with E0499. The operands it passed to translated callees also carried those checks, so a chain of translated calls ending in a size witness expanded exponentially. Both are now empty for translated bodies; macros keep their checks.
Callback signatures are reconciled twice, the second time with the C-unwind thunks generated for native calls. A function type that a thunk made inconsistent lost its signature in the second pass, but address getters still used the first pass's marker, so the generated support did not compile. Since fallback call roots are planned, a translation that falls back could trigger this. Address, field and enum adapters now see only the second pass's markers, so such an address is refused instead.
A fallback reason came from the translated body's analysis, but the emission reports the call root's analysis, so its token range pointed past or into unrelated nodes. Fallback reasons now keep their code, message and spans without a token range.
A macro that delegates to another with a composite argument passes its own operand checks in the descriptor's checks bracket. In a size context that bracket is the only place the caller's unsafe tokens are checked, but emptying it passed every test. The context oracle now requires E0133 for unsafe operands of such a delegation under `sizeof`.
Demand planning marks every function a renderable root calls as called, so no rendered call can reach an identity-only adapter today and no test exercises the filter. The comment now says it guards calls that planning did not see.
…inters A body that multiplies and adds checks every value against the composed floating-point profile, which the default -ffp-contract=on rejects. The check that keeps such bodies native saw only floating locals and declared values, so a body that read or wrote a double through a member, an index or a dereference was translated and none of its calls compiled. It now also refuses a body with a floating object type the compiler proved, or an access or call through a pointer whose type analysis did not establish.
The search for pop_macro pragmas read raw file bytes, but Clang removes line splices before it reads a pragma. A pop spelled across a splice restored a macro the search never saw, so a body that reached that macro was translated with the restored definition and silently differed from C. Splices are now removed before the search. The crate's splice recognition now works on bytes, so the pragma search, token spellings and directive line ends share it. A file with a ??/ trigraph or a bare carriage return, which can splice in ways the search does not model, leaves no body that names a macro, so the fixture bodies that use them move to their own header.
Inline definition text and body checks read each header through the clang crate's File::get_contents, which panics on a header that is not UTF-8 and stops at a NUL byte. Under mandatory generation that panic failed the build with no useful context. Headers are now read from their files as bytes, the way the pop pragma search already reads them. A definition whose own text is not UTF-8 gets neither a body nor a native call adapter, since its text is kept as a string and libclang spells its tokens lossily.
The floating check for translated bodies named the deferred node kinds that could hold a floating value, so a kind added later would be assumed not floating. It now names the deferred kinds that cannot: an address, a string, and a node that takes the type of an operand the check covers on its own. Every other deferred node without a proved object type keeps the body native under a profile that rejects its floating arithmetic.
… value A translated body read a member of a record value, such as a call's result, from a copy of the record. The runtime refuses an array member there, since it would decay into a pointer into that copy, so a body that indexed one was translated and none of its calls compiled. Emission now refuses an array member of a record value. A translated body is also refused when such a member's type is not established, because its types do not depend on its caller, and it keeps the native call.
Inline definition bytes were cached by span path, which names the file Clang parsed only while spans join a relative path to the same working directory that Clang resolved it from. The cache is now keyed by Clang's file identity, so a definition is always read from the file Clang parsed. A test checks that two headers with the same relative path, one beside the main header and one found through a relative include directory, keep their own text and bodies.
The search for pop_macro pragmas read only the unit's files. A -D definition whose replacement list is a _Pragma that pops a macro restores that macro wherever a header expands it, so a body that reached the popped name was translated with the end-of-unit definition and silently differed from C. The search now also reads each -D definition, whether it is joined to the option or follows it.
An inline definition whose own text was not UTF-8, such as one with a Latin-1 byte in a comment, lost its native call adapter along with its body. Emission then reported that the source exceeded its size budget, which pointed away from the encoding. The text is now kept with lossy conversion to document the native call, and only the body is refused, since libclang spells its tokens lossily.
The architecture said a header need not be UTF-8, but analysis reads the compiler's preprocessed output and macro dumps as UTF-8. A macro definition that is not UTF-8 fails inspection, and a string literal that is not fails symbolic expansion and the native support compile, each with an error. Converting that output lossily would change a literal analysis compares or compiles. The limitation is now documented, and a test checks that such a literal fails preparation instead of being changed.
The documentation shell of a translated static inline function kept the diverging placeholder, and so did a macro whose witness reached a call to one. A translated body's shell now yields `type_witness::<M>()` for its result marker, or `()` for a void function, after borrowing every argument. In a witness, a translated call collapses to its declared result like a native call, so no witness names the callee's internal value arm. A result whose C type is an integer typedef is named through its `__pgrx_c_types` alias. Generated macros of ordinary builds are unchanged.
A generated call to a translated inline function passed the caller's safety checks in each operand's checks bracket. The translation expands only each operand's value, in its match scrutinee, so those checks were never expanded, although comments said that a sizeof in the callee checked them. The checks bracket now holds the same compile_error! as the other non-value brackets, so a change that expands it fails to build instead of silently checking nothing. A macro whose size witness wraps a translated call in unsafe still checks its own operands. A test requires every operand of a translated call to carry only a value.
A macro or translated body that calls a translated inline function documented that call as a guarded native call, and its Panics section left out the callee's own conditions. For example, BTPageGetMeta! omitted the checked pointer arithmetic of the PageGetContents translation it runs. Each translation's conditions, including those of the translations it expands, are now collected once per batch with callees before callers. A call that expands a translation documents those conditions instead of the guarded native call, which native and indirect calls still document.
Whether a static inline function's body is translated, and so whether a call needs unsafe, depends on the PostgreSQL build and on the libclang that inspects it, not only on the version. An --enable-cassert build keeps bodies that use Assert native, because Assert reports its file and line, and some libclang releases keep a body native when one of its statements starts with a macro. The guide and the pgrx-c-macros README now say so, and that any later definition of a macro the body uses keeps the native call, even an identical one. The return macro tests keep their unsafe blocks for builds where a Datum conversion is a native call and allow unused_unsafe where it is pure.
Release generation ran on x86_64-unknown-linux-gnu with PGDG PostgreSQL 15.19, 16.15, 17.11, 18.6 and 19rc1 configured and every version required. The snapshot now documents translated inline functions with typed witnesses. Its support gains the field markers that translated bodies use and drops call signatures that only native calls of now-translated functions needed. Every shell is typed, and the bindings and OID files are unchanged.
On x86_64, Clang 14 and 15 without optimization restore %rax when a preserve_most function returns, so the fixture helper's result was replaced and both the original C and the generated call printed garbage. The helper now returns nothing and scales a value through a pointer, which keeps the calling convention no generated adapter calls. The test also checks that the refused macro's analysis is a candidate, since only candidates reach demand planning.
Typed lowering skips identity-only function adapters when it resolves a call, because they define no call wrapper. Demand planning normally marks every called function, so no test reached this filter and removing it left every test passing. The new unit test plans only an address use of a static inline function and then renders a macro that calls it. It checks that the call is refused instead of naming a wrapper the support never defines.
A call that demand planning missed could reach a function whose adapter was generated only as an address witness. Typed lowering skipped that adapter and fell through to the other ways to call the function, so the call was refused only because the function had no binding or its prototype needed native transport. FunctionAdapters::call_refusal now refuses each identity-only function along with each rejected adapter, and typed lowering checks it before any adapter or binding. The public pipeline plans every call it renders, so generated bindings and macros do not change. The unit test now also covers a plain-char function that has a bindgen binding.
BindingCatalog still had a public inline_translations field that batch emission filled while rendering. A caller could set it so emit_with_bindings expanded a translated inline body instead of calling the function, and so a macro documented different panics. The translations now live with the generated adapters in private rendering state that only batch emission fills, and the field is gone. Callers that left the field empty, as pgrx-bindgen does, get the same output. A compile_fail doctest checks that the field cannot be named on BindingCatalog.
eeeebbbbrrrr
force-pushed
the
macro_rules-inline-bodies
branch
from
October 29, 2026 20:35
8a8d30f to
1563e3c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated macros for PostgreSQL static inline functions always called the original function through its guarded native binding, so PG16+ conversions such as
Int32GetDatumpaid forsigsetjmpand a backend-thread check on every call.The generator now translates a function's body when the analyzer supports every statement, emitting it like a macro body whose
returnyields the value, with each argument evaluated once and converted to its parameter type.Macros in the body are expanded only when the definition parse proves each of them, and every macro they reach, unchanged since its use;
__LINE__-style builtins, token pasting, records passed by value, and floating-point objects reached through pointers keep the native call.Generated calls to a translated function expand its translation, except among functions that reach themselves, and calls to everything else keep their guarded bindings.
A translation that performs no pointer, global, or call operation no longer needs
unsafe, but which functions translate also depends on the PostgreSQL build and on libclang, so code built against several installations should keepunsafeand allowunused_unsafe.docs.rs documents each translated function and call with a typed shell whose result type matches the translation.
Stacked on #2391; #2390 is rebased onto this branch.