Skip to content

Should tokens passed through macro_rules be able to join? #127123

Open

Description

@petrochenkov

Should this example

macro_rules! mac {
    ($a:tt sep $b:tt) => { 1 $a$b 2 } 
}

fn main() {
    mac!(= sep =);
}

emit 1 == 2 and compile successfully?
Or, in other words, should the first emitted = have the Spacing::Joint set, if the output is processed by a proc macro?

We need some well-defined rule for setting spacing for tokens produced from macro variables.

Possible alternatives:

    • Always use Joint.
      • This is very unreasonable.
    • Always use Alone.
      • This is the first reasonable alternative.
    • Inherit from the passed token - that means Alone in the example above because there is a space after the first = in mac!(= sep =).
      • Probably unreasonable, the token sequence in the input have little relation to the output sequence for which we emit the spacing, in the example above it is demonstrated by sep being present in the input but not in the output.
      • Note: the issue is about single passed token trees (or last token trees in sequences), tokens passed in the middle of something like $($tt)* should of course use this option.
    • Inherit from the macro parameter declaration - i.e. $a:tt in the example (or just the $a part of it).
      • This doesn't seem reasonable to me
    • Inherit from the macro parameter use, that means Joint in the example above because there is no space after $a in 1 $a$b 2.
      • This is the second reasonable alternative.

Right now the spacing is emitted inconsistently, sometimes it follows the rule 2., sometimes 3..
Adopting any consistent rule may cause breakage (@nnethercote may remember which exactly) and needs to go through crater.

  • I would personally try the rule 5. first, because it gives macro author freedom to control and change spacing as they want, unlike rule 2. that prevents joining entirely.
  • However we may be forced to adopt rule 2. instead due to backward compatibility issues. It would be less flexible, but macro variables creating a "barrier" for joining is also a reasonable enough model.

Activity

  1. added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Jun 29, 2024
  2. added
    T-langRelevant to the language team
    A-macrosArea: All kinds of macros (custom derive, macro_rules!, proc macros, ..)
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    A-proc-macro-back-compatArea: Backwards compatibility hacks for proc macros
    WG-macrosWorking group: Macros
    on Jun 29, 2024
  3. petrochenkov commented on Jun 29, 2024

    @petrochenkov
    ContributorAuthor

    Partially switching tokens from tt matcher to Spacing::Alone was tested with crater in #119412 (comment) and there were no regressions.
    (Group tokens and tokens in sequences were not switched.)

  4. nnethercote commented on Jul 1, 2024

    @nnethercote
    Contributor

    I saw some cases like this in #125174, where identifiers were being joined together. At first I thought about inheriting (from either use or declaration, or a combination), but after a while I concluded that Alone was the most sensible thing.

  5. removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Jul 5, 2024
  6. added
    C-discussionCategory: Discussion or questions that doesn't represent real issues.
    on Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-macrosArea: All kinds of macros (custom derive, macro_rules!, proc macros, ..)A-proc-macro-back-compatArea: Backwards compatibility hacks for proc macrosC-discussionCategory: Discussion or questions that doesn't represent real issues.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.T-langRelevant to the language teamWG-macrosWorking group: Macros

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions