Repository navigation
Better Error Message When Parsing Greek Question Mark (and similar confusing characters) #25957
Description
Current:
fn main() {
let a = 5;
}<anon>:2:14: 2:15 error: unknown start of token: \u{37e}
<anon>:2 let a = 5;
^
playpen: application terminated with error code 101
Do you know what the error message is here? It's that I'm using the Greek Question Mark instead of a semicolon, causing a parse error. And while there is an error message, it's extremely opaque.
A better error message would actually print the token that is in error directly in the error message, e.g. as @niconii suggsted:
error: expected one of `.`, `;`, or an operator, found unicode character U+37E `;`
Using the human readable U+37E over the Rust encoding thing (\u{37E}) would definitely help.
In the specific case of the Greek Question Mark, it looks exactly like a semicolon, and symbols that look like other symbols are pretty common in Unicode, but perfectly enumerable. If we had a table of unicode point to actually used symbol, we could display an even better notice message:
error: expected one of `.`, `;`;, or an operator, found unicode character U+37E `;`
note: Unicode character U+37E (Greek Question Mark) looks like a semicolon, but is not.
Reacted by BestCodes
Reacted by BestCodes
Activity
DanielKeep commentedon Jun 2, 2015
Useful resource might be the list of Unicode "confusables": http://www.unicode.org/Public/security/revision-06/confusables.txt
Just going on printable ASCII punctuation, the list contains 486 possibly confusing glyphs. For reference, the command I used to work that out was:
Manishearth commentedon Nov 14, 2015
Note that the error is in the lexer, not parser, so errors of the type "expected ... found" are right out. The parser only knows what to expect once everything has been converted into tokens.
The most we can do here is make it detect the confusable unicode characters.
Manishearth commentedon Nov 14, 2015
Willing to mentor this.
You basically need to maintain an array of confusable characters and check against them where this error is emitted in
src/libsyntax/parse/lexer/mod.rsThe error would finally look something like:
Manishearth commentedon Nov 14, 2015
Let me lknow if you need help or clarification!
Auto merge of #29837 - Wafflespeanut:unicode_chars, r=Manishearth
pickfire commentedon Sep 15, 2020
I think it may still not be good enough, maybe we should have a clearer suggestion to change it to semicolon? Changing the wording would be good enough probably.