Skip to content

JRuby: test_scan_integer_base_16 failed only with ubuntu-latest + jruby-head #125

Closed
Closed
JRuby: test_scan_integer_base_16 failed only with ubuntu-latest + jruby-head#125

Description

@kou
kou
opened on Dec 12, 2024

https://github.com/ruby/strscan/actions/runs/12288841240/job/34293324916#step:11:10

Error: test_scan_integer_base_16(TestStringScanner): Java::JavaLang::ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
org.jruby.dist/org.jruby.util.ByteList.get(ByteList.java:632)
org.jruby.ext.strscan.RubyStringScanner.scan_base16_integer(RubyStringScanner.java:621)
org.jruby.ext.strscan.RubyStringScanner$INVOKER$i$0$0$scan_base16_integer.call(RubyStringScanner$INVOKER$i$0$0$scan_base16_integer.gen)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:82)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:201)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:188)
org.jruby.dist/org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:220)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:466)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:244)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
org.jruby.dist/org.jruby.RubyClass.finvokeWithRefinements(RubyClass.java:525)
org.jruby.dist/org.jruby.RubyBasicObject.send(RubyBasicObject.java:1667)
org.jruby.dist/org.jruby.RubyBasicObject$INVOKER$i$send.call(RubyBasicObject$INVOKER$i$send.gen)
org.jruby.dist/org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:423)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:466)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:244)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
org.jruby.dist/org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:118)
org.jruby.dist/org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:136)
org.jruby.dist/org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)
org.jruby.dist/org.jruby.runtime.Block.call(Block.java:148)
org.jruby.dist/org.jruby.RubyProc.call(RubyProc.java:322)
org.jruby.dist/org.jruby.RubyProc$INVOKER$i$call.call(RubyProc$INVOKER$i$call.gen)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
org.jruby.dist/org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
org.jruby.dist/org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
org.jruby.dist/org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:88)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:238)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:225)
org.jruby.dist/org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:228)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:476)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:293)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:324)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:118)
org.jruby.dist/org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:136)
org.jruby.dist/org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:76)
org.jruby.dist/org.jruby.runtime.Block.yieldSpecific(Block.java:158)
org.jruby.dist/org.jruby.ir.runtime.IRRuntimeHelpers.yieldSpecific(IRRuntimeHelpers.java:499)
org.jruby.dist/org.jruby.ir.instructions.YieldInstr.interpret(YieldInstr.java:84)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.processOtherOp(StartupInterpreterEngine.java:155)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:98)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:128)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:115)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:446)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:92)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:103)
org.jruby.dist/org.jruby.ir.instructions.CallBase.interpret(CallBase.java:545)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:363)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
org.jruby.dist/org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:76)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:164)
org.jruby.dist/org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:151)
org.jruby.dist/org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:212)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:456)
org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)
org.jruby.dist/org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:346)
org.jruby.dist/org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:66)
org.jruby.dist/org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:118)
org.jruby.dist/org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:136)
org.jruby.dist/org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)
org.jruby.dist/org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
org.jruby.dist/org.jruby.runtime.Block.call(Block.java:144)
org.jruby.dist/org.jruby.RubyProc.call(RubyProc.java:354)
org.jruby.dist/org.jruby.Ruby$ProcExitFunction.applyAsInt(Ruby.java:5844)
org.jruby.dist/org.jruby.Ruby$ProcExitFunction.applyAsInt(Ruby.java:5829)
org.jruby.dist/org.jruby.Ruby.userTeardown(Ruby.java:3421)
org.jruby.dist/org.jruby.Ruby.tearDown(Ruby.java:3319)
org.jruby.dist/org.jruby.Ruby.tearDown(Ruby.java:3309)
org.jruby.dist/org.jruby.Main.internalRun(Main.java:285)
org.jruby.dist/org.jruby.Main.run(Main.java:225)
org.jruby.dist/org.jruby.Main.main(Main.java:197)

Activity

  1. kou commented on Dec 12, 2024

    @kou
    kou
    on Dec 12, 2024
    MemberAuthor

    @headius @byroot Do you think this is a strscan problem or a JRuby problem?

  2. headius commented on Dec 12, 2024

    @headius
    headius
    on Dec 12, 2024
    Contributor

    Probably a bug in the Java strscan logic.

  3. headius commented on Dec 12, 2024

    @headius
    headius
    on Dec 12, 2024
    Contributor

    FWIW if you ever see a Java exception come out of executing normal Ruby stuff in JRuby, it's a bug. Even if the caller of scan_base16_integer was doing something weird, they shouldn't be able to produce a broken ByteList that would do this.

  4. headius commented on Dec 12, 2024

    @headius
    headius
    on Dec 12, 2024
    Contributor

    Looks like this was landed by @byroot in eb875ea. The C code and the Java code have some subtle differences. I'll see if I can find the bug.

  5. headius commented on Dec 12, 2024

    @headius
    headius
    on Dec 12, 2024
    Contributor

    I could not reproduce the crash locally... perhaps whatever test is hitting this has input that varies from run to run?

    I did the following attempt to re-port the C version of the code, and it works, but that doesn't say much when it doesn't crash for me in the first place:

    diff --git a/ext/jruby/org/jruby/ext/strscan/RubyStringScanner.java b/ext/jruby/org/jruby/ext/strscan/RubyStringScanner.java
index 7e7c492..23364d3 100644
--- a/ext/jruby/org/jruby/ext/strscan/RubyStringScanner.java
+++ b/ext/jruby/org/jruby/ext/strscan/RubyStringScanner.java
@@ -608,40 +608,44 @@ public class RubyStringScanner extends RubyObject {
             throw runtime.newEncodingCompatibilityError("ASCII incompatible encoding: " + str.getEncoding());
         }
 
-
         ByteList bytes = str.getByteList();
         int curr = this.curr;
 
-        int bite = bytes.get(curr);
-        if (bite == '-' || bite == '+') {
-            curr++;
-            bite = bytes.get(curr);
+        int remaining_len = bytes.realSize() - curr;
+
+        if (remaining_len <= 0) {
+            return context.nil;
         }
 
-        if (bite == '0' && bytes.get(curr + 1) == 'x') {
-            curr += 2;
-            bite = bytes.get(curr);
+        int len = 0;
+
+        if (bytes.get(len) == '-' || bytes.get(len) == '+') {
+            len++;
         }
 
-        if (!((bite >= '0' && bite <= '9') || (bite >= 'a' && bite <= 'f') || (bite >= 'A' && bite <= 'F'))) {
-            return context.nil;
+        if ((remaining_len >= (len + 2)) && bytes.get(len) == '0' && bytes.get(len + 1) == 'x') {
+            len += 2;
         }
 
-        while ((bite >= '0' && bite <= '9') || (bite >= 'a' && bite <= 'f') || (bite >= 'A' && bite <= 'F')) {
-            curr++;
-            if (curr >= bytes.getRealSize()) {
-                break;
-            }
-            bite = bytes.get(curr);
+        if (len >= remaining_len || !isHexChar(bytes.get(len))) {
+            return context.nil;
         }
 
-        int length = curr - this.curr;
-        prev = this.curr;
-        this.curr = curr;
         setMatched();
         adjustRegisters();
+        prev = curr;
+
+        while (len < remaining_len && isHexChar(bytes.get(len))) {
+            len++;
+        }
+
+        this.curr = curr + len;
+
+        return ConvertBytes.byteListToInum(runtime, bytes, 0, len, 16, true);
+    }
 
-        return ConvertBytes.byteListToInum(runtime, bytes, prev, curr, 16, true);
+    private static boolean isHexChar(int c) {
+        return Character.isDigit(c) || ('a' <= c && c <= 'f') || ('A' <= c && c <= 'F');
     }

  6. byroot commented on Dec 12, 2024

    @byroot
    byroot
    on Dec 12, 2024
    Member

    @headius could you submit this as a PR?

  7. byroot commented on Dec 12, 2024

    @byroot
    byroot
    on Dec 12, 2024
    Member

    As for the failure, I honestly have no idea what could have happened, but my Java knowledge is very limited, so if there's indeed a difference with the C implementation, we should start by making them converge and see if the problem happens again.

  8. headius commented on Dec 12, 2024

    @headius
    headius
    on Dec 12, 2024
    Contributor

    I will push this as a PR along with some code to better log a backtrace if we see this happen again. As it is the Java trace only has JRuby interpreter frames in it, so I can't see what line in the test caused it to fail.

  9. headius
    added 3 commits that reference this issue on Dec 12, 2024

    Re-port scan_base16_integer from the C code

    acf7b67

    Re-port scan_base10_integer from the C code

    de7d9c0

    Add env for JRuby to force compile for better errors

    79f682f
  10. kou
    closed this as completedin ea0786bon Dec 13, 2024
  11. kou
    added a commit that references this issue on Dec 13, 2024

    Re-port new integer-scanning utility methods (#127)

    ea0786b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions