Regression in rlua crate tests between rustc 1.23.0 and 1.24.0, only on windows.

[The following explanation of the problem contains a lot of assumptions about the problem's nature, I could be extremely wrong! Take this explanation with a grain of salt]

Wild guess, it has something to do with #46833.

So, I'm not sure this is a rust bug exactly, but I wanted to make you aware of a crate regression that's causing me quite a lot of problems. Ultimately, I believe this comes from me doing some things that are not.. exactly definitely kosher, but that I don't have a convenient workaround for.

Ultimately, the problem stems from the Lua C API having error handling fundamentally based around setjmp / longjmp. In order to write a usable wrapper in Rust, without having to write inconvenient, slow C shims, Rust must, at times, call into Lua C API functions that in turn trigger a longjmp, and that longjmp must necessarily pass over Rust frames.

I realize how unsafe this is! However, there are some limitations here that I think make it at least somewhat reasonable to be able to do:

There are only Copy types on the rust stack frame being jumped over.
Rust is never calling setjmp, only triggering a longjmp through a Lua C API call (such as lua_error).
Making this unsafe to do means that it is impossible to wrap a C API such as Lua's without resorting to writing more C.

This last point is unfortunate, and causes me a lot of headaches, and at least one strange practical problem. We at Chucklefish build rlua for consoles, but for consoles the gcc crate does not work, and we have to build Lua out of tree (also to include Lua console specific patches). Needing to include C shims in rlua means that we would also have to copy paste those shims into the console projects, and this is a less than ideal situation.

The reason I think this issue has to do with #46833 is that I can actually fix the regression if I simply add an #[unwind] attribute to each rust function that calls lua_error. However, this is only necessary for some reason on windows, and the attribute is currently unstable!

I think it's completely reasonable to need to mark functions as #[unwind] if they will trigger a longjmp, but I'm not exactly sure what to do in the meantime until unwind_attributes becomes stable. Maybe this behavior on windows IS simply a bug? Maybe it's a bug on linux / macos that this doesn't abort? I would especially not like to have to write C shims if ultimately I can get rid of them when unwind_attributes becomes stable.

(Side Note: Chucklefish actually uses the stable rust compiler in production now, so this is especially annoying. We were very excited about the 1.24 release, because it includes incremental compilation and rustfmt-preview while also being stable, and we noticed this bug in trying to switch to the stable compiler on windows.)

Edit: If people have to dig into the rlua source, I'm sorry, it is a tangled nest of "unsafe, unsafe everywhere". The best example to look at is the test_error test, and the method Lua::create_callback_function. Adding the #[unwind] attribute to: callback_call_impl, safe_pcall, and safe_xpcall fixes that test on windows, on unstable.

Edit 2: Edited for clarity, but also I need to point out that I need to do something, because this bug currently makes rlua just basically broken on windows. It's not "just" a test failure, error handling in general will just abort.

One possibility as to why that it only happens on windows is that the implementation of longjmp on that platform uses the same exception framework that Rust uses for panics.

One possibility as to why that it only happens on windows is that the implementation of longjmp on that platform uses the same exception framework that Rust uses for panics.

That is interesting! That WOULD explain the behavior I'm seeing.

I'm not exactly sure what to do in the meantime until unwind_attributes becomes stable.

It's possible to temporarily unlock unstable features on stable channel with env var (RUSTC_BOOTSTRAP=1), seems acceptable if you have a critical situation like this.

It's possible to temporarily unlock unstable features on stable channel with env var (RUSTC_BOOTSTRAP=1), seems acceptable if you have a critical situation like this.

That's incredibly helpful, thank you!

Here is a summary of the current situation as I understand it: mlua-rs/rlua#71 (comment)

I may have gotten some details wrong, if I have feel free to ~~yell at me~~ point that out.

I guess when they say it's a "C API" they are not kidding. :p

Would it make sense to make an standalone wrapper, called "Universal API for Lua, with C calling conventions"?

I guess when they say it's a "C API" they are not kidding. :p

Lua's API is not my favorite :(

@wycats

Hmm. I think there's a good case to be made for disabling #46833 -- or at least making it opt-in -- until there are stable attributes to opt-out. @wycats points out that this likely to affect Helix as well. I would definitely prefer not to have public packages relying on RUSTC_BOOTSTRAP=1 -- it's a recipe for undermining the whole Rust stability system.

Re-reading the comments on that PR, I see that I wrote this:

Note though that this is a change in behavior -- albeit only quasi-defined behavior -- and it feels like it ought to go through the RFC process. Still, it'd be good to have a working implementation so that we can do a crater run and assess possible impact.

But this got overlooked.

I would definitely prefer not to have public packages relying on RUSTC_BOOTSTRAP=1 -- it's a recipe for undermining the whole Rust stability system.

I knew that was wrong when I did it, I'll forget this hack exists :P

Note though that this is a change in behavior -- albeit only quasi-defined behavior -- and it feels like it ought to go through the RFC process. Still, it'd be good to have a working implementation so that we can do a crater run and assess possible impact.

But this got overlooked.

Something I wanted to point out, is that even if a crater run was done, it wouldn't have caught my issue at least because as I understand it they're only done on linux?

If there was some kind of donation box for running crater on windows I would contribute to it!

@kyren

@kyren I believe crater actually tests windows these days, but I could be wrong.

Also, cross-posting from the chucklefish/rlua repo:

I'm trying to understand this a bit better -- it seems like the problem is pretty specific to SEH. That is, ordinarily, longjmp wouldn't trigger this panic, precisely because it doesn't unwind in a structured fashion, but rather just directly pops off the frames. In other words, we can't really "catch" a longjmp normally. But I guess that in windows longjmp is "exception-handling aware". That means, if I understand correctly, that ironically SEH is probably the one environment where a longjmp would actually be safe in Rust, since we would run dtors like normal. But precisely because we can catch it, it's also the one environment where longjmp will panic. Maybe I'm missing some subtlety though. @alexcrichton may remember more about the details of SEH.

Can anyone verify if this is correct?

I think in my ideal world, we would not require any attributes -- rather, we would have a way to skip the panic in the case where Rust is using the native unwind facility (but you would opt-in to that, probably, similar to #[repr(C)]). I suppose that is what the #[unwind] attribute basically says, but it seems a bit wrong. (That is, it says "this function uses Rust's unwind abilities", but I think I'd rather that you declare how Rust should implement unwinding...? Have to think about it.)

@nikomatsakis

this requires reverting a bunch of changes to gtk-rs and related crates

Say more? Did they have their own panic guards which were removed?

@nikomatsakis Yes, exactly that :) But fortunately not in any released version yet (unless there was a release of some crate outside of the gtk-rs organization that I'm not aware of). It was planned to release this really soon now though...

I don't really know the state of it or whether it would be appropriate for a stable point release, but it would be nice if #48572 could be merged instead of a straight rollback, to avoid missing out on the feature for another rust stable version.

I do feel a bit guilty for triggering the reverted behavior, not having to worry about panic unwinds at extern function boundaries is quite nice, even if I'm not currently relying on it.

@kyren

We plan to discuss this in the core team meeting tonight. Given #48572, we may forego the revert. @kyren, is it ok w/ you to wait until the next beta to have a fix? (I'm not sure how I feel about backporting #48572 to beta, but I guess it's ok -- backporting to stable seems too risky to me, personally.)

@kyren

@kyren Did you try @petrochenkov 's suggestion to change longjmp to __instrinsic_longjmp ? If so, did it work?

@kyren

@kyren
Not sure if it will be of any help in this situation, but you can actually longjmp on MSVC without performing unwinding by using two-argument int __intrinsic_setjmp(jmp_buf buf, void* frame_address) with NULL as a second argument instead of standard setjmp.
I've seen it used for jumping from jitted code living on a separate stack (you can't unwind between stacks) - people basically used #define setjmp(buf) __intrinsic_setjmp((buf), NULL) on top of their code.

So I finally got around to trying this and I can confirm that this DOES work, thank you very much for the suggestion!

Since I basically have a fix for this, I'm not in a huge rush to have this change in the rust stable compiler, but this is only speaking for me. This issue probably causes problems for other people that have to deal with at least any of Lua or Ruby or maybe libpng, depending on how strict they are about handling errors properly or how deep into the APIs they go.

I would LIKE it if the fix was in the next stable version of Rust, but even then it's not critical because like I said, I have a fix. The main thing I really needed from all of this was an assurance that longjmp APIs are at least supposed to be compatible with Rust in general, which just practically saves me a lot of heartache. If #48572 lands in nightly and the regression test for longjmp stays long term, but I have to wait two rust stable cycles to remove the now small rlua hack, that's certainly not the end of the world. The only way this would be a problem were if the fix for longjmping APIs ended up requiring an unstable feature that was a very long way off, or the hacky fix I have broke for some other reason in the interim, or if #48572 or similar was not merged, or if there was another decision that similarly changed direction so that I can't reliably call into a longjmp based API. If the fix did make it into beta though that would certainly feel a lot more "certain" to me and allow me to relax about it a lot more :D

@kyren Did you try @petrochenkov 's suggestion to change longjmp to __instrinsic_longjmp ? If so, did it work?

I was typing this reply as I saw your comment :D

rustc: Tweak funclet cleanups of ffi functions

triage: leaving assigned to niko but it might be good for him to delegate to alex or someone else if there's someone else taking point on this now.

Rollup merge of rust-lang#48572 - alexcrichton:noexcept-msvc2, r=eddyb

Given #48572, we may forego the revert.

1.24.1 was released with this reverted now.

@alexcrichton

So, we talked about this in the core team meeting, and -- as you can see -- decided to revert after all. It was a bit of a tough call, but we felt like when in doubt, we ought to bias towards "back off and try again more carefully". The plan is to land @alexcrichton's change in #48572, which should enable us to re-enable the abort-guards by default. We are also keeping the explicit #[unwind] attributes I added in #48380 (#[unwind(aborts)] and #[unwind(allowed)]), though those remain unstable.

(I still think we ought to address the interop question in a more thorough way, however.)

@sdroege I do apologize for whatever churn this caused gtk-rs and anyone else!

@nikomatsakis

@nikomatsakis No problem, "git revert" is cheap and there was no release yet. All options were bad, let's just hope it sticks the next time :)

Since the revert made it into 1.24.1, this is now fixed! Thanks for the great work, closing!

Navigation Menu

Repository navigation

Regression in rlua crate tests between rustc 1.23.0 and 1.24.0, only on windows. #48251

72 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Regression in rlua crate tests between rustc 1.23.0 and 1.24.0, only on windows. #48251

Description

Activity

sfackler commented on Feb 16, 2018

kyren commented on Feb 16, 2018

petrochenkov commented on Feb 16, 2018

kyren commented on Feb 16, 2018

kyren commented on Feb 17, 2018

SoniEx2 commented on Feb 18, 2018

kyren commented on Feb 19, 2018

nikomatsakis commented on Feb 19, 2018

nikomatsakis commented on Feb 19, 2018

kyren commented on Feb 19, 2018

kyren commented on Feb 19, 2018

nikomatsakis commented on Feb 19, 2018

72 remaining items

sdroege commented on Feb 28, 2018

kyren commented on Feb 28, 2018

nikomatsakis commented on Feb 28, 2018

diwic commented on Feb 28, 2018

kyren commented on Feb 28, 2018

pnkfelix commented on Mar 1, 2018

sdroege commented on Mar 2, 2018

nikomatsakis commented on Mar 2, 2018

sdroege commented on Mar 2, 2018

kyren commented on Mar 3, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions