Tracking Issue for waker_getters

Feature gate: #![feature(waker_getters)]

This is a tracking issue for getters of data and vtable pointers for RawWaker, and the method to get RawWaker from Waker.

Public API

// core::task
impl RawWaker {
    pub fn data(&self) -> *const ();
    pub fn vtable(&self) -> &'static RawWakerVTable;
}
impl Waker {
    pub fn as_raw(&self) -> &RawWaker;
}

Steps / History

Implementation: Implement RawWaker and Waker getters for underlying pointers #91828
Final comment period (FCP)
Stabilize waker_getters #129919

Unresolved Questions

Do we also need fn into_raw(self) -> RawWaker for Waker? Feature: Provide getters of data and vtable pointer for RawWaker? #87021 (comment)

Maybe we could stabilize this? Note sure what I could do next. @rust-lang/libs-api

I am concerned about as_raw, it's not clear to me how it is intended to be used. It takes Waker by reference and returns a RawWaker, which you can then decompose into a vtable and data pointer. However I don't see how you can then re-construct a reference to the original Waker: doing so with from_raw will create a second Waker which will call the drop function in the vtable when dropped. This can lead to a double-drop once the original Waker is dropped as well.

@Amanieu

@Amanieu

I am concerned about as_raw, it's not clear to me how it is intended to be used. It takes Waker by reference and returns a RawWaker

It returns &RawWaker, not RawWaker, thus it's a getter rather than an ownership transfer. Its purpose is to allow getting or checking the underlying data. Reconstructing a [Raw]Waker from the result pointers is indeed incorrect, and unnecessary due to the existence of Waker::clone.
If we do want reconstruction, I'd consider adding another fn into_raw(Waker) -> RawWaker, which also consists with Waker::from_raw. But I think it's less useful, because we can only access &mut Context and get a &Waker. We don't own Waker without explicitly cloning it somewhere.

Then I'm afraid that I don't understand how as_raw is intended to be used. What can you actually do with the returned vtable and data? I can understand passing it through FFI, but then you eventually have to pass it back to Rust code to use it as a Waker.

@Amanieu

@Amanieu

Then I'm afraid that I don't understand how as_raw is intended to be used.

It's effectively downcast_ref for Waker. It allows validating Waker VTables, ad-hoc optimization, or retrieving data from known Wakers, where the async runtime only supports Wakers from itself. I found an example in,
https://github.com/embassy-rs/embassy/blob/be55d2a39eb399d693d11f37dfa4b87cd2bd3c7a/embassy-executor/src/raw/waker.rs#L42

I can understand passing it through FFI, but then you eventually have to pass it back to Rust code to use it as a Waker.

In case of passing through FFI, we need do &Waker -> [*const (); 2] -> &Waker ¹ without ownership transfer. The last step can be done via reconstructing ManuallyDrop<Waker> and using the reference to inner. If we want to transfer the ownership via, to say fn into_raw(Waker) -> RawWaker, we need an extra Waker::clone during every poll.

Note that the ManuallyDrop<Waker> - &Waker pattern above can be found in,
https://github.com/tokio-rs/tokio/blob/c693ccd210c7a318957b0701f4a9632b2d545d6a/tokio/src/util/wake.rs#L18-L22

Since the VTable representation is repr(Rust). Here we actually need to create a new repr(C) VTable for forwarding method calls, rather than to pass the pointer through FFI directly. This is doable currently. ↩

I feel that a better story is needed for getting a &Waker from a &RawWaker. Perhaps something like this?

impl Waker {
    unsafe fn with_raw<T>(raw: &RawWaker, f: impl FnOnce(&Waker) -> T) -> T;
}

Also I do think that we should add into_raw for consistency with from_raw.

I feel that a better story is needed for getting a &Waker from a &RawWaker. Perhaps something like this?
impl Waker {
    unsafe fn with_raw<T>(raw: &RawWaker, f: impl FnOnce(&Waker) -> T) -> T;
}

This API seems to be the only choice since we cannot let temporary Waker instance escape, but it would be less flexible for the tokio case. It's not possible to return a impl Deref<Target = Waker> via this API. For the FFI case, it seems to work though.

Also this API is more like a helper, which is able to be implemented outside std (RawWaker::data cannot, due to repr(Rust)). Should it also be a part of waker_getters?

I think it should be inlcuded as the counterpart to Waker::as_raw, otherwise it isn't clear how the RawWaker returned by as_raw can be safely used. This is important for the doc examples for these functions.

For tokio, it seems to only be used here. It should be easy to refactor that to use with_raw.

@Amanieu

@Amanieu
Noticing that Waker is repr(transparent), it should be safe to transmute from &RawWaker to &Waker I think?
Then the with_raw you suggested can be replaced by,

impl Waker {
    pub unsafe fn from_raw_ref(raw: &RawWaker) -> &Waker { transmute(raw) }
}

@Thomasdezeeuw

~~I have a pretty simple use case for this: checking if two Wakers are the same.~~

The Future contract requires wake to be called on the Waker from the latest poll, so on every poll a Future that keeps its Waker stored have to replace its stored Waker with one newly cloned from the Context. Waker::as_raw would allow us to first compare the two Wakers, avoiding replacement (and the atomic operations involved) if they are the same.

Didn't realize will_wake does this already! Thanks @Thomasdezeeuw

Because Waker has private fields, the #[repr(transparent)] is an implementation detail and not an API guarantee that users can rely on. It isn't shown in rustdoc for this reason.

The problem with from_raw_ref is that it effectively turns this into a public guarantee: it might be something we can commit to but it would have a higher barrier for acceptance.

@wishawa

To add another use case is storing the waker atomically. You can split the data and vtable into two AtomicU64 and store those atomically separately (with some careful programming) or in a single AtomicU128 (on 64 bit archs at least).

I have a pretty simple use case for this: checking if two Wakers are the same.

The Future contract requires wake to be called on the Waker from the latest poll, so on every poll a Future that keeps its Waker stored have to replace its stored Waker with one newly cloned from the Context. Waker::as_raw would allow us to first compare the two Wakers, avoiding replacement (and the atomic operations involved) if they are the same.

@wishawa wote that task::Waker::will_wake exists for this.

I think it might be natural to think that a Waker could do more than just wake(). Like how a Box<dyn Error> might be a specific kind of error with some fields we would care about.

I can imagine as situation where you might want to check if the current Waker is a FooWaker in order to extract a field from it, eg a TaskID.
(I am exploring an idea at the moment to hack in a go like context.Context typemap into core::task::Context through this mechanism.)

In order to do this, I could verify that waker.raw_waker().vtable() == &FooWakerVTable.

(in a similar respect to Error and source, it would be interesting to have a waker chain, especially with the recent push to structured concurrency with many nested sub-executors, but that is for a separate issue and another time)

As I have already mentioned, this feature is not really usable with alloc::task::Wake.

Could it be possible to add the following implementation?

impl<W: Wake + Send + Sync + 'static> Arc<W> {
    const fn waker_vtable() -> &'static RawWakerVTable { ... }
}

It would allow to match the vtable returned by RawWaker::vtable.

this feature is not really usable with alloc::task::Wake.

Does Waker::from(&arc_wake).as_raw().vtable() not work?

Does Waker::from(&arc_wake).as_raw().vtable() not work?

Yes it works, but it requires to already have an allocated arc_wake, which makes the thing unusable.

The goal is to compare the vtable of a random waker to know if I can cast its data pointer to my own waker implementation. I don't want to rely for this comparison on a lazily allocated waker to get its vtable (especially as this lazy waker may requires some parameters random to be initialized). In short, this is unusable.
That's too bad, because Wake is a convenient implementation shortcut, but I have to rewrite it myself to use this feature.

use nightly waker_getters APIs

If I understand correctly, the FCP has happened and ended with a disposition to merge. Is there some reason why the FCP checkbox in the first comment is still unticked? In terms of the stabilization PR, is that just waiting for someone to do it, or is there some missing prerequisite?

This is just waiting for someone to submit a stabilization PR.

Submitted a stabilization PR: #129919

Move the `data` and `vtable` methods from `RawWaker` to `Waker`

Add `Waker::new` and `LocalWaker::new`

Rollup merge of rust-lang#129919 - kevinmehall:waker-getters, r=dtolnay

Unrolled build for rust-lang#129919

Rollup merge of #129919 - kevinmehall:waker-getters, r=dtolnay

embassy-executor: update to use stabilized waker_getters

Navigation Menu

Repository navigation

Tracking Issue for `waker_getters` #96992

Public API

Steps / History

Unresolved Questions

65 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Tracking Issue for waker_getters #96992

Description

Public API

Steps / History

Unresolved Questions

Activity

oxalica commented on Sep 8, 2022

Amanieu commented on Dec 8, 2022

oxalica commented on Dec 8, 2022

Amanieu commented on Dec 8, 2022

oxalica commented on Dec 8, 2022

Footnotes

Amanieu commented on Dec 8, 2022

oxalica commented on Dec 10, 2022

Amanieu commented on Dec 10, 2022

oxalica commented on Dec 17, 2022

wishawa commented on Dec 28, 2022

Amanieu commented on Dec 29, 2022

Thomasdezeeuw commented on Jan 1, 2023

conradludgate commented on Jan 19, 2023

65 remaining items

wyfo commented on Apr 6, 2024

conradludgate commented on Apr 7, 2024

wyfo commented on Apr 7, 2024

Hawk777 commented on Jun 9, 2024

Amanieu commented on Jul 8, 2024

kevinmehall commented on Sep 2, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Tracking Issue for `waker_getters` #96992