[Vipitis]

docs preview

---

Potentially remaining tasks

• implement physical size for canvas context to make both imgui examples work again
• understand and fix the problem in creating a timestamp query set
• implement jswriter.py as a Patcher and add the comment injector
• how do we package the wheel? (strip out unused code, pure python - but only in the browser?)
• try a graphic example without rendercanvas
• docs gallery (copy from rendercanvas)
• can we avoid copying buffers
• run tests on CI?
• cleanup all the commented out debug prints
• Warn users about glsl not being supported (or try wasm compiled naga?)
• texture.create_view() is might be the wrong dimension : local pyodide preview fastplotlib/fastplotlib#1052 (comment)

---

new weekend, new project...

I think there is two options to get wgpu-py into the browser: compile wgpu-native for wasm and package that, or call the js backend directly. I run into compilation errors with the rust code, so gave up there... but:

basically autocompleted my way through errors to see what kind of patterns are needed... everything around moving data requires more effort. While pyodide provides some functions, they feel buggy and unpredictable.
It is likely possible to codegen the vast majority of this and then fix up all the api diff - might get to that over the next few days.
structs have potential to make this easier.

I changed some of the examples to auto layout since I couldn't get .create_bindgroup_layout() to work - and you don't need it with auto layout.

works with pygfx/rendercanvas#115
couldn't get the cube example to work just yet, but triangle does - so the potential is there

more to come

[Vipitis]

there are were many headaches around the type conversion which aren't well documented... but I got to the cube in the end.
Haven't looked at any code gen approach as there is quite a few specialties like when it's okay to use keywords when calling the js function. For exmaple:

self._internal.getMappedRange(offset=js_offset, size=data.nbytes)

vs

self._internal.getMappedRange(0, size)

And the error you get is about not of type unsigned long long because these function parameters are GPUSize64 which lead me down a rabbit hole of using BigInt - and now I am not sure if that is required anymore.
Or when your dict contains the key "type" it accesses dict on the js side, not the value...
For anyone else giving this a try or making their branch from here - the comments will be all over the place and likely contradict themselves.

browser_cube.mp4

I will hopefully find some more time this coming week to continue and maybe get some more interesting examples to run (pygfx?/fastplotlib?).

[Vipitis]

super exciting to get imgui working with a few tweaks

imgui_example.mp4

cc @pthom thanks a lot for your article I read a few weeks ago, that motivated me to give it a try here!

[pthom]

@Vipitis : many thanks for the info, that looks very promising. Please keep me in the loop!

[almarklein]

I was expecting codegen to come a long way here. The codegen knows when the arguments of a function were actually wrapped in a dict in IDL, so we can generate the code to reconstruct the dict before passing it to the JS WebGPU API call.

[Vipitis]

I also think that codegen can do a lot, I just need to give it a try. The to_js method has a few more arguments to make use of, for example dict_converter which sounds like solves some problems. The custom accessor currently has two functions: access the ._internal object and replace dict/struct keys with camelCase. However it overwrites the default dict conversion.
It can likely also do the data conversion and more. So the whole API might look like the following which should be trivial to codegen.

def some_function(self, *args, **kwargs):
    js_args = to_js(args, eager_converter=js_acccessor)
    js_kwrags = to_js(kwargs, eager_converter=js_accessor, dict_converter=Object.from_entries)
    self._internal.someFunction(*js_args, js_kwargs)

[almarklein]

Whatever way this goes, what I care most about, is that when the IDL changes for a certain method, it will place some FIXME comment in the code for the JS backend, so that we won't forget to update that method there.

[Vipitis]

I feel like I have finally moved passt all the headaches and found a "general" approach to most functions. I switched to the pyodide dev branch as the upcoming 0.29 release makes changes to how dictionaries are converted... which has been a ton of pain and the upcoming version seems to work much better. I couldn't find any release timeline so it might still be month until there is a release...
Structs were the key to get all the default values while renaming keys to camel case.

Also got started with a codegen prototype and I am feeling confident this is largely going to work, depends on how much time I find in the coming week.

There was also some weirdness with css scaled canvas for click events and resizing with the imgui example - so the rendercanvas PR likely needs some more fixes, I will see if I can find time for that too.

[almarklein]

@Vipitis are the stack switching problems you mentioned related to the async methods? Are there any other bottlenecks left?

[almarklein]

@Alex-PLACET from what I can see pyjs has nothing to do with pyodide, right?

I suspect it would need its own backend, or maybe it can be made to Just Work with the compiled backend. Do you know whether pyjs compiles to Emscriptem or Wasi?

[Alex-PLACET]

@almarklein pyjs is used in emscripten-forge packages, not pyodide.
So yes we use emscripten.
And yes it deserves its own backend.
I have to move forward on the subject but I don't have so much time for the moment. I wil try to check it again in 3 or 4 weeks

[Vipitis]

@Vipitis are the stack switching problems you mentioned related to the async methods? Are there any other bottlenecks left?

Yes stack switching enables the pyodide run_snyc which implements our sync_wait (maybe it should be _sync_wait instead). It should also work on Firefox now as they support JSPI.

• If you try the unmodified example from pygfx https://pygfx--1273.org.readthedocs.build/1273/_gallery/validation/validate_view_offset.html#interactive-example
  It broke when rendercanvas offscreencanvas got the async downloader. My current hypothesis is that this happens because it's doing an wait while inside a wait itself. Pyodide fixed a lot of the aysnc detection we had workarounds for with patch releases of 314.0.x and there is PRs like Do not overwrite current running loop when new event loop is created pyodide/pyodide#6342 which I haven't tried yet. My intention was to minimize this into maybe just a rendercanvas example to understand what's happening.
  (The streamed blending example also breaks in a similar fashion but that's due to the imageio patch).

• If you try the current FPL release branch here a lot of things break due to map_sync which is an API diff. I need to look into that and maybe implement it backend specific

• Finally the current codegen ergonomics are awful. It needs a refactor so it's a single file. Where custom implementation can be made and the remaining methods are generated. So they likely need to be marked. I also considered doing it via ast instead of string manipulation. But that's really not needed.

There will be more issues, especially along the spec/native divide and doing zero copies for performance etc. But I'd say the majority of examples already work with no changes to usercode.

[almarklein]

@Vipitis @Alex-PLACET it looks like it would be possible to support the browser through wgpu! This would require a new version of Emscripten released just last week. I have a POC that suggest that it works, but to tell for sure, we'd need to wait for Pyodide to adopt that new Emscripten version.

It would also require a change in how the binding works; we'd need an extension module instead of a dll that we cfii into, but we want to go there anyway for other reasons (performance being one).

[Vipitis]

that will be interesting to see. But is this wgpu directly or wgpu-native?

My understanding was that wgpu already compiles to wasm (maybe compatible with pyodide on the newer releases), but wgpu-native currently doesn't. (perhaps that is also possible soon?). If it's the former, won't we need the whole mapping layer to wgpu?

[Alex-PLACET]

@almarklein can you tell us more, which version of emscripten is needed ? On Emscripten-forge we are currently moving to 6.0.8

[almarklein]

But is this wgpu directly or wgpu-native?

Either, really, as long as its compiled as an extension module. Though I am leaning towards binding to wgpu directly via pyo3.

can you tell us more, which version of emscripten is needed ? On Emscripten-forge we are currently moving to 6.0.8

Good to know you're moving to 6.0.8! Current research status:

• Emscripten v6.0.1 first introduced support for wasm-bindgen.
• v6.0.5 is what the latest Pyodide (py315) uses. I tried this just now, but does not work.
• v6.0.8 is what E-forge will be using will have a look at this next. also tried, and does not work.
• v6.0.10 works, but neither Pyodide nor PyJS uses it yet (understandably, since its just a week old).

I'll post more details later, when I submit my PR to wgpu.

[Alex-PLACET]

@almarklein Ok, don't hesitate to ping me on your pullrequest. I will check on my side if we can move to 6.0.10 soon.
This may leads to really cool projects and notebooks in our browsers!

[almarklein]

See gfx-rs/wgpu#10515

[almarklein]

I will check on my side if we can move to 6.0.10 soon.

That could be nice to be able to test our new backend. Though I imagine much more is needed to run wgpu-py in PyJS, being fully typed and all that?

This may leads to really cool projects and notebooks in our browsers!

Just to be clear, we still have to develop the new wgpu-pyo3 backend first :)

I think it's worth continuing with the current PR, since it's already close, and it means we can compare the two approaches and select the "best" one eventually. For PyJS it's probably easier to wait for the new backend.

[almarklein]

I suppose this is done temporary during dev. Would be good to have a proper way for this 🤔

[Vipitis]

I have to check the commits, but something was complaining when building the wheel. IIRC it might have been the codegen report being different because I skipp the C codegen (which takes a bit) when launching pyodide examples

[almarklein]

Idea: rather than try to codegen as much as we can, only codegen the public API. The rest is manual code (we can simply use the currently generated api). Then, we use mypy to validate that we use the WebGPU API correctly. How? We generate the JS WebGPU API, but in typed Python. So we get a webgpu.pyi, that we use for validation.

Maybe use this for the new wgpu backend also: less/simpler codegen, rely on validation by using types.

[Vipitis]

That sounds like a good approach and maybe what I was going for in a more convoluted way (using the struts for descriptors to handle default values).
I didn't figure out how to keep the API diff in a separate file due to the circular imports with constructors. So there might be some meta level to solve of all it. I have also see an approach (referenced above) where all methods are abstracted by capturing ._call with a getattr on the jsproxy. But I am not sure this would work.

I am currently hunting down a race condition where a buffer isn't unmapped before submission and I am not sure if this is a pyodide bug (hoping for 315 to just fix all of it magically). It's happening during FPL examples that use imgui when you right click. And the pygfy get pixel info call is happening multiple times (maybe during an event callback?). pygfx ensures the unmap with try...finally but imgui dispatches the render with try...finally too. So unclear what's happening inside pyodide. I should simplify it to get an A/B example, which is my hope for this weekend.

[almarklein]

What browser do you use to test it on? If Chrome, you're essentially testing against Dawn, which may behave slightly differently than wgpu ... could be worth checking race conditions like that on both FF and Chrome.

[Vipitis]

I think there finally is a solution to the run_sync problems: pygfx/rendercanvas#258

Didn't get to test all examples in pygfy or fpl yet, but at least the right click menus now work... so maybe this is it? Perhaps it would help to run test suites with pyodide to see where remaining problems are.