Async reactor API for WASI exports

[Siddapa]

The goal of this API is to merge the functionality of both the WASIReactor and the WASIWorkerHost by having all WASI exports be wrapped as a Promise so long-running function calls won't halt any functionality in the main context. In addition, the API enforces a specification for how export argument data is handled by the WASIReactorWorkerHost so that the underlying WebWorker can pass arguments to any WASI export with type safety on the following types: Int32, Uint32, string, Int32Array, Uint32Array. It achieves this by tagging all arguments with a distinct type tag as they're stored in the WASI memory buffer (exports.memory.buffer). The exported WASI function will then need to parse the memory buffer for those arguments and repeat the same process for any values it would like to return.

Depicted above is a sample implementation showcasing how to call an export. The parameters for constructing the WASIReactorWorkerHost slightly vary compared to WASIReactor and WASIWorkerHost by enforcing a start index in the memory buffer for which all arguments should begin storing data (the 8) as well as an error handler function for the error code that should be returned by the exported function.

This specification is quite verbose and enforces a lot of work to be done by the WASM binary creator to ensure the binary is properly reading from the memory the way the API is storing in it. However, with well laid out documentation, I still think this approach is quite effective overall and establishes good calling conventions that developers should abide by until the WASM standard is able to support more diverse types.

[Siddapa]

Here's also a link to the Zig memory store/load parser that's used in the compiled WASI binary shown above:
https://github.com/Siddapa/optimal-pkmn-battler/blob/main/battler/src/gen1/wasm/memory.zig

[taybenlor]

Thanks Siddapa!

This is really interesting as a contribution, but I worry that it's a mix of opinion and standard.

In the WASM Component Model there is a way to do the kind of communication you are trying with typed arguments like strings, however in preview 1 that's really left to the implementer. Storing values into memory to initiate a function call feels like something that isn't Runno's responsibility as a WASI runtime, instead I would expect the user to do this kind of action. Particularly since every language implements strings (and potentially other types) differently.

Is there a different way to do this that exposes the memory to the user without having opinions about storage?

I realise this is one of the most awkward parts of working with WebAssembly binaries, and maybe it would be better for you to move more toward the WASI 0.2 style?

[Siddapa]

Well, I felt that enforcing some specification into the memory storage would be of some help to the designer/user of the WebAssembly binary, but I see how it isn't really in the spirit of a runtime. I think the simplest method for memory management that follows the async nature already designed would be to let each of the Promise-wrapped exports take 2 functions as arguments that dictate how memory should be stored/loaded, like the following:

wasmWorker.initialize();

const store = (memoryBuffer: SharedArrayBuffer, args: any[]) => {
    // Do whatever memory management desired
    storeArray(memoryBuffer, args[0]);
    storeString(memoryBuffer, args[1]);
    storeUint32(memoryBuffer, args[2]);
};
const load = (memoryBuffer: SharedArrayBuffer) => {
    // Do whatever memory management desired
    const arr = loadArray(memoryBuffer, index);
    const str = loadString(memoryBuffer, index);
    const int = loadUint32(memoryBuffer, index);
    return [arr, str, int];
};

const returnValues = await wasmWorker.test_memory(/* Whatever other arguments that need to be passed */, store, load);

This would then change the true call order in WASIReactorWorkerHost to store(), export(), then load(). This ensures flexibility in the design of the memory without having the API force any restriction for the user to abide by. In addition, the API user can also generalize any data types or specific schemes they would like the memory layout to follow into their own abstractions.

Also, I took a look at the WASI preview2 specification for the first time as I believed they weren't that far ahead with the project. A lot of the issues with my approach have pretty good solutions from the interface design they're working with so preview2 definitely seems promising. For the time being though, the implementation laid out above should suffice for a preview1 version of the API.