License Issue

[etcimon]

Hello,

I'm currently upgrading the spasm framework to be able to develop web applications in webassembly much like you do with React or Angular. I modified your library to be compatible with better C to generate wasm code: https://github.com/etcimon/libwasm/tree/master/fast

I intend on putting more work on libwasm to allow developers to create their apps through it mostly for mobile development, but I noticed the GPL3 license recently. I don't think anyone would want to create an application using a tool that forces them to make it open source. Do you think it could be changed to a more permissive license for this specific case? (removing the SIMD parts)

Thanks!

[mleise]

Hello etcimon,

The choice of GPL-3 was deliberate and before I abandoned the repository, the plan was to provide a shared library build that closed-source software could link to at the cost of losing CTFE functionality. I'm open to you taking over the repository and changing the licensing to your liking though, if that is an option, as I wont continue its development.

[etcimon]

I think that would be nice, if it can be MIT licensed. I think currently the more popular json serialization libraries in D are asdf and in vibe.d, but for high performance big data operations this is definitely the go-to. There's a lot of value in the processor-level optimizations you found that makes this library so great, I think the only hickup is the ~"000" padding that triggers a GC allocation which could be solved by providing the length beforehand so an array resize can be done. Anyways, if you intend on letting me manage the repository I would merge a few of my changes myself and play around with the programming language benchmarks the way I still need to do with the BigInt in Botan. Mainly, it would save me some work in libwasm because my other option is to integrate mir-ion which has a lot of dependencies to adjust to betterC

Thanks

[mleise]

If you rename your fork I can transfer it directly to your account and turn this into an automatic redirect.

[etcimon]

Ok it's renamed

[mleise]

GitHub still complains. Apparently you cannot even have a secondary fork.

etcimon already has a repository in the mleise/fast network

Is it feasible for you to delete your fork from GH and merge it back in when the transfer is complete?

About the GC allocation, there were two major implications. I wanted to be able to return proper immutable strings from the JSON without reallocation in most cases. So when the constructor is called with a const(char)[], a side effect of the potential GC reallocation (only if the capacity was too small) is that it turns the const(char)[] into a unique new string and string returning functions can just return a slice of it from there on. That's what the m_isString flag keeps track of.
On the CPU side, there is also a version of the string scan functionality that takes an explicit length and works without appending zeroes. It was 2-3 times slower on my Haswell CPU though IIRC, so that wasn't an option.

But you can go full @nogc by manually preparing the input char[] with the 16 zeroes padding and setting the simdPrep flag in the constructor to No. Then, whenever you need to read a string, use borrowString() instead. This will either return a slice of the original text or - in case there were escape sequences that had to be converted - a slice into a temporary buffer that will be overwritten next time (so make a copy if you need to keep it around).

[etcimon]

It should be deleted now.

I was thinking of using this to estimate the serialization length for numbers a little faster:

https://github.com/ssvb/speedy-stdio/blob/main/speedy/stdio.d#L241

And this part is a CT conversion table for numbers:

https://github.com/ssvb/speedy-stdio/blob/main/speedy/stdio.d#L272

Perhaps that would speed it up further.

[mleise]

Pretty cool bit hack! I have no idea what's going on, but that's usually expected until you run them step by step. ;-)
I already initiated the repo transfer by the way.

[mleise]

When it comes to huge CTFE loops like the conversion table, I used to prefer code generators, because there was no way to cache the result and avoid the compile time hit. In any case, it will be interesting to see how the lookup for batches of 4 digits compares to the traditional loop that converts one digit at a time.