Provide async API

[zeenix]

Since it's an IPC API, it should be async. Moreover, I think async should be the primary/main API, with blocking wrappers on top.

I understand this is going to be a major change/break but given that currently this crate has not many users, I think that shouldn't be a concern. It'll involve bumping the major number of course.

The real problem (apart from this being a big project), is going to be implementing an async runtime-agnostic solution. The bad news is that there is no good solution for this (for now). The good news is that I've dealt with this problem in zbus and we've a not-so-bad solution now: be runtime agnostic by default using smol-rs crates and providing tight integration with tokio (through an optional feature).

It might be possible to reuse some or all of the abstractions we've in zbus for this.

[zeenix]

@haraldh seems you already thought about this some years back and your thinking was to use use tokio. Although that would alienate many folks who don't or can't use tokio but it does certainly makes the task much easier.

[haraldh]

I'll accept all PRs which add a solution to this problem :)

[zeenix]

I'll accept all PRs which add a solution to this problem :)

Cool. It may or may not be me, depending on if I can possibly get some paid time to work on this. :) I just wanted to create the issue with some pointers on possible approaches.

[haraldh]

This looks interesting https://www.firezone.dev/blog/sans-io

[cgwalters]

While my project (bootc) happens to use async pretty heavily, I am not sure about imposing an async requirement for this crate. (On the other hand, uptake of varlink is quite minimal outside of systemd so maybe we do just add a hard async requirement and if someone complains then we add a blocking wrapper on top?)

This looks interesting https://www.firezone.dev/blog/sans-io

Yeah, I think a basic implementation of that here would be an API on Connection where the caller does "read until NUL" and then can feed that whole thing as a &[u8] or so, and one can pop off complete parsed messages from it?

[zeenix]

In case anyone gets the motivation and time to work on this before I get a chance, my rough plan would be to modify the Stream trait to be based on futures::{AsyncRead, AsyncWrite} and using async-io crate for the impls, very similar to how we do in the zbus project (although in zbus we don't use Read/Write traits but zbus' usecase is much more complex as it involves other transports than just unix socket and connection sharing).

For tokio compatibility, we have 2 choices:

1. A tokio feature, that when enabled we implement tokio::io::{AsyncRead, AsyncWrite} only.
2. Make use of async_compat crate.

[zeenix]

I am not sure about imposing an async requirement for this crate

It's a library that does blocking stuff and hence users will expect it to be async. I don't think this would be an impossition for most users but rather a very useful feature, allowing them to integrate varlink easily into their code.

[zeenix]

we add a blocking wrapper on top

Oh, I missed this part. That's always possible, relatively easy and IMO should be done anyway. There is a reason why zbus::blocking mod exists. :)

[cooperlees]

+1 on wanting a tokio async API - Would be looking to use it for monitord - That gets systemd stats today via DBus - https://github.com/cooperlees/monitord

• With the goal to eventually remove zbus if enough varlink APIs are presented from systemd
  • I expect to support both for quite some time tho

I've been holding off adding the support waiting to see what an async API here would look like. But I think I might start and wrap all the blocking IO in spawn_blocking threads for now.

[cryptomilk]

Hi,

I'm Andreas and I'm working on a library to provide a simple to use API for systemd's userdb. It is called kirmes. I already have the Rust part prototyped and I'm currently writing the C API, but it would also be nice to provide an async API in future.

[zeenix]

Anyone subscribed here, I just rolled out zlink 0.1.