Add FUSE_DEV_IOC_CLONE support for multi-threaded reading #421
+289
−4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add clone_fd() method to Channel that clones the /dev/fuse fd using ioctl - Add from_fd() constructor to create Channel from an OwnedFd - Add channel() accessor to Session to get reference to the underlying Channel - Export channel module publicly for external use This enables multi-threaded FUSE request processing by allowing multiple threads to read from cloned fds in parallel. Each cloned fd shares the same FUSE connection but can independently read and process requests.
The ioctl request type is i32 on musl but c_ulong on glibc. Use conditional compilation to handle both cases.
Adds Session::from_fd_initialized() for use with FUSE_DEV_IOC_CLONE multi-reader setups. When using clone_fd() to create additional reader threads, those sessions need to be marked as initialized since the INIT handshake only happens on the primary session. Without this, cloned sessions return EIO for all requests because the FUSE protocol requires initialization before processing requests.
When using FUSE_DEV_IOC_CLONE to clone /dev/fuse file descriptors for multi-reader setups, replies must be written to the ORIGINAL fd, not the cloned fd. Previously, from_fd_initialized would use the cloned fd's ChannelSender for replies, causing EIO errors at high concurrency. Changes: - Add reply_sender: Option<ChannelSender> field to Session - Update from_fd_initialized() to require a ChannelSender parameter - Update run() to use reply_sender when available for Request creation - from_fd() sets reply_sender: None (not needed for single-fd usage)
- Add #[cfg(target_os = "linux")] guard to clone_fd() method - Add target_os = "linux" to ioctl constant cfg attributes - Tighten unsafe blocks with SAFETY comments - Improve documentation with Platform Support and Errors sections - Add intra-doc link from from_fd() to clone_fd()
- Use standard rustdoc sections (# Arguments, # Important) - Add intra-doc links to Channel::clone_fd() and ChannelSender - Document all parameters explicitly - Clarify consequence of using with uninitialized fd
Remove the reply_sender parameter from from_fd_initialized() since each cloned FUSE fd handles its own request/response pairs - the FUSE kernel requires that the fd which reads a request is the same fd that sends the response. This change: - Removes the reply_sender field from Session struct - Simplifies from_fd_initialized() signature - Updates documentation to clarify cloned fd behavior
During unmount, the kernel aborts all pending FUSE requests. When our reply arrives, the kernel returns ENOENT because the request was already cleaned up. This is expected behavior, not an error. libfuse explicitly handles this: https://github.com/libfuse/libfuse/blob/master/lib/fuse_lowlevel.c "ENOENT means the operation was interrupted" Before this change, clean unmounts would log spurious errors like: ERROR reply{unique=X}: fuser::reply: Failed to send FUSE reply: ... Now these are silently ignored, matching libfuse behavior.
When umount() returns EBUSY during Mount::drop(), fall through to fuse_unmount_pure() which uses MNT_DETACH for lazy unmount. This prevents spurious ERROR logs during parallel test execution when the kernel still has transient references to the FUSE filesystem. EBUSY can occur transiently because: - The FUSE protocol is still completing cleanup after FUSE_DESTROY - Kernel inode/dentry caches have brief reference counts - Async I/O machinery is still completing Using MNT_DETACH is safe here because FUSE_DESTROY has already been sent, so no new operations can start.
Resolve conflict in src/lib.rs by keeping both: - pub mod channel (our multi-reader support) - pub mod experimental (upstream async API) Also add doc comment for ChannelSender.
Owner
|
Please add a test for this functionality. Also, I don't really like the idea of making |
Replace manual nul-terminated byte string with Rust 1.77+ C string literal syntax. This fixes clippy::manual_c_str_literals warning.
Per review feedback, keep the channel module as an internal implementation detail and expose clone_fd() directly on Session. Changes: - Make channel module private (remove pub from mod channel) - Add Session::clone_fd() that delegates to Channel::clone_fd() - Remove Channel::from_fd() (no longer needed externally) - Remove Session::channel() (no longer needed) Users now call session.clone_fd() instead of session.channel().clone_fd().
Add integration tests for the clone_fd and from_fd_initialized APIs: - clone_fd_multi_reader: Verifies clone_fd() creates a valid fd that can be used for multi-reader setups - from_fd_initialized_works: Tests that multiple reader threads can process FUSE requests concurrently using cloned fds
Sort std::sync imports alphabetically (Arc before atomic).
cberner
requested changes
Dec 20, 2025
| thread::sleep(Duration::from_millis(100)); | ||
|
|
||
| // Access the mountpoint - this triggers FUSE requests | ||
| let _ = std::fs::metadata(tmpdir.path()); |
Owner
There was a problem hiding this comment.
I'd like to see a few more things covered in this test:
- the test should verify that both the filesystems in
sessionandreader_sessionreceive requests to test that the multithreading support is working as expected - it should check that the result of
metadata()is the expected value
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add support for multi-threaded FUSE request processing using
FUSE_DEV_IOC_CLONEioctl.This enables multiple threads to read FUSE requests in parallel from cloned file descriptors, improving throughput for high-concurrency workloads.
API
Changes
Session::clone_fd()- clones the FUSE fd usingFUSE_DEV_IOC_CLONEioctl (Linux only)Session::from_fd_initialized()- creates a session from a cloned fd that skips INIT handshakeENOENTgracefully when sending replies during unmount (matches libfuse behavior)EBUSYduring unmount with lazy unmount fallbackchannelmodule private per review feedbackTests
Added two integration tests:
clone_fd_multi_reader- verifiesclone_fd()creates a valid fdfrom_fd_initialized_works- tests concurrent readers processing requestsPlatform Support
clone_fd()is only available on Linux (#[cfg(target_os = "linux")]).