This repository was archived by the owner on Oct 2, 2025. It is now read-only.
pcfs correctness continued #24
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
work on making PCFS booting more stable on a modern console with
Cafe SDK. this commit is part one, but ideally when the entire commit
is merged, we'll have correctness at every level of the PCFS/FSEmul
stack.
Some of the things we've done:
- bumped the version number for when this commit is merged
- update some minor documentation on FSEmul/PCFSServeer docs in README.
- add a flag to 'force unique file descriptors' when using our host
filesystem, which makes debugging much easier as fd's are able to be
reused and can make it harder to trace.
- add in a "sata wal log", and tool to generate it from pcap
- this is a very buggy and debug oriented tool that makes it easy
to figure out what PCFS is doing for a packet capture/server.
- this isn't perfect yet, but the goal is if we keep expanding it
more, and more, and make it a bit less dev only style workflow
it can be a very useful debugging tool.
- this has already actually solved several differences that were
causing buggy behavior even in it's imperfect state.
- create debug tool for generating debug logs/padlogs from
pcaps/pcapng's
- currently this is just a thin wrapper around tshark, but ideally
one day we actually integrate it as rust code.
- create an etched out PCFSServer.exe that just runs a server, and
isn't actually a reimplementation yet.
- this makes it easy to see what my PCFS server actually reponds to
one off requests.
- this has made it easier to debug issues, and is eventually needed
anyway for parity.
- add docsrs tags to all `cfg(feature = ` blocks, so it's easy to
tell on docsrs what things are feature dependent.
- host filesystem now does more checks on the CafeSDK directory.
- it also is capable of placing down more files, even some the
official sdk doesn't place down, but are necessary for preventing
error lines, etc.
- fixes to host filesystem read path (host filesystem now does not do
padding, and it's read is more inline with what is to be expected)
- directory iterators from host filesystems are now pre-generated, and
sorted.
- this is required for PCFSServer to function correctly.
- fix atapi's read file implementation to actually handle large files
and truncate/pad correctly.
- fix edge cases in read file padding
- this is mostly around when padding is not applied when you are
greater than `first_read_size`, a value the client was sending
us before but we were not tracking.
- padding now should match the official sdk 100% of the time.
- add support for `SetFilePosition` for PCFS/SATA
- add support for `Rename` for PCFS/Sata
- add more info to `info_by_query`
- support the special `0x8000_0000` fd info for the disc
emu path, when no disc is inserted
- add timeouts to MION HTTP requests
- add extra notes about TCP Server concurrency, and how exactly
it works
Contributor
Author
|
This PR is not yet in it's "final ready" state, but wanted to at least push it up to get a CI run on it. With this PR at this commit, PCFS booting is consistently getting successfully into COS, but something in ios/starbuck is panicking very early on. The goal is to fix that before merging this in (and hopefully some of the other pieces as well, but we may not be able to get to all that). |
Windows will often complain about removing the symlink to a directory even if the user created the symlink, and has permissions to both directories. as a result, we now won't attempt to delete the symlink, and let the OS clean it up itself in the final remove directory that is being called.
this just barely happened to be pushed out like right as i pushed my change. one small location we didn't have the correct item, so now we do.
after this commit COS is now consistently starting up, without any
iOS failures. COS still panics very early on into it's startup, but
we are getting much much further.
i also did a tiny bit of cleanup on the
`dbg-generate-sata-wal-from-pcap` binary, making a bit more ready for
prime time, although there's still some issues with padlogs, and clippy.
update dependencies.
- allow writing to `%DISC_EMU_DIR/save` which can be written to in
certain OS booting cases, so use that to actually get some logs.
- update prepare_for_serving to be more correct
- tmd's weren't correct so aren't copied, same with empty ccr.xml
- capitilize title ids in the disc emu dir
- keep track of whether or not the FFIO buffer has grown at all
- this helps us byte match exactly with the official PCFSServer which
makes it easier to debug.
- even though the console does eem to be able ot handle both types of
padding.
- support make folder flags within get_info_by_query
- open file respond with `PathNotExistError` in more cases
- before this wasn't always being sent, and this caused errors with
things like the tmd.
- update padding logic for read_file to respond with 0's if buffer has
ever grown
- fix wal generation, so now SATA WALs should be fully complete.
the rename sata command has support for copying a file instead of moving it when specific user0/user1 flags are passed in. this corrects the implementation, which brings us very far into the boot process.
Contributor
Author
|
Just want to say with the latest commit pushed here: 7031a7a16016ee5d1274c2ecd305263d6fc3e3ad, we can now successfully get into the Wii U boot animation on legacy, and modern Cat-Dev's.... consistently. Here is the first boot that I happened to run with it, and we can now consistently get to this point:
|
- first off the removal of DEBUG_OUT
- DEBUG_OUT i originally thought was a way to get streaming output
logs.... based off of some poor logging.
- turns out it's not. I'm guessing given it's pair of ports
DEBUG_CONTROL/DEBUG_OUT, that it's actually used for debugging.
like gdb style debugging.
- anyway rip it out for now, it's not correct.
- this means tail now requires a serial port (and exits if not given
one).
- figure out a way to make that not required (SDIO connection post
launch?)
- case-insensitive entire folder structure for cafe-sdk
- before we only lowercased the title ids, but near the end of
booting we find one background video that was uppercase but is
required to be accessed with an entirely lowercased filename.
- this now means we have to rename every single folder/file inside
of a mounted directory.
- use synchronous versions of anything except file read/write.
- tokio is still doing synchronous file operations under the hood
(no proper io_uring support or things yet :( )
- hostfilesystem is initialized from async -> sync -> async which
causes problems as tokio blocking threads require the runtime
in order to progress. this means renames would cause freezing
after awhile.
- as a result now we entirely use synchronous versions of every
api except file read/file write. may be beneficial to move that
stuff to synchronous file apis too.
Contributor
Author
|
With the latest commit 18898fbd40eba6bb032f7a5540da3568dfe81989, we now are getting all the way to "Menu Window: Init", and COS Shell is active and responding to requests. It also logs things like the DRC Home Menu being pressed. So it's "up and running". PCFS seems to be byte for byte matching now, and seems we get some messages on SDIO that are weird, so I'm guessing it's waiting on a signal from SDIO of some form to let it know to continue launching. Either that or it could just be I need to implement enough of COS Shell to launch a title, and auto-launch a title. Anyway this is Very very close. |
Contributor
Author
|
|
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
work on making PCFS booting more stable on a modern console with Cafe SDK. this commit is part one, but ideally when the entire commit is merged, we'll have correctness at every level of the PCFS/FSEmul stack.
Some of the things we've done:
cfg(feature =blocks, so it's easy to tell on docsrs what things are feature dependent.first_read_size, a value the client was sending us before but we were not tracking.SetFilePositionfor PCFS/SATARenamefor PCFS/Satainfo_by_query0x8000_0000fd info for the disc emu path, when no disc is inserted