Skip to content

Home

Jump to bottom
Ravin Perera edited this page Apr 30, 2020 · 4 revisions

hpfs is a FUSE-based userspace virtual filesystem intended for Hot Pocket state management. This is a checkpointing filesystem where you start with a seed (initial) state and subsequent writes will always be logged under separately tracked checkpoints without overriding seed state. Reads can be performed on any checkpoint giving the illusion of a coherent filesystem containing all the changes done up to the checkpoint. hpfs also offers the ability to merge-forward the seed state with logged writes thereby "collapsing" the logged writes onto the seed state.

Supported operations

  • ReadWrite session start - This will create a session where write operations performed will not overwrite existing last-known state of the filesystem. Instead any new writes would be tracked/stored separately. Any reads performed within the RW session will see a consistent filesystem including the writes performed within the session.

  • ReadWrite session complete - This will end the RW session by creating checkpoint which includes all the modifications that was captured during the session.

  • ReadOnly session start - ReadOnly session can be initiated on any checkpoint. Any reads performed within the RO session will see a consistent filesystem which includes filesystem state ONLY UPTO the checkpoint specified during session start. RO session will not see any changes made by any ongoing RW session.

  • ReadOnly session complete - This will end the RO session thereby freeing the checkpoint from this RO session. Merge operation (see below) can only be performed up to the oldest checkpoint that is serving a RO session.

  • Merge forward - This will merge the seed state with the oldest checkpoint if there are no ongoing RO sessions associated with that checkpoint. This can be performed repeatedly until all the checkpoints are exhausted which would make the seed state contain the latest physical state of all files and folders.

Guarantees/restrictions

  • Seed state can always be READ without going through hpfs. It's a regular set of files and folders solely managed by the physical filesystem. (Modifying the seed state manually may lead to corruption of hpfs virtual filesystem)
  • Simultaneous ReadOnly sessions can happen on multiple checkpoints.
  • Simultaneous ReadOnly sessions can happen on same checkpoint.
  • Only a single ReadWrite session can happen at any given time.
  • ReadWrite session will always be based off of the latest state of the filesystem.
  • Merge operation can only be peformed up to the point of oldest checkpoint that is serving a ReadOnly session.

Example flow

Virtual filesystem always starts with a seed state.

seed

Perform ReadWrite session start

seed -> rwsession

Perform ReadWrite session complete

seed -> checkpoint1

Perform ReadWrite session start

seed -> checkpoint1 -> rwsession

Perform ReadOnly session start on checkpoint1

seed -> checkpoint1 -> rwsession
             |
        (rosession1)

Perform ReadWrite session complete

seed -> checkpoint1 -> checkpoint2
             |
        (rosession1)

Perform ReadWrite session start

seed -> checkpoint1 -> checkpoint2 -> rwsession
             |
        (rosession1)

Perform ReadOnly session complete on rosession1

seed -> checkpoint1 -> checkpoint2 -> rwsession

Perform Merge forward up to checkpoint1

seed -> checkpoint2 -> rwsession
  |
(now includes checkpoint1)

Perform ReadWrite session complete

seed -> checkpoint2 -> checkpoint3

General usage scenario

seed -> checkpoint1 -> checkpoint2 -> checkpoint3 -> checkpoint4 -> rwsession
             |                                            |
  (rosession1,rosession2)                            (rosession3)

1. Architecture overview

2. Log file

Clone this wiki locally