From 4faa1b0c560b7bd33c2fc4215849c9580b6bfb26 Mon Sep 17 00:00:00 2001 From: Ada Christiansen <60967674+luccaflower@users.noreply.github.com> Date: Fri, 16 Jan 2026 12:09:29 +0100 Subject: [PATCH 1/8] Introduce UW-Madison GitLab and replace filesystem project --- docs/computer-science/systems/ostep/index.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/docs/computer-science/systems/ostep/index.md b/docs/computer-science/systems/ostep/index.md index 97ca600..5cf9f0c 100644 --- a/docs/computer-science/systems/ostep/index.md +++ b/docs/computer-science/systems/ostep/index.md @@ -92,7 +92,7 @@ This course was originally taught as CS 537 at the University of Wisconsin by th | Filesystems | [[39]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-intro.pdf) [[40]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-implementation.pdf) | [[11.4]](https://youtu.be/EDFoFlzZ8_w) [[12.1]](https://youtu.be/QMjJlCqUYW4) [[12.2]](https://youtu.be/87vv7nVdTDA) [[12.3]](https://youtu.be/5n0AdNuBObU) | | | Crash consistency, journaling, FFS | [[41]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-ffs.pdf) [[42]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-journaling.pdf) | [[13.1]](https://youtu.be/piwPJ0sLV0Y) [[13.2]](https://youtu.be/MgnQV-ss1wc) [[13.3]](https://youtu.be/wwvMNItRyl8) | | | LFS and SSDs | [[43]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-lfs.pdf) [[44]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf) | [[14.1]](https://youtu.be/59XSFnXQ-9Q) [[14.2]](https://youtu.be/6fbm9u7__L0) [[14.3]](https://youtu.be/vvttbstRdj8) [[14.4]](https://youtu.be/sKTyhqvTUBU) | | -| Data Integrity and Protection | [[45]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-integrity.pdf) | | File System Checker: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/filesystems-checker) | +| Data Integrity and Protection | [[45]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-integrity.pdf) | | Mini-WFS: [details](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p6/p6-base), [hints](#mini-wfs) | | *Summary* | [[46]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-dialogue.pdf) | | | | **End of class review** | | [Part 1](https://www.youtube.com/watch?v=TSiMDPFquO4), [Part 2](https://youtu.be/jIcw7B36oBU) | | @@ -128,9 +128,9 @@ On macOS, you'll need to install a cross-compiler `gcc` suite capable of produci On Windows, you can use a Linux virtual machine for the homework and projects. Some of these packages are not yet supported on Apple M1 computers, and virtual machine software has not yet been ported to the new processor architecture; some students have used a VPS to do the homework and projects instead. -In our experience, modern Linux systems may run into compatibility issues when trying to build and/or run the xv6 kernel. Ubuntu 18.04 has shown to be a known-good version of linux for both building, running, and debugging the last version of xv6. You can run this version of Ubuntu in either a virtual machine or a Docker image. [This blog post](https://0x1eaf.dev/blog/run-xv6-ostep/) contains instructions on how to run and debug xv6 in an Ubuntu Docker image. +In our experience, modern Linux systems may run into compatibility issues when trying to build and/or run the xv6 kernel. Ubuntu 18.04 has shown to be a known-good version of linux for both building, running, and debugging the last version of xv6. You can run this version of Ubuntu in either a virtual machine or a Docker image. [This blog post](https://0x1eaf.dev/blog/run-xv6-ostep/) contains instructions on how to run and debug xv6 in an Ubuntu Docker image. UW-Madison has also made a Docker image available [here](https://git.doit.wisc.edu/cdis/cs/courses/cs537/useful-resources/cs537-docker). -Next, clone the `ostep-homework` and `ostep-projects` repositories: +For the homeworks and most projects, we use the GitHub repositories authored by Remzi. You can get them by cloning the `ostep-homework` and `ostep-projects` repositories: ```sh git clone https://github.com/remzi-arpacidusseau/ostep-homework/ git clone https://github.com/remzi-arpacidusseau/ostep-projects/ @@ -142,6 +142,9 @@ You'll have to clone [the `xv6-public` repository](https://github.com/mit-pdos/x mkdir src git clone https://github.com/mit-pdos/xv6-public src ``` + +In some cases, we have found that the projects are either lacking in tests or in compatibility with the publicly available revisions of xv6. In those cases, we have chosen to replace those projects with projects from the [UW Madison GitLab](https://git.doit.wisc.edu/cdis/cs/courses/cs537/). Any projects from here will have a dedicated git repository for you to clone, which includes any material you may need such as an appropriate version of the xv6-kernel. For these projects, make your changes in the `solution`-directory of the repository, and run `tests/run-tests.sh` from the root of the repository in order to check your work. + ### Hints and tips for Projects #### Debugging @@ -191,6 +194,9 @@ The thread implementation itself should be straightforward to verify as it can b Instead, we recommend having a critical section of code that invokes multiple system calls. This guarantees interrupts, increasing the likelyhood of race conditions causing problems. A good candidate is printf. The xv6 implementation of printf invokes a write-syscall of a single byte for each printed character. Thus, if you use printf concurrently across multiple threads, there is a high chance that the terminal output gets garbled, unless you hold a working mutex lock for the duration of the printf-call. +#### Mini-WFS +TBW + #### Miscellaneous You'll need a general sense of how Makefiles work in order to use the Makefile for xv6. [This tutorial](https://makefiletutorial.com) covers much more than you need; just read the "Getting Started" and "Targets" sections and come back to the rest later if you need to look something up (but you shouldn't have to). From 85f0a6f9913925bb2946d210b38a274e98791bb6 Mon Sep 17 00:00:00 2001 From: Ada Christiansen <60967674+luccaflower@users.noreply.github.com> Date: Sat, 17 Jan 2026 21:20:10 +0100 Subject: [PATCH 2/8] Introduce trial projects Remove legacy projects from the project table. Move project details to project directory section. --- docs/computer-science/systems/ostep/index.md | 74 +++++++++++++++++--- 1 file changed, 64 insertions(+), 10 deletions(-) diff --git a/docs/computer-science/systems/ostep/index.md b/docs/computer-science/systems/ostep/index.md index 5cf9f0c..4a846ca 100644 --- a/docs/computer-science/systems/ostep/index.md +++ b/docs/computer-science/systems/ostep/index.md @@ -44,7 +44,7 @@ If you are getting stuck on these, please don't spend too much time on them. The ## Extended Approach -If you've chosen this option, it is a *must* that you complete *both* courses in [Intro to Computer Systems](../computer-systems/index.md) or equivalent first. +If you've chosen this ophttps://www.reddit.com/r/Twitch/comments/11oy4y2/extremely_low_quality_streams_0_quality_on_all/tion, it is a *must* that you complete *both* courses in [Intro to Computer Systems](../computer-systems/index.md) or equivalent first. That said, if you're able to commit the time required for the prerequisites, we believe the reward is well worth the effort: this course is exciting, interesting, and quite useful for other fields of computer science and programming. One big attraction of this course is the opportunity to see a simplified but fully-functional Unix-like operating system in action and understand the concepts and design decisions that went into it as well as the low-level implementation details. @@ -67,22 +67,22 @@ This course was originally taught as CS 537 at the University of Wisconsin by th | Topic | Readings | Lectures | Projects | | ------- | -------- | -------- | ---------| -| Intro | [[pre]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/preface.pdf) [[1]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/dialogue-threeeasy.pdf) [[2]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/intro.pdf) | [[1.1]](https://www.youtube.com/watch?v=3uMbb9dLtlE) [[1.2]](https://www.youtube.com/watch?v=K4qbAiC77Yo) | Unix Utilities: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/initial-utilities), [discussion](https://youtu.be/rgcq9x8LtGQ) | +| Intro | [[pre]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/preface.pdf) [[1]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/dialogue-threeeasy.pdf) [[2]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/intro.pdf) | [[1.1]](https://www.youtube.com/watch?v=3uMbb9dLtlE) [[1.2]](https://www.youtube.com/watch?v=K4qbAiC77Yo) | [Unix Utilities](#unix-utilities) | | **Virtualization** | | | | | *Dialogue* | [[3]](https://pages.cs.wisc.edu/~remzi/OSTEP/dialogue-virtualization.pdf) | | | -| Processes | [[4]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-intro.pdf) [[5]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-api.pdf) [[6]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-mechanisms.pdf) | [[1.3]](https://youtu.be/LVxN7ZkGh3w) [[2.1]](https://youtu.be/oTd72Yp2m8w) | Unix Shell: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/processes-shell), [discussion](https://youtu.be/76PfvXTwF04), [hints](Project-2A-processes-shell) | -| Scheduling | [[7]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched.pdf) [[8]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-mlfq.pdf) [[9]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-lottery.pdf) [[10]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-multi.pdf) | [[2.2]](https://youtu.be/Q09UgVfragU) [[2.3]](https://youtu.be/fin5-82L-r8) [[3.1]](https://youtu.be/cAiwISFta4g) | xv6 Intro: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/initial-xv6), [discussion](https://www.youtube.com/watch?v=vR6z2QGcoo8), [hints](Project-1B-initial-xv6)
xv6 Lottery Scheduler: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/scheduling-xv6-lottery), [discussion](https://www.youtube.com/watch?v=eYfeOT1QYmg), [hints](Scheduling-xv6-lottery) | +| Processes | [[4]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-intro.pdf) [[5]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-api.pdf) [[6]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-mechanisms.pdf) | [[1.3]](https://youtu.be/LVxN7ZkGh3w) [[2.1]](https://youtu.be/oTd72Yp2m8w) | [Unix Shell](#unix-shell) | +| Scheduling | [[7]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched.pdf) [[8]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-mlfq.pdf) [[9]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-lottery.pdf) [[10]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-multi.pdf) | [[2.2]](https://youtu.be/Q09UgVfragU) [[2.3]](https://youtu.be/fin5-82L-r8) [[3.1]](https://youtu.be/cAiwISFta4g) | [xv6 Intro](#xv6-intro)
[STCF-scheduler](#stcf-scheduler) | | *Summary* | [[11]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-dialogue.pdf) | | | | *Dialogue* | [[12]](https://pages.cs.wisc.edu/~remzi/OSTEP/dialogue-vm.pdf) | | | | Virtual Memory | [[13]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-intro.pdf) [[14]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-api.pdf) [[15]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-mechanism.pdf) [[16]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-segmentation.pdf) [[17]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-freespace.pdf) | [[3.2]](https://youtu.be/I0RIlSN0DzM) [[3.3]](https://youtu.be/0WVoWlOT-kY) | | | Paging | [[18]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-paging.pdf) [[19]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-tlbs.pdf) [[20]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-smalltables.pdf) | [[4.1]](https://youtu.be/wAx_h3HkIX0) [[4.2]](https://youtu.be/7BOXM2XgGO4) [[4.3]](https://youtu.be/LprKOBsALGA) [[5.1]](https://www.youtube.com/watch?v=ggPkFxOTwHY) | | -| Beyond physical | [[21]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys.pdf) [[22]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys-policy.pdf) [[23]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-complete.pdf) | [[5.2]](https://www.youtube.com/watch?v=4tPXkN5nRQs) | xv6 Virtual Memory: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/vm-xv6-intro), [discussion 1](https://youtu.be/z6dqk6iBBRY?t=1305), [discussion 2](https://www.youtube.com/watch?v=WVHRaqom0y), [hints](vm-xv6-intro) | +| Beyond physical | [[21]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys.pdf) [[22]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys-policy.pdf) [[23]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-complete.pdf) | [[5.2]](https://www.youtube.com/watch?v=4tPXkN5nRQs) | [Virtual Memory](#virtual-memory) | | *Summary* | [[24]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-dialogue.pdf) | | | | **Concurrency** | | | | | *Dialogue* | [[25]](https://pages.cs.wisc.edu/~remzi/OSTEP/dialogue-concurrency.pdf) | | | | | Threads |[[26]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-intro.pdf) [[27]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-api.pdf) | [[6.1]](https://www.youtube.com/watch?v=4PghlMdp9cU) | | | Concurrency primitives | [[28]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-locks.pdf) [[29]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-locks-usage.pdf) [[30]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf) [[31]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf) | [[6.2]](https://www.youtube.com/watch?v=hivv8F-LjzY) [[6.3]](https://youtu.be/BoLYvNp2Lc4) [[7.1]](https://www.youtube.com/watch?v=G95w4ghn42A) [[7.2]](https://www.youtube.com/watch?v=X5clCyJ4uuk) [[7.3]](https://youtu.be/RN8A9EvKBdY) [[8.1]](https://youtu.be/U1LfmL7f1h8) [[8.2]](https://youtu.be/cuY8r8RXqAY) | | -| More topics in concurrency | [[32]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-bugs.pdf) [[33]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-events.pdf) | [[9.1]](https://youtu.be/Fnp_K63ss44) [[9.2]](https://youtu.be/AMG29dlH8t0) | Parallel Zip: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-pzip), [discussion 1](https://www.youtube.com/watch?v=z6dqk6iBBRY), [discussion 2](https://www.youtube.com/watch?v=WVHRaqom0yo)
MapReduce: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-mapreduce), [discussion](https://youtu.be/tSiJ_oBSOZE?t=34), [Q/A](https://www.youtube.com/watch?v=jVmWrr8y0Uw)
Kernel Threads: [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-xv6-threads), [discussion](https://www.youtube.com/watch?v=G9nW9UbkT7s), [hints](#xv6-kernel-threads) | +| More topics in concurrency | [[32]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-bugs.pdf) [[33]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-events.pdf) | [[9.1]](https://youtu.be/Fnp_K63ss44) [[9.2]](https://youtu.be/AMG29dlH8t0) | [MapReduce](#mapreduce) | | *Summary* | [[34]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-dialogue.pdf) | | | | **Midterm review** | | [Review](https://www.youtube.com/watch?v=doHSi2ffu9I) | | | **Persistence** | @@ -92,7 +92,7 @@ This course was originally taught as CS 537 at the University of Wisconsin by th | Filesystems | [[39]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-intro.pdf) [[40]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-implementation.pdf) | [[11.4]](https://youtu.be/EDFoFlzZ8_w) [[12.1]](https://youtu.be/QMjJlCqUYW4) [[12.2]](https://youtu.be/87vv7nVdTDA) [[12.3]](https://youtu.be/5n0AdNuBObU) | | | Crash consistency, journaling, FFS | [[41]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-ffs.pdf) [[42]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-journaling.pdf) | [[13.1]](https://youtu.be/piwPJ0sLV0Y) [[13.2]](https://youtu.be/MgnQV-ss1wc) [[13.3]](https://youtu.be/wwvMNItRyl8) | | | LFS and SSDs | [[43]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-lfs.pdf) [[44]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf) | [[14.1]](https://youtu.be/59XSFnXQ-9Q) [[14.2]](https://youtu.be/6fbm9u7__L0) [[14.3]](https://youtu.be/vvttbstRdj8) [[14.4]](https://youtu.be/sKTyhqvTUBU) | | -| Data Integrity and Protection | [[45]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-integrity.pdf) | | Mini-WFS: [details](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p6/p6-base), [hints](#mini-wfs) | +| Data Integrity and Protection | [[45]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-integrity.pdf) | | [Mini-WFS](#mini-wfs) | | *Summary* | [[46]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-dialogue.pdf) | | | | **End of class review** | | [Part 1](https://www.youtube.com/watch?v=TSiMDPFquO4), [Part 2](https://youtu.be/jIcw7B36oBU) | | @@ -194,9 +194,6 @@ The thread implementation itself should be straightforward to verify as it can b Instead, we recommend having a critical section of code that invokes multiple system calls. This guarantees interrupts, increasing the likelyhood of race conditions causing problems. A good candidate is printf. The xv6 implementation of printf invokes a write-syscall of a single byte for each printed character. Thus, if you use printf concurrently across multiple threads, there is a high chance that the terminal output gets garbled, unless you hold a working mutex lock for the duration of the printf-call. -#### Mini-WFS -TBW - #### Miscellaneous You'll need a general sense of how Makefiles work in order to use the Makefile for xv6. [This tutorial](https://makefiletutorial.com) covers much more than you need; just read the "Getting Started" and "Targets" sections and come back to the rest later if you need to look something up (but you shouldn't have to). @@ -206,3 +203,60 @@ Additional (optional) resources include: * [Linker Scripts](https://sourceware.org/binutils/docs/ld/Scripts.html#Scripts): a guide to writing scripts for the GNU linker `ld`. * [OSDev Wiki](https://wiki.osdev.org): a great resource for all kinds of OS concepts and implementation details. * [*Linux Kernel Development*](https://www.amazon.com/dp/0672329468): if you want to apply your xv6 knowledge toward contributing to the Linux kernel, this is a great read after OSTEP. + +### STCF-scheduler +**Trial project**: This project is currently being tried as a replacement for the xv6 lottery scheduler project. If you wish to do the old project instead, click [here](#xv6-lottery-scheduler). +- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p3/p3-base) + +The discussion video and hints for the lottery scheduler project will likely be helpful for this project too. + +## Project directory + +### Unix Utilities +- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/initial-utilities) +- [discussion](https://youtu.be/rgcq9x8LtGQ) + +### Unix Shell +- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/processes-shell) +- [discussion](https://youtu.be/76PfvXTwF04) +- [hints](Project-2A-processes-shell) + +### xv6 intro +- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/initial-xv6) +- [discussion](https://www.youtube.com/watch?v=vR6z2QGcoo8) +- [hints](Project-1B-initial-xv6) + +### Virtual Memory +**Trial project**: This project is currently being tried as a replacement for the xv6 virtual memory project. If you wish to do the old project instead, click [here](#xv6-virtual-memory). +- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall24/public/p5) + +### MapReduce +- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-mapreduce) +- [discussion](https://youtu.be/tSiJ_oBSOZE?t=34) +- [Q/A](https://www.youtube.com/watch?v=jVmWrr8y0Uw) + +### Mini-WFS +- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p6/p6-base) + +In this project you will implement a basic filesystem in userspace (FUSE) not unlike the Very Simple Filesystem described in the book. +#### Hints +TBW + +### xv6 lottery scheduler +**Legacy project:** This project will be removed sometime in the future. +- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/scheduling-xv6-lottery) +- [discussion](https://www.youtube.com/watch?v=eYfeOT1QYmg) +- [hints](Scheduling-xv6-lottery) + +### xv6 virtual memory +**Legacy project:** This project will be removed sometime in the future. +- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/vm-xv6-intro) +- [discussion 1](https://youtu.be/z6dqk6iBBRY?t=1305) +- [discussion 2](https://www.youtube.com/watch?v=WVHRaqom0y) +- [hints](vm-xv6-intro) + +### Kernel Threads +**Legacy project** +- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-xv6-threads) +- [discussion](https://www.youtube.com/watch?v=G9nW9UbkT7s) +- [hints](#xv6-kernel-threads) From 754d834c1e8d21c4162e35626e6b9e410503c54a Mon Sep 17 00:00:00 2001 From: ada Date: Mon, 19 Jan 2026 18:05:50 +0100 Subject: [PATCH 3/8] Massive rework of projects --- ...oject-1B-initial-xv6.md => Initial-xv6.md} | 5 +- .../systems/ostep/Kernel-Threads-xv6.md | 29 ++++++ ...v6-lottery.md => Lottery-Scheduler-xv6.md} | 11 ++- .../systems/ostep/MapReduce.md | 11 +++ .../systems/ostep/Memory-Mapping-xv6.md | 14 +++ .../systems/ostep/Mini-WFS.md | 26 +++++ .../systems/ostep/STCF-Scheduler.md | 14 +++ ...ct-2A-processes-shell.md => Unix-Shell.md} | 11 +-- .../systems/ostep/Unix-Utilities.md | 13 +++ ...6-intro.md => Virtual-Memory-Intro-xv6.md} | 9 +- docs/computer-science/systems/ostep/index.md | 94 ++----------------- 11 files changed, 135 insertions(+), 102 deletions(-) rename docs/computer-science/systems/ostep/{Project-1B-initial-xv6.md => Initial-xv6.md} (95%) create mode 100644 docs/computer-science/systems/ostep/Kernel-Threads-xv6.md rename docs/computer-science/systems/ostep/{Scheduling-xv6-lottery.md => Lottery-Scheduler-xv6.md} (97%) create mode 100644 docs/computer-science/systems/ostep/MapReduce.md create mode 100644 docs/computer-science/systems/ostep/Memory-Mapping-xv6.md create mode 100644 docs/computer-science/systems/ostep/Mini-WFS.md create mode 100644 docs/computer-science/systems/ostep/STCF-Scheduler.md rename docs/computer-science/systems/ostep/{Project-2A-processes-shell.md => Unix-Shell.md} (96%) create mode 100644 docs/computer-science/systems/ostep/Unix-Utilities.md rename docs/computer-science/systems/ostep/{vm-xv6-intro.md => Virtual-Memory-Intro-xv6.md} (74%) diff --git a/docs/computer-science/systems/ostep/Project-1B-initial-xv6.md b/docs/computer-science/systems/ostep/Initial-xv6.md similarity index 95% rename from docs/computer-science/systems/ostep/Project-1B-initial-xv6.md rename to docs/computer-science/systems/ostep/Initial-xv6.md index 9f34bc8..486d2c4 100644 --- a/docs/computer-science/systems/ostep/Project-1B-initial-xv6.md +++ b/docs/computer-science/systems/ostep/Initial-xv6.md @@ -1,7 +1,10 @@ -# Project 1B +# Intro to xv6 ### all thanks to [palladian](https://github.com/palladian1) +- [Project details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/initial-xv6) +- [Discussion](https://www.youtube.com/watch?v=vR6z2QGcoo8) + ### Linux Installation * Make sure you have a compatible compiler toolchain; if you're on Linux, gcc should work perfectly. diff --git a/docs/computer-science/systems/ostep/Kernel-Threads-xv6.md b/docs/computer-science/systems/ostep/Kernel-Threads-xv6.md new file mode 100644 index 0000000..bfbfb52 --- /dev/null +++ b/docs/computer-science/systems/ostep/Kernel-Threads-xv6.md @@ -0,0 +1,29 @@ +# Kernel Threads xv6 +**Note:** This project does not include any tests, and as a result we have decided to mark it as *optional*. We still believe it to be highly educational to complete this project, so we have provided instructions on how one might test their own implementation in the hints section. + +- [Project details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-xv6-threads) +- [Discussion](https://www.youtube.com/watch?v=G9nW9UbkT7s) + + +## Hints +The discussion provides some critical information about x86 calling conventions and how to manipulate the stack that you *will* need in order to complete this project. + +The kernel stack referred to in the project description does not refer to the user stack that gets allocated on the heap. It refers to the special stack managed by the kernel to save and load the process context. What changes need to be made here in order to run the thread? + +When a newly created process runs for the first time, it first returns to a special routine called "forkret" which releases the lock held on the process table before finally returning to the user program. The allocproc-routine takes care of setting up the process to do this. If your process does not enter forkret, then you will cause a kernel panic. + +Remember that the user stack grows from top to bottom, and that the pointer you get from malloc points to the bottom. Make sure the sp-register points to the top. + +As of December 2025 this project does not have any tests to verify your work, so you will have to create that yourself. We suggest the creating verification steps for the following: +1. Thread implementation + - Verify that the cloned actually runs the supplied function. + - Verify that it shares memory with its parent + - Verify that the supplied stack-address gets returned by the subsequent join + - Verify that if a thread doesn't call exit, it returns to 0xffffffff, causing a kernel trap, as opposed to the function that created the thread. +2. Lock + - Create a race condition and verify that it actually triggers + - Verify that the lock protects against the race condition + +The thread implementation itself should be straightforward to verify as it can be done by simply setting some variables and comparing them after a join. The tricky part is the race condition. You might be tempted to recreate the multi-threaded loop-counting example from the book, however, it turns out that it can be very difficult to make that race condition trigger at all. We believe this is a quirk of running xv6 inside QEMU. + +Instead, we recommend having a critical section of code that invokes multiple system calls. This guarantees interrupts, increasing the likelyhood of race conditions causing problems. A good candidate is printf. The xv6 implementation of printf invokes a write-syscall of a single byte for each printed character. Thus, if you use printf concurrently across multiple threads, there is a high chance that the terminal output gets garbled, unless you hold a working mutex lock for the duration of the printf-call. diff --git a/docs/computer-science/systems/ostep/Scheduling-xv6-lottery.md b/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md similarity index 97% rename from docs/computer-science/systems/ostep/Scheduling-xv6-lottery.md rename to docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md index e758e70..455b69e 100644 --- a/docs/computer-science/systems/ostep/Scheduling-xv6-lottery.md +++ b/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md @@ -1,8 +1,13 @@ -# Scheduling xv6 Lottery +# Lottery Scheduler xv6 +**Legacy Project:** this project has been replaced by the [STCF scheduler](STCF-Scheduler-xv6) project. -## all thanks to [palladian](https://github.com/palladian1) -### General Tips +- [Project details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/scheduling-xv6-lottery) +- [Discussion](https://www.youtube.com/watch?v=eYfeOT1QYmg) + +all thanks to [palladian](https://github.com/palladian1) + +## General Tips * Read chapter 9 in the OSTEP book and watch the video for discussion 5. Lottery ticket schedulers aren't discussed in the lectures, so you really do have to read the book for this one. diff --git a/docs/computer-science/systems/ostep/MapReduce.md b/docs/computer-science/systems/ostep/MapReduce.md new file mode 100644 index 0000000..4ee840f --- /dev/null +++ b/docs/computer-science/systems/ostep/MapReduce.md @@ -0,0 +1,11 @@ +# MapReduce +**Note:** This project currently does not contain any tests. A ZIP-file containing several test cases and a makefile + example C-program to run them has been uploaded to the Discord server. At some point in the future we will get it refined and published on GitHub. +- [Project details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-mapreduce) +- [Discussion](https://youtu.be/tSiJ_oBSOZE?t=34) +- [Q/A](https://www.youtube.com/watch?v=jVmWrr8y0Uw) + +## Hints +- This project is quite challenging, so don't get discouraged if you're struggling. If you find that the difficulty curve gets too steep, consider doing the [Parallel Zip](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-pzip) project first. Use the test-files in the initial utilities zip project to check your work. +- For this project it is highly advantageous to work in small increments: focus on making a correct sequential implementation first, and then think about how you can parallelize it. +- The producer-consumer queue using locks and condition variables is your central data structure for parallelizing work. It is described in detail in the chapter on **Condition Variables**. +- Remember to make copies of the keys and values passed to your `MR_Emit` implementation, otherwise you'll end up with a heap of garbage. diff --git a/docs/computer-science/systems/ostep/Memory-Mapping-xv6.md b/docs/computer-science/systems/ostep/Memory-Mapping-xv6.md new file mode 100644 index 0000000..35a1f2c --- /dev/null +++ b/docs/computer-science/systems/ostep/Memory-Mapping-xv6.md @@ -0,0 +1,14 @@ +# Memory Mapping xv6 +**Trial project**: This project is currently being tried as a replacement for the xv6 virtual memory project. If you wish to do the old project instead, click [here](#xv6-virtual-memory). + +- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall24/public/p5) + +## Instructions +1. Git clone the project repository. +2. Read the instructions provided in the README.md file. +3. Implement your solution in the `solution`-directory. +4. Run the `runtests.py` script located in the `tests` directory to evaluate your work. + +### Hints +- For this project you will need to have finished the **Virtual Memory** chapters of the OSTEP textbook. +- Pay particular attention to the chapters on **Address Translation**, **Segmentation**, and **Paging**. diff --git a/docs/computer-science/systems/ostep/Mini-WFS.md b/docs/computer-science/systems/ostep/Mini-WFS.md new file mode 100644 index 0000000..7d94ed3 --- /dev/null +++ b/docs/computer-science/systems/ostep/Mini-WFS.md @@ -0,0 +1,26 @@ +# Mini-WFS + +- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p6/p6-base) + +In this project you will implement a basic filesystem in userspace (FUSE) not unlike the Very Simple Filesystem described in the book. + +## Instructions +- Git clone the project repository +- Follow the instructions in the README.md file. +- Implement your solution in the solution directory. +- Run the `run-tests.sh` script located in the `tests`-directory to check your work. + +#### Hints +- This may be the most substantial project in the entire course. It is a lot of work, but in the process you will learn not just about file systems but about the FUSE framework as well, which is used in a lot of places. +- Part 1 represents roughly 90% of the work, so don't get discouraged if you've been at it for a week and you're still "only at part 1". Part 1 is "working filesystem", and that is a big deal in its own right. +- Work in increments. Start with the `wfs_getattr` callback and work your way down from there. If you're in doubt about what to do next, run the tests and start with the first test-failure you encounter. +- The starter files include a bunch of useful helpers that you should use: + - `allocate_block`: takes a bitmap array and an array length, flips the next free bit, and returns its number. This works for both the inode and data bitmaps. + - `fillin_inode`: initializes the provided inode. Use this for any newly allocated inodes. + - `wfs_color_from_code`: returns a color name and ansi-espace sequences for the provided color code. + - `parse_color_name`: takes a color name string and sets the `out_code` parameter to the parsed color code. BEWARE: this function returns `1` on success and ` 0` on failure. Check the return value. +- The VSFS described in the Filesystem Implementation chapter has directory entries for `.` and `..` in the data blocks for each directory inode. **You should not do this. You will get test failures**. Only start allocating data blocks for the directory inode when you start adding files and directories to it. +- Think about how to handle direct versus indirect blocks. How do you determine when to use one over the other. What information do you have available to determine that. Do you need some sort of transition case? +- Directory inodes need certain mode bits to be set on creation, and these will not be set by the `mode` argument passed to the callback. You can set them by setting the mode field on the inode to `mode | S_IFDIR`. +- The "Read test for prebuilt image" test (test 25) is likely to fail due to incompatibilities between your memory layout and theirs. Feel free to skip this test in particular. +- The timestamp tests appear to be buggy and pass no matter what. You should still implement the timestamp updates. diff --git a/docs/computer-science/systems/ostep/STCF-Scheduler.md b/docs/computer-science/systems/ostep/STCF-Scheduler.md new file mode 100644 index 0000000..bd713d6 --- /dev/null +++ b/docs/computer-science/systems/ostep/STCF-Scheduler.md @@ -0,0 +1,14 @@ +# STCF-Scheduler xv6 + +**Trial project**: This project is currently being tried as a replacement for the xv6 lottery scheduler project. If you wish to do the old project instead, click [here](#xv6-lottery-scheduler). +- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p3/p3-base) + +## Instructions +- Git clone the project repository +- Follow the instructions in the README.md file. +- Implement your solution in the solution directory. +- Run the `run-tests.sh` script located in the `tests`-directory to check your work. + +## Hints +- This project is very similar to the old [Lottery Scheduler](../Lottery-Scheduler-xv6) project. +- Read the hints and watch the discussion video for that project to get an idea of how to implement this project. diff --git a/docs/computer-science/systems/ostep/Project-2A-processes-shell.md b/docs/computer-science/systems/ostep/Unix-Shell.md similarity index 96% rename from docs/computer-science/systems/ostep/Project-2A-processes-shell.md rename to docs/computer-science/systems/ostep/Unix-Shell.md index eaf451b..8f16c73 100644 --- a/docs/computer-science/systems/ostep/Project-2A-processes-shell.md +++ b/docs/computer-science/systems/ostep/Unix-Shell.md @@ -1,14 +1,9 @@ -# Project 2A +# Unix Shell ### all thanks to [Palladian](https://github.com/palladian1/) -- [x] Interactive mode -- [x] Batch mode -- [x] exit -- [x] cd -- [x] path -- [x] Redirection -- [x] Parallel commands +- [Project Details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/processes-shell) +- [Discussion](https://youtu.be/76PfvXTwF04) ### Tips diff --git a/docs/computer-science/systems/ostep/Unix-Utilities.md b/docs/computer-science/systems/ostep/Unix-Utilities.md new file mode 100644 index 0000000..2466361 --- /dev/null +++ b/docs/computer-science/systems/ostep/Unix-Utilities.md @@ -0,0 +1,13 @@ +# Unix Utilities + +- [Project description](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/initial-utilities) +- [Discussion](https://youtu.be/rgcq9x8LtGQ) + +## Instructions +1. Git clone the ostep-projects repository if you haven't already. +2. Navigate to the initial-utilities directory. +3. Follow the instructions in the README.md file. + +### Hints +- This project is meant as a litmus-test for your proficiency at programming in C. It covers the very basics of being able to write useful programs. If you're struggling with this assignment, it may be a sign for you to take a step back, and get some more experience with C, as things will only get harder from here on out. +- If you still feel you need more C practice after completing this project, we recommend doing the [intial reverse](https://github.com/luccaflower/ostep-projects/tree/master/initial-reverse) project. But be mindful: The error messages that are needed to pass the tests are wrong! The provided text say `"error: ..."` but the tests expect `"reverse: ..."` so make sure to match the tests' expectations in your code. diff --git a/docs/computer-science/systems/ostep/vm-xv6-intro.md b/docs/computer-science/systems/ostep/Virtual-Memory-Intro-xv6.md similarity index 74% rename from docs/computer-science/systems/ostep/vm-xv6-intro.md rename to docs/computer-science/systems/ostep/Virtual-Memory-Intro-xv6.md index 6666339..70aa8fe 100644 --- a/docs/computer-science/systems/ostep/vm-xv6-intro.md +++ b/docs/computer-science/systems/ostep/Virtual-Memory-Intro-xv6.md @@ -1,10 +1,15 @@ - -# Intro To xv6 Virtual Memory +# Virtual Memory Intro xv6 +**This is a legacy project which has been replaced by the [Memory Mapping xv6 Project]().** Credit goes to [palladian](https://github.com/palladian1) +- [Project details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/vm-xv6-intro) +- [Discussion 1](https://youtu.be/z6dqk6iBBRY?t=1305) +- [Discussion 2](https://www.youtube.com/watch?v=WVHRaqom0y) + ### WARNING: + ***The project doesn't match the currently available xv6 source code, which already has this project implemented in it!*** [palladian](https://github.com/palladian1) tracked down a different xv6 source from the Github page of a U of Wisconsin student. We had to edit the `Makefile` to find the QEMU executable correctly. We added `null.c` to the `user` folder (also edited `makefile.mk` there), which demonstrates the lack of memory safety. diff --git a/docs/computer-science/systems/ostep/index.md b/docs/computer-science/systems/ostep/index.md index 4a846ca..8ef0f1c 100644 --- a/docs/computer-science/systems/ostep/index.md +++ b/docs/computer-science/systems/ostep/index.md @@ -67,22 +67,22 @@ This course was originally taught as CS 537 at the University of Wisconsin by th | Topic | Readings | Lectures | Projects | | ------- | -------- | -------- | ---------| -| Intro | [[pre]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/preface.pdf) [[1]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/dialogue-threeeasy.pdf) [[2]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/intro.pdf) | [[1.1]](https://www.youtube.com/watch?v=3uMbb9dLtlE) [[1.2]](https://www.youtube.com/watch?v=K4qbAiC77Yo) | [Unix Utilities](#unix-utilities) | +| Intro | [[pre]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/preface.pdf) [[1]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/dialogue-threeeasy.pdf) [[2]](https://pages.cs.wisc.edu/~remzi/Classes/537/Spring2018/Book/intro.pdf) | [[1.1]](https://www.youtube.com/watch?v=3uMbb9dLtlE) [[1.2]](https://www.youtube.com/watch?v=K4qbAiC77Yo) | [Unix Utilities](Unix-Utilities) | | **Virtualization** | | | | | *Dialogue* | [[3]](https://pages.cs.wisc.edu/~remzi/OSTEP/dialogue-virtualization.pdf) | | | -| Processes | [[4]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-intro.pdf) [[5]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-api.pdf) [[6]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-mechanisms.pdf) | [[1.3]](https://youtu.be/LVxN7ZkGh3w) [[2.1]](https://youtu.be/oTd72Yp2m8w) | [Unix Shell](#unix-shell) | -| Scheduling | [[7]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched.pdf) [[8]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-mlfq.pdf) [[9]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-lottery.pdf) [[10]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-multi.pdf) | [[2.2]](https://youtu.be/Q09UgVfragU) [[2.3]](https://youtu.be/fin5-82L-r8) [[3.1]](https://youtu.be/cAiwISFta4g) | [xv6 Intro](#xv6-intro)
[STCF-scheduler](#stcf-scheduler) | +| Processes | [[4]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-intro.pdf) [[5]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-api.pdf) [[6]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-mechanisms.pdf) | [[1.3]](https://youtu.be/LVxN7ZkGh3w) [[2.1]](https://youtu.be/oTd72Yp2m8w) | [Unix Shell](Unix-Shell) | +| Scheduling | [[7]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched.pdf) [[8]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-mlfq.pdf) [[9]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-lottery.pdf) [[10]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-multi.pdf) | [[2.2]](https://youtu.be/Q09UgVfragU) [[2.3]](https://youtu.be/fin5-82L-r8) [[3.1]](https://youtu.be/cAiwISFta4g) | [Intro to xv6](Initial-xv6)
[STCF-scheduler](STCF-Scheduler) | | *Summary* | [[11]](https://pages.cs.wisc.edu/~remzi/OSTEP/cpu-dialogue.pdf) | | | | *Dialogue* | [[12]](https://pages.cs.wisc.edu/~remzi/OSTEP/dialogue-vm.pdf) | | | | Virtual Memory | [[13]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-intro.pdf) [[14]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-api.pdf) [[15]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-mechanism.pdf) [[16]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-segmentation.pdf) [[17]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-freespace.pdf) | [[3.2]](https://youtu.be/I0RIlSN0DzM) [[3.3]](https://youtu.be/0WVoWlOT-kY) | | | Paging | [[18]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-paging.pdf) [[19]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-tlbs.pdf) [[20]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-smalltables.pdf) | [[4.1]](https://youtu.be/wAx_h3HkIX0) [[4.2]](https://youtu.be/7BOXM2XgGO4) [[4.3]](https://youtu.be/LprKOBsALGA) [[5.1]](https://www.youtube.com/watch?v=ggPkFxOTwHY) | | -| Beyond physical | [[21]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys.pdf) [[22]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys-policy.pdf) [[23]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-complete.pdf) | [[5.2]](https://www.youtube.com/watch?v=4tPXkN5nRQs) | [Virtual Memory](#virtual-memory) | +| Beyond physical | [[21]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys.pdf) [[22]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys-policy.pdf) [[23]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-complete.pdf) | [[5.2]](https://www.youtube.com/watch?v=4tPXkN5nRQs) | [Memory Mapping](Memory-Mapping-xv6) | | *Summary* | [[24]](https://pages.cs.wisc.edu/~remzi/OSTEP/vm-dialogue.pdf) | | | | **Concurrency** | | | | | *Dialogue* | [[25]](https://pages.cs.wisc.edu/~remzi/OSTEP/dialogue-concurrency.pdf) | | | | | Threads |[[26]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-intro.pdf) [[27]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-api.pdf) | [[6.1]](https://www.youtube.com/watch?v=4PghlMdp9cU) | | | Concurrency primitives | [[28]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-locks.pdf) [[29]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-locks-usage.pdf) [[30]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf) [[31]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf) | [[6.2]](https://www.youtube.com/watch?v=hivv8F-LjzY) [[6.3]](https://youtu.be/BoLYvNp2Lc4) [[7.1]](https://www.youtube.com/watch?v=G95w4ghn42A) [[7.2]](https://www.youtube.com/watch?v=X5clCyJ4uuk) [[7.3]](https://youtu.be/RN8A9EvKBdY) [[8.1]](https://youtu.be/U1LfmL7f1h8) [[8.2]](https://youtu.be/cuY8r8RXqAY) | | -| More topics in concurrency | [[32]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-bugs.pdf) [[33]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-events.pdf) | [[9.1]](https://youtu.be/Fnp_K63ss44) [[9.2]](https://youtu.be/AMG29dlH8t0) | [MapReduce](#mapreduce) | +| More topics in concurrency | [[32]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-bugs.pdf) [[33]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-events.pdf) | [[9.1]](https://youtu.be/Fnp_K63ss44) [[9.2]](https://youtu.be/AMG29dlH8t0) | Optional: [Kernel Threads xv6](Kernel-Threads-xv6)
[MapReduce](MapReduce) | | *Summary* | [[34]](https://pages.cs.wisc.edu/~remzi/OSTEP/threads-dialogue.pdf) | | | | **Midterm review** | | [Review](https://www.youtube.com/watch?v=doHSi2ffu9I) | | | **Persistence** | @@ -92,7 +92,7 @@ This course was originally taught as CS 537 at the University of Wisconsin by th | Filesystems | [[39]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-intro.pdf) [[40]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-implementation.pdf) | [[11.4]](https://youtu.be/EDFoFlzZ8_w) [[12.1]](https://youtu.be/QMjJlCqUYW4) [[12.2]](https://youtu.be/87vv7nVdTDA) [[12.3]](https://youtu.be/5n0AdNuBObU) | | | Crash consistency, journaling, FFS | [[41]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-ffs.pdf) [[42]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-journaling.pdf) | [[13.1]](https://youtu.be/piwPJ0sLV0Y) [[13.2]](https://youtu.be/MgnQV-ss1wc) [[13.3]](https://youtu.be/wwvMNItRyl8) | | | LFS and SSDs | [[43]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-lfs.pdf) [[44]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-ssd.pdf) | [[14.1]](https://youtu.be/59XSFnXQ-9Q) [[14.2]](https://youtu.be/6fbm9u7__L0) [[14.3]](https://youtu.be/vvttbstRdj8) [[14.4]](https://youtu.be/sKTyhqvTUBU) | | -| Data Integrity and Protection | [[45]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-integrity.pdf) | | [Mini-WFS](#mini-wfs) | +| Data Integrity and Protection | [[45]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-integrity.pdf) | | [Mini-WFS](Mini-WFS) | | *Summary* | [[46]](https://pages.cs.wisc.edu/~remzi/OSTEP/file-dialogue.pdf) | | | | **End of class review** | | [Part 1](https://www.youtube.com/watch?v=TSiMDPFquO4), [Part 2](https://youtu.be/jIcw7B36oBU) | | @@ -168,32 +168,6 @@ However, that book glosses over a lot of the details in the code that you might Also [here](https://www.youtube.com/playlist?list=PLbtzT1TYeoMhTPzyTZboW_j7TPAnjv9XB) is an excellent video series walking through much of the xv6 code. -#### Initial Reverse -The error messages that are needed to pass the tests were wrong! The provided text said `"error: ..."` but the tests expected `"reverse: ..."` so make sure to match the tests' expectations in your code. - -#### xv6 Kernel Threads -The [discussion](https://www.youtube.com/watch?v=G9nW9UbkT7s) provides some critical information about x86 calling conventions and how to manipulate the stack that you *will* need in order to complete this project. - -The kernel stack referred to in the project description does not refer to the user stack that gets allocated on the heap. It refers to the special stack managed by the kernel to save and load the process context. What changes need to be made here in order to run the thread? - -When a newly created process runs for the first time, it first returns to a special routine called "forkret" which releases the lock held on the process table before finally returning to the user program. The allocproc-routine takes care of setting up the process to do this. If your process does not enter forkret, then you will cause a kernel panic. - -Remember that the user stack grows from top to bottom, and that the pointer you get from malloc points to the bottom. Make sure the sp-register points to the top. - -As of December 2025 this project does not have any tests to verify your work, so you will have to create that yourself. We suggest the creating verification steps for the following: -1. Thread implementation - - Verify that the cloned actually runs the supplied function. - - Verify that it shares memory with its parent - - Verify that the supplied stack-address gets returned by the subsequent join - - Verify that if a thread doesn't call exit, it returns to 0xffffffff, causing a kernel trap, as opposed to the function that created the thread. -2. Lock - - Create a race condition and verify that it actually triggers - - Verify that the lock protects against the race condition - -The thread implementation itself should be straightforward to verify as it can be done by simply setting some variables and comparing them after a join. The tricky part is the race condition. You might be tempted to recreate the multi-threaded loop-counting example from the book, however, it turns out that it can be very difficult to make that race condition trigger at all. We believe this is a quirk of running xv6 inside QEMU. - -Instead, we recommend having a critical section of code that invokes multiple system calls. This guarantees interrupts, increasing the likelyhood of race conditions causing problems. A good candidate is printf. The xv6 implementation of printf invokes a write-syscall of a single byte for each printed character. Thus, if you use printf concurrently across multiple threads, there is a high chance that the terminal output gets garbled, unless you hold a working mutex lock for the duration of the printf-call. - #### Miscellaneous You'll need a general sense of how Makefiles work in order to use the Makefile for xv6. [This tutorial](https://makefiletutorial.com) covers much more than you need; just read the "Getting Started" and "Targets" sections and come back to the rest later if you need to look something up (but you shouldn't have to). @@ -204,59 +178,3 @@ Additional (optional) resources include: * [OSDev Wiki](https://wiki.osdev.org): a great resource for all kinds of OS concepts and implementation details. * [*Linux Kernel Development*](https://www.amazon.com/dp/0672329468): if you want to apply your xv6 knowledge toward contributing to the Linux kernel, this is a great read after OSTEP. -### STCF-scheduler -**Trial project**: This project is currently being tried as a replacement for the xv6 lottery scheduler project. If you wish to do the old project instead, click [here](#xv6-lottery-scheduler). -- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p3/p3-base) - -The discussion video and hints for the lottery scheduler project will likely be helpful for this project too. - -## Project directory - -### Unix Utilities -- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/initial-utilities) -- [discussion](https://youtu.be/rgcq9x8LtGQ) - -### Unix Shell -- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/processes-shell) -- [discussion](https://youtu.be/76PfvXTwF04) -- [hints](Project-2A-processes-shell) - -### xv6 intro -- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/initial-xv6) -- [discussion](https://www.youtube.com/watch?v=vR6z2QGcoo8) -- [hints](Project-1B-initial-xv6) - -### Virtual Memory -**Trial project**: This project is currently being tried as a replacement for the xv6 virtual memory project. If you wish to do the old project instead, click [here](#xv6-virtual-memory). -- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall24/public/p5) - -### MapReduce -- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-mapreduce) -- [discussion](https://youtu.be/tSiJ_oBSOZE?t=34) -- [Q/A](https://www.youtube.com/watch?v=jVmWrr8y0Uw) - -### Mini-WFS -- [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p6/p6-base) - -In this project you will implement a basic filesystem in userspace (FUSE) not unlike the Very Simple Filesystem described in the book. -#### Hints -TBW - -### xv6 lottery scheduler -**Legacy project:** This project will be removed sometime in the future. -- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/scheduling-xv6-lottery) -- [discussion](https://www.youtube.com/watch?v=eYfeOT1QYmg) -- [hints](Scheduling-xv6-lottery) - -### xv6 virtual memory -**Legacy project:** This project will be removed sometime in the future. -- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/vm-xv6-intro) -- [discussion 1](https://youtu.be/z6dqk6iBBRY?t=1305) -- [discussion 2](https://www.youtube.com/watch?v=WVHRaqom0y) -- [hints](vm-xv6-intro) - -### Kernel Threads -**Legacy project** -- [details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/concurrency-xv6-threads) -- [discussion](https://www.youtube.com/watch?v=G9nW9UbkT7s) -- [hints](#xv6-kernel-threads) From a7d07756097bcbfa083283e2870eddae2c8f9634 Mon Sep 17 00:00:00 2001 From: ada Date: Mon, 19 Jan 2026 18:07:20 +0100 Subject: [PATCH 4/8] Fix path --- docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md b/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md index 455b69e..9df0e64 100644 --- a/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md +++ b/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md @@ -1,5 +1,5 @@ # Lottery Scheduler xv6 -**Legacy Project:** this project has been replaced by the [STCF scheduler](STCF-Scheduler-xv6) project. +**Legacy Project:** this project has been replaced by the [STCF scheduler](../STCF-Scheduler-xv6) project. - [Project details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/scheduling-xv6-lottery) From 005c0425b4a9073ffc6458829f3c757c8dcd5187 Mon Sep 17 00:00:00 2001 From: ada Date: Mon, 19 Jan 2026 18:09:07 +0100 Subject: [PATCH 5/8] Path to STCF-scheduler page does not include xv6 --- docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md b/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md index 9df0e64..3daa4a3 100644 --- a/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md +++ b/docs/computer-science/systems/ostep/Lottery-Scheduler-xv6.md @@ -1,5 +1,5 @@ # Lottery Scheduler xv6 -**Legacy Project:** this project has been replaced by the [STCF scheduler](../STCF-Scheduler-xv6) project. +**Legacy Project:** this project has been replaced by the [STCF Scheduler](../STCF-Scheduler) project. - [Project details](https://github.com/remzi-arpacidusseau/ostep-projects/tree/master/scheduling-xv6-lottery) From 5e418aed2e69f4454e913651253d86ebeded4d21 Mon Sep 17 00:00:00 2001 From: ada Date: Mon, 19 Jan 2026 18:11:38 +0100 Subject: [PATCH 6/8] Fill out mmap url in xv6 vm intro page --- docs/computer-science/systems/ostep/Virtual-Memory-Intro-xv6.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/computer-science/systems/ostep/Virtual-Memory-Intro-xv6.md b/docs/computer-science/systems/ostep/Virtual-Memory-Intro-xv6.md index 70aa8fe..dd2b0cb 100644 --- a/docs/computer-science/systems/ostep/Virtual-Memory-Intro-xv6.md +++ b/docs/computer-science/systems/ostep/Virtual-Memory-Intro-xv6.md @@ -1,5 +1,5 @@ # Virtual Memory Intro xv6 -**This is a legacy project which has been replaced by the [Memory Mapping xv6 Project]().** +**This is a legacy project which has been replaced by the [Memory Mapping xv6 Project](../Memory-Mapping-xv6).** Credit goes to [palladian](https://github.com/palladian1) From c4a6570116d525c8ab2da881c7c6ed7b6d826ecd Mon Sep 17 00:00:00 2001 From: ada Date: Mon, 19 Jan 2026 18:14:56 +0100 Subject: [PATCH 7/8] Replace broken anchors with urls --- docs/computer-science/systems/ostep/Memory-Mapping-xv6.md | 2 +- docs/computer-science/systems/ostep/STCF-Scheduler.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/computer-science/systems/ostep/Memory-Mapping-xv6.md b/docs/computer-science/systems/ostep/Memory-Mapping-xv6.md index 35a1f2c..20b4497 100644 --- a/docs/computer-science/systems/ostep/Memory-Mapping-xv6.md +++ b/docs/computer-science/systems/ostep/Memory-Mapping-xv6.md @@ -1,5 +1,5 @@ # Memory Mapping xv6 -**Trial project**: This project is currently being tried as a replacement for the xv6 virtual memory project. If you wish to do the old project instead, click [here](#xv6-virtual-memory). +**Trial project**: This project is currently being tried as a replacement for the xv6 virtual memory project. If you wish to do the old project instead, click [here](../Virtual-Memory-Intro-xv6). - [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall24/public/p5) diff --git a/docs/computer-science/systems/ostep/STCF-Scheduler.md b/docs/computer-science/systems/ostep/STCF-Scheduler.md index bd713d6..86cf07a 100644 --- a/docs/computer-science/systems/ostep/STCF-Scheduler.md +++ b/docs/computer-science/systems/ostep/STCF-Scheduler.md @@ -1,6 +1,6 @@ # STCF-Scheduler xv6 -**Trial project**: This project is currently being tried as a replacement for the xv6 lottery scheduler project. If you wish to do the old project instead, click [here](#xv6-lottery-scheduler). +**Trial project**: This project is currently being tried as a replacement for the xv6 lottery scheduler project. If you wish to do the old project instead, click [here](../Lottery-Scheduler-xv6). - [Project repository](https://git.doit.wisc.edu/cdis/cs/courses/cs537/fall25/p3/p3-base) ## Instructions From 7a0281ad15c32dd4221215f7700550d325fe9563 Mon Sep 17 00:00:00 2001 From: ada Date: Mon, 19 Jan 2026 18:52:23 +0100 Subject: [PATCH 8/8] What the hell lol --- docs/computer-science/systems/ostep/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/computer-science/systems/ostep/index.md b/docs/computer-science/systems/ostep/index.md index 8ef0f1c..a6e4c85 100644 --- a/docs/computer-science/systems/ostep/index.md +++ b/docs/computer-science/systems/ostep/index.md @@ -44,7 +44,7 @@ If you are getting stuck on these, please don't spend too much time on them. The ## Extended Approach -If you've chosen this ophttps://www.reddit.com/r/Twitch/comments/11oy4y2/extremely_low_quality_streams_0_quality_on_all/tion, it is a *must* that you complete *both* courses in [Intro to Computer Systems](../computer-systems/index.md) or equivalent first. +If you've chosen this option, it is a *must* that you complete *both* courses in [Intro to Computer Systems](../computer-systems/index.md) or equivalent first. That said, if you're able to commit the time required for the prerequisites, we believe the reward is well worth the effort: this course is exciting, interesting, and quite useful for other fields of computer science and programming. One big attraction of this course is the opportunity to see a simplified but fully-functional Unix-like operating system in action and understand the concepts and design decisions that went into it as well as the low-level implementation details.