NetBSDfr · csdvrx · Dec 9, 2023 · Dec 9, 2023 · Dec 10, 2023 · Dec 10, 2023
diff --git a/README.md b/README.md
@@ -1,56 +1,116 @@
-NetBSD
-======
-
-NetBSD is a free, fast, secure, and highly portable Unix-like Open
-Source operating system.  It is available for a [wide range of
-platforms](https://wiki.NetBSD.org/ports/), from large-scale servers
-and powerful desktop systems to handheld and embedded devices.
+NetBSD-perf
+===========
+
+This is my attempt at replicating iMil results that were reported on https://mail-index.netbsd.org/tech-kern/2024/01/23/msg029450.html and https://old.reddit.com/r/BSD/comments/197vfmq by using the perf branch he published on https://github.com/NetBSDfr/NetBSD-src
+
+Currently, I'm still about 5x slower as I boot in 200 ms, but this is a good beginning:
+
+```
+[   1.0000000] cpu_rng: rdrand/rdseed
+[   1.0000000] entropy: ready
+[   1.0000000] NetBSD 10.99.10 (MICROVM)       Notice: this software is protected by copyright
+[   1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
+[   1.0000000]     2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
+[   1.0000000]     2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
+[   1.0000000]     2024
+[   1.0000000]     The NetBSD Foundation, Inc.  All rights reserved.
+[   1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
+[   1.0000000]     The Regents of the University of California.  All rights reserved.
+[   1.0000000] NetBSD 10.99.10 (MICROVM) #5: Tue Jan 30 19:22:09 CST 2024
+[   1.0000000]  charlotte@x1ng2:/home/charlotte/obj/sys/arch/amd64/compile/MICROVM
+[   1.0000000] total memory = 127 MB
+[   1.0000000] avail memory = 77184 KB
+[   1.0000000] timecounter: Timecounters tick every 10.000 msec
+[   1.0000000] timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100
+[   1.0000030] Hypervisor: KVM
+[   1.0000030] VMM: Generic PVH
+[   1.0000030] mainbus0 (root)
+[   1.0000030] mainbus0: Intel MP Specification (Version 1.4) (QBOOT    000000000000)
+[   1.0000030] cpu0 at mainbus0 apid 0
+[   1.0000030] cpu0: Use lfence to serialize rdtsc
+[   1.0000030] got tsc from vmware compatible cpuid
+[   1.0000030] cpu0: TSC freq CPUID 2496000000 Hz
+[   1.0000030] cpu0: 12th Gen Intel(R) Core(TM) i7-1270P, id 0x906a2
+[   1.0000030] cpu0: node 0, package 0, core 0, smt 0
+[   1.0000030] mpbios: bus 0 is type ISA
+[   1.0000030] ioapic0 at mainbus0 apid 2: pa 0xfec00000, version 0x20, 24 pins
+[   1.0000030] isa0 at mainbus0
+[   1.0000030] com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, 16-byte FIFO
+[   1.0000030] com0: console
+[   1.0000030] allocated pic ioapic0 type edge pin 4 level 8 to cpu0 slot 0 idt entry 129
+[   1.0000030] pv0 at mainbus0
+[   1.0000030] virtio0 at pv0
+[   1.0000030] kernel parameters: root=ld0c rw console=com -z -v virtio_mmio.device=512@0xfeb00e00:12 virtio_mmio.device=512@0xfeb00c00:11
+[   1.0000030] viommio: 512@0xfeb00e00:12
+[   1.0000030] virtio0: VirtIO-MMIO-v2
+[   1.0000030] virtio0: entropy device (id 4, rev. 0x01)
+[   1.0000030] viornd0 at virtio0: features: 0x110000000<V1,INDIRECT_DESC>
+[   1.0000030] virtio0: allocated 32768 byte for virtqueue 0 for Entropy request, size 1024
+[   1.0000030] allocated pic ioapic0 type level pin 12 level 6 to cpu0 slot 1 idt entry 96
+[   1.0000030] virtio0: interrupting on -1
+[   1.0000030] virtio1 at pv0
+[   1.0000030] viommio: 512@0xfeb00c00:11
+[   1.0000030] virtio1: VirtIO-MMIO-v2
+[   1.0000030] virtio1: block device (id 2, rev. 0x01)
+[   1.0000030] ld0 at virtio1: features: 0x110000a54<V1,INDIRECT_DESC,CONFIG_WCE,FLUSH,BLK_SIZE,GEOMETRY,SEG_MAX>
+[   1.0000030] virtio1: allocated 4227072 byte for virtqueue 0 for I/O request, size 1024
+[   1.0000030] virtio1: using 4194304 byte (262144 entries) indirect descriptors
+[   1.0000030] allocated pic ioapic0 type level pin 11 level 6 to cpu0 slot 2 idt entry 97
+[   1.0000030] virtio1: interrupting on -1
+[   1.0000030] ld0: 512 MB, 1040 cyl, 16 head, 63 sec, 512 bytes/sect x 1048576 sectors
+[   1.0000030] timecounter: Timecounter "lapic" frequency 999998000 Hz quality -100
+[   1.0000030] timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
+[   1.0000030] timecounter: Timecounter "TSC" frequency 2496000000 Hz quality 3000
+[   1.0001656] boot device: ld0
+[   1.0001656] root on ld0c dumps on ld0b
+[   1.0001656] root file system type: ffs
+[   1.0001656] kern.module.path=/stand/amd64/10.99.10/modules
+[   1.0001656] WARNING: clock gained 4 days
+[   1.0050716] boot: 199ms (entry tsc: 281364007)
+[   1.0050716] exec /sbin/init: error 8
+[   1.0050716] init: trying /sbin/oinit
+bslinit v7 starting on NetBSD, will handle 12 signals, reaping zombies every 30 s
+        arg 0: oinit
+oinit: mount /proc failed: Invalid argument
+# Epoch 1706665417, time 2024-01-31T01:43:37
+```
+
+Note however that the qemu append doesn't seem to survive a reboot:
+
+```
+# init: can't add utmpx record for `system boot': No such file or directory
+init: can't add utmpx record for `system down': No such file or directory
+qemu-system-x86_64: terminating on signal 2
+[   1.0000000] cpu_rng: rdrand/rdseed
+[   1.0000000] entropy: ready
+[   1.0000000] NetBSD 10.99.10 (MICROVM)       Notice: this software is protected by copyright
+[   1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
+[   1.0000000]     2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
+[   1.0000000]     2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
+[   1.0000000]     2024
+[   1.0000000]     The NetBSD Foundation, Inc.  All rights reserved.
+[   1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
+[   1.0000000]     The Regents of the University of California.  All rights reserved.
+(...)
+[   1.0155438] root on ld0c dumps on ld0b
+[   1.0155438] vfs_mountroot: can't open root device
+[   1.0155438] cannot mount root, error = 6
+[   1.0155438] root device (default ld0c): qemu-system-x86_64: terminating on signal 2
+```
 
 Building
 --------
 
-You can cross-build NetBSD from most UNIX-like operating systems.
-To build for amd64 (x86_64), in the src directory:
-
-    ./build.sh -U -u -j4 -m amd64 -O ~/obj release
-
-Additional build information available in the [BUILDING](BUILDING) file.
+    sh ./compile.sh
 
 Binaries
 --------
 
-- [Daily builds](https://nycdn.NetBSD.org/pub/NetBSD-daily/HEAD/latest/)
-- [Releases](https://cdn.NetBSD.org/pub/NetBSD/)
-
-Testing
--------
-
-On a running NetBSD system:
-
-    cd /usr/tests; atf-run | atf-report
-
-Troubleshooting
----------------
-
-- Send bugs and patches [via web form](https://www.NetBSD.org/cgi-bin/sendpr.cgi?gndb=netbsd).
-- Subscribe to the [mailing lists](https://www.NetBSD.org/mailinglists/).
-  The [netbsd-users](https://www.NetBSD.org/mailinglists/#netbsd-users) list is a good choice for many problems; watch [current-users](https://www.NetBSD.org/mailinglists/#current-users) if you follow the bleeding edge of NetBSD-current.
-- Join the community IRC channel [#netbsd @ libera.chat](https://web.libera.chat/#netbsd).
-
-Latest sources
---------------
-
-To fetch the main CVS repository:
-
-    cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -P src
-
-To work in the Git mirror, which is updated every few hours from CVS:
+When it'll be ready, you'll see the releases on the right handside
 
-    git clone https://github.com/NetBSD/src.git
+TODO
+----
 
-Additional Links
-----------------
+Adding flamecharts to understand why it's so slow
 
-- [The NetBSD Guide](https://www.NetBSD.org/docs/guide/en/)
-- [NetBSD manual pages](https://man.NetBSD.org/)
-- [NetBSD Cross-Reference](https://nxr.NetBSD.org/)
+Adding a basic rootfs with simple tools such as bslinit, the tslog script to create flamecharts, and a shell
diff --git a/compile.sh b/compile.sh
@@ -0,0 +1,45 @@
+#!/bin/sh
+# Replication of iMil result reported on https://mail-index.netbsd.org/tech-kern/2024/01/23/msg029450.html
+KERNELCONFIG="MICROVM"
+TOOLCHAIN="$HOME/obj"
+CPUS=4
+
+# Get the sources
+#git clone https://github.com/NetBSDfr/NetBSD-src
+
+# Show the branches available
+#git branch -a
+
+# Change to result-replication, based on the perf branch
+git branch| grep result-replication || git switch result-replication
+
+#cf https://www.netbsd.org/docs/guide/en/chap-build.html#chap-boot-cross-compiling-kernel
+echo "# Building the cross compilation toolchain into ${TOOLCHAIN}":
+
+stat -c "%y %A %n size: %s" ${TOOLCHAIN}/tooldir*x86_64/bin/nbmake \
+ || sh ./build.sh -U -O ${TOOLCHAIN} -j ${CPUS} -m amd64 -a x86_64 tools \
+ || exit 1
+
+echo "# Using the kernel configuration ${KERNELCONFIG}":
+stat -c "%y %A %n size: %s" sys/arch/amd64/conf/${KERNELCONFIG} \
+ || exit 2
+
+echo "# Checking pvclock is disabled as it requires the missing <dev/pv/pvreg.h>"
+grep -v ^pvclock sys/arch/amd64/conf/${KERNELCONFIG} \
+ || exit 3
+
+echo "# Refreshing the kernel build with -u" \
+ && sh ./build.sh -u -U -O ${TOOLCHAIN} -j ${CPUS} -m amd64  kernel=${KERNELCONFIG} \
+ && stat -c "%y %A %n size: %s" ${TOOLCHAIN}/sys/arch/amd64/compile/${KERNELCONFIG}/netbsd \
+ || exit 4
+
+## Alternatives:
+# Use nbconfig and nbmake directly with sys/arch/amd64/conf/kern-compile.sh
+
+# Don't just update the kernel: remove the '-u' that's for update only
+
+# Compile from scratch the release and the kernel
+#sh ./build.sh -U -O ${TOOLCHAIN} -j ${CPUS} -m amd64 -a x86_64 release kernel=${KERNELCONFIG}
+
+# TODO: add qemu scripts, which should have console=com etc
+# TODO: add a basic init_args parsing arguments like sys/arch/x86/x86/x86_autoconf.c
diff --git a/share/man/man4/pv.4 b/share/man/man4/pv.4
@@ -0,0 +1,44 @@
+.\"	$NetBSD$
+.\"
+.\" Copyright (c) 2024 The NetBSD Foundation, Inc.
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions
+.\" are met:
+.\" 1. Redistributions of source code must retain the above copyright
+.\"    notice, this list of conditions and the following disclaimer.
+.\" 2. Redistributions in binary form must reproduce the above copyright
+.\"    notice, this list of conditions and the following disclaimer in the
+.\"    documentation and/or other materials provided with the distribution.
+.\"
+.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+.\" PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+.\" POSSIBILITY OF SUCH DAMAGE.
+.\"
+.\" ported from OpenBSD
+.Dd January 2024
+.Dt PV 4
+.Os
+.Sh NAME
+.Nm pv
+.Nd paravirtual device tree root
+.Sh SYNOPSIS
+.Cd "pv* at pvbus?"
+.Sh DESCRIPTION
+.Nm
+driver is used on virtual machines that are running on hypervisors.
+It provides a pseudo-bus for all paravirtual devices that do not
+attach to a well-known bus like
+.Xr pci 4 .
+.Sh SEE ALSO
+.Xr virtio 4 ,
+.Xr mmio_virtio 4
diff --git a/share/man/man4/pvclock.4 b/share/man/man4/pvclock.4
@@ -0,0 +1,74 @@
+.\"	$NetBSD$
+.\"
+.\" Copyright (c) 2024 The NetBSD Foundation, Inc.
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions
+.\" are met:
+.\" 1. Redistributions of source code must retain the above copyright
+.\"    notice, this list of conditions and the following disclaimer.
+.\" 2. Redistributions in binary form must reproduce the above copyright
+.\"    notice, this list of conditions and the following disclaimer in the
+.\"    documentation and/or other materials provided with the distribution.
+.\"
+.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+.\" PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+.\" POSSIBILITY OF SUCH DAMAGE.
+.\"
+.\" ported from OpenBSD
+.\"	$OpenBSD: pvclock.4,v 1.2 2018/11/23 17:21:37 jmc Exp $
+.\"
+.\" Copyright (c) 2018 Reyk Floeter <reyk@openbsd.org>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd January 2024
+.Dt PVCLOCK 4
+.Os
+.Sh NAME
+.Nm pvclock
+.Nd paravirtual clock driver
+.Sh SYNOPSIS
+.Cd "pvclock* at pv?"
+.Sh DESCRIPTION
+The
+.Nm
+driver supports the paravirtual clock that is available in KVM and
+other hypervisors.
+.Nm
+uses a shared page between the host and the hypervisor to synchronize
+the TSC clock in an efficient way.
+.Sh SEE ALSO
+.Xr pv 4
+.Sh HISTORY
+The
+.Nm
+driver first appeared in
+.Nx 11 .
+.Sh AUTHORS
+.An -nosplit
+The
+.Nm
+driver was written by
+.An Reyk Floeter Aq Mt reyk@openbsd.org .
+.Nx port was made by
+.An Emile 'iMil' Heitor Aq Mt imil@NetBSD.org .