From 7b9e9de65abab29e735e1b4536414d46e69ede94 Mon Sep 17 00:00:00 2001 From: lif <> Date: Wed, 14 May 2025 02:03:49 +0000 Subject: [PATCH] Emulated eXtensible Host Controller Interface (xHCI) device Special thanks to @luqmana getting this started in early 2023: https://github.com/luqmana/propolis/commits/xhci/ The version of the standard referenced throughout the comments in this module is xHCI 1.2, but we do not implement the features required of a 1.1 or 1.2 compliant host controller - that is, we are only implementing a subset of what xHCI version 1.0 requires of an xHC, as described by version 1.2 of the *specification*. https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/extensible-host-controler-interface-usb-xhci.pdf At present, the only USB device supported is a USB 2.0 `NullUsbDevice` with no actual functionality, which exists as a proof-of-concept and as a means to show that USB `DeviceDescriptor`s are successfully communicated to the guest in phd-tests. ``` +---------+ | PciXhci | +---------+ | has-a +-----------------------------+ | XhciState | |-----------------------------| | PCI MMIO registers | | XhciInterrupter | | DeviceSlotTable | | Usb2Ports + Usb3Ports | | CommandRing | | newly attached USB devices | +-----------------------------+ | has-a | +-------------------+ | has-a | XhciInterrupter | +-----------------+ |-------------------| | DeviceSlotTable | | EventRing | |-----------------| | MSI-X/INTxPin | | DeviceSlot(s) |___+------------------+ +-------------------+ | DCBAAP | | DeviceSlot | | Active USB devs | |------------------| +-----------------+ | TransferRing(s) | +------------------+ ``` Conventions =========== Wherever possible, the framework represents `Trb` data through a further level of abstraction, such as enums constructed from the raw TRB bitfields before being passed to other parts of the system that use them, such that the behavior of identifying `TrbType` and accessing their fields properly according to the spec lives in a conversion function rather than strewn across implementation of other xHC functionality. The nomenclature used is generally trading the "Descriptor" suffix for "Info", e.g. the high-level enum-variant version of an `EventDescriptor` is `EventInfo` (which is passed to the `EventRing` to be converted into Event TRBs and written into guest memory). For 1-based indices defined by the spec (slot ID, port ID), we use [SlotId] and [PortId] and their respective `.as_index()` methods to index into our internal arrays of slots and ports, such that we aspire to categorically avoid off-by-one errors of omission (of `- 1`). (Note that indexing into the DCBAA is *not* done with this method, as position 0 in it is reserved by the spec for the Scratchpad described in xHCI 1.2 section 4.20) Implementation ============== `DeviceSlotTable` ----------------- When a USB device is attached to the xHC, it is enqueued in a list within `XhciState` along with its `PortId`. The next time the xHC runs: - it will update the corresponding **PORTSC** register and inform the guest with a TRB on the `EventRing`, and if enabled, a hardware interrupt. - it moves the USB device to the `DeviceSlotTable` in preparation for being configured and assigned a slot. When the guest xHCD rings Doorbell 0 to run an `EnableSlot` Command, the `DeviceSlotTable` assigns the first unused slot ID to it. Hot-plugging devices live (i.e. not just attaching all devices defined by the instance spec at boot time as is done now) is not yet implemented. Device-slot-related Command TRBs are handled by the `DeviceSlotTable`. The command interface methods are written as translations of the behaviors defined in xHCI 1.2 section 4.6 to Rust, with liberties taken around redundant `TrbCompletionCode` writes; i.e. when the outlined behavior from the spec describes the xHC placing a `Success` into a new TRB on the `EventRing` immediately at the beginning of the command's execution, and then overwriting it with a failure code in the event of a failure, our implementation postpones the creation and enqueueing of the event until after the outcome of the command's execution (and thus the Event TRB's values) are all known. Ports ----- Root hub port state machines (xHCI 1.2 section 4.19.1) and port registers are managed by `Usb2Port`, which has separate methods for handling register writes by the guest and by the xHC itself. TRB Rings --------- **Consumer**: The `CommandRing` and each slot endpoint's `TransferRing` are implemented as `ConsumerRing` and `ConsumerRing`. Dequeued work items are converted from raw `CommandDescriptor`s and `TransferDescriptor`s, respectively). Starting at the dequeue pointer provided by the guest, the `ConsumerRing` will consume non-Link TRBs (and follow Link TRBs, as in xHCI 1.2 figure 4-15) into complete work items. In the case of the `CommandRing`, `CommandDescriptor`s are each only made up of one `Trb`, but for the `TransferRing` multi-TRB work items are possible, where all but the last item have the `chain_bit` set. **Producer**: The only type of producer ring is the `EventRing`. Events destined for it are fed through the `XhciInterrupter`, which handles enablement and rate-limiting of PCI-level machine interrupts being generated as a result of the events. Similarly (and inversely) to the consumer rings, the `EventRing` converts the `EventInfo`s enqueued in it into `EventDescriptor`s to be written to guest memory regions defined by the `EventRingSegment` Table. Doorbells --------- The guest writing to a `DoorbellRegister` makes the host controller process a consumer TRB ring (the `CommandRing` for doorbell 0, or the corresponding slot's `TransferRing` for nonzero doorbells). The ring consumption is performed by the doorbell register write handler, in `process_command_ring` and `process_transfer_ring`. Timer registers --------------- The value of registers defined as incrementing/decrementing per time interval, such as **MFINDEX** and the `XhciInterrupter`'s **IMODC**, are simulated with `VmGuestInstant`s and `Duration`s rather than by repeated incrementation. (`VmGuestInstant` is a new type defined in this change with a similar interface to `std::time::Instant`, but using timestamps provided by the VMM.) Migration --------- The types defined for serializing the device state are admittedly undercooked; advice about what a cleaner approach could look like before committing to the 'V1' of the format are quite welcome. DTrace support ============== To see a trace of all MMIO register reads/writes and TRB enqueue/dequeues: ```sh pfexec ./scripts/xhci-trace.d -p $(pgrep propolis-server) ``` The name of each register as used by DTrace is `&'static`ally defined in `registers::Registers::reg_name`. --- Cargo.lock | 1 + bin/propolis-server/src/lib/initializer.rs | 43 + .../src/lib/spec/api_spec_v0.rs | 33 +- bin/propolis-server/src/lib/spec/builder.rs | 43 +- bin/propolis-server/src/lib/spec/mod.rs | 4 +- bin/propolis-server/src/lib/vm/ensure.rs | 1 + bin/propolis-standalone/src/main.rs | 7 + .../src/instance_spec/components/devices.rs | 27 + .../src/instance_spec/v0.rs | 2 + crates/propolis-config-toml/src/lib.rs | 4 + crates/propolis-config-toml/src/spec.rs | 48 +- lib/propolis/Cargo.toml | 1 + lib/propolis/src/common.rs | 19 +- lib/propolis/src/hw/mod.rs | 1 + lib/propolis/src/hw/pci/bits.rs | 5 + lib/propolis/src/hw/pci/device.rs | 2 +- lib/propolis/src/hw/usb/mod.rs | 9 + .../src/hw/usb/usbdev/demo_state_tracker.rs | 269 ++++ lib/propolis/src/hw/usb/usbdev/descriptor.rs | 613 ++++++++ lib/propolis/src/hw/usb/usbdev/mod.rs | 57 + lib/propolis/src/hw/usb/usbdev/requests.rs | 177 +++ .../src/hw/usb/xhci/bits/device_context.rs | 527 +++++++ lib/propolis/src/hw/usb/xhci/bits/mod.rs | 1341 +++++++++++++++++ .../src/hw/usb/xhci/bits/ring_data.rs | 900 +++++++++++ lib/propolis/src/hw/usb/xhci/controller.rs | 1140 ++++++++++++++ lib/propolis/src/hw/usb/xhci/device_slots.rs | 1020 +++++++++++++ lib/propolis/src/hw/usb/xhci/interrupter.rs | 599 ++++++++ lib/propolis/src/hw/usb/xhci/mod.rs | 250 +++ lib/propolis/src/hw/usb/xhci/port.rs | 560 +++++++ lib/propolis/src/hw/usb/xhci/registers.rs | 369 +++++ .../src/hw/usb/xhci/rings/consumer/command.rs | 467 ++++++ .../hw/usb/xhci/rings/consumer/doorbell.rs | 181 +++ .../src/hw/usb/xhci/rings/consumer/mod.rs | 533 +++++++ .../hw/usb/xhci/rings/consumer/transfer.rs | 611 ++++++++ lib/propolis/src/hw/usb/xhci/rings/mod.rs | 6 + .../src/hw/usb/xhci/rings/producer/event.rs | 636 ++++++++ .../src/hw/usb/xhci/rings/producer/mod.rs | 6 + lib/propolis/src/util/regmap.rs | 9 +- lib/propolis/src/vmm/hdl.rs | 5 + lib/propolis/src/vmm/mem.rs | 10 + lib/propolis/src/vmm/time.rs | 43 + openapi/propolis-server.json | 81 + phd-tests/framework/src/test_vm/config.rs | 37 +- phd-tests/tests/src/lib.rs | 1 + phd-tests/tests/src/xhci.rs | 43 + scripts/xhci-trace.d | 161 ++ 46 files changed, 10893 insertions(+), 9 deletions(-) create mode 100644 lib/propolis/src/hw/usb/mod.rs create mode 100644 lib/propolis/src/hw/usb/usbdev/demo_state_tracker.rs create mode 100644 lib/propolis/src/hw/usb/usbdev/descriptor.rs create mode 100644 lib/propolis/src/hw/usb/usbdev/mod.rs create mode 100644 lib/propolis/src/hw/usb/usbdev/requests.rs create mode 100644 lib/propolis/src/hw/usb/xhci/bits/device_context.rs create mode 100644 lib/propolis/src/hw/usb/xhci/bits/mod.rs create mode 100644 lib/propolis/src/hw/usb/xhci/bits/ring_data.rs create mode 100644 lib/propolis/src/hw/usb/xhci/controller.rs create mode 100644 lib/propolis/src/hw/usb/xhci/device_slots.rs create mode 100644 lib/propolis/src/hw/usb/xhci/interrupter.rs create mode 100644 lib/propolis/src/hw/usb/xhci/mod.rs create mode 100644 lib/propolis/src/hw/usb/xhci/port.rs create mode 100644 lib/propolis/src/hw/usb/xhci/registers.rs create mode 100644 lib/propolis/src/hw/usb/xhci/rings/consumer/command.rs create mode 100644 lib/propolis/src/hw/usb/xhci/rings/consumer/doorbell.rs create mode 100644 lib/propolis/src/hw/usb/xhci/rings/consumer/mod.rs create mode 100644 lib/propolis/src/hw/usb/xhci/rings/consumer/transfer.rs create mode 100644 lib/propolis/src/hw/usb/xhci/rings/mod.rs create mode 100644 lib/propolis/src/hw/usb/xhci/rings/producer/event.rs create mode 100644 lib/propolis/src/hw/usb/xhci/rings/producer/mod.rs create mode 100644 phd-tests/tests/src/xhci.rs create mode 100755 scripts/xhci-trace.d diff --git a/Cargo.lock b/Cargo.lock index c3355b611..4372d8497 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4607,6 +4607,7 @@ dependencies = [ "bhyve_api 0.0.0", "bitflags 2.6.0", "bitstruct", + "bitvec", "byteorder", "cpuid_utils", "crossbeam-channel", diff --git a/bin/propolis-server/src/lib/initializer.rs b/bin/propolis-server/src/lib/initializer.rs index 2c6f084be..2d5d04588 100644 --- a/bin/propolis-server/src/lib/initializer.rs +++ b/bin/propolis-server/src/lib/initializer.rs @@ -42,6 +42,7 @@ use propolis::hw::qemu::{ ramfb, }; use propolis::hw::uart::LpcUart; +use propolis::hw::usb::xhci; use propolis::hw::{nvme, virtio}; use propolis::intr_pins; use propolis::vmm::{self, Builder, Machine}; @@ -98,6 +99,9 @@ pub enum MachineInitError { #[error("failed to specialize CPUID for vcpu {0}")] CpuidSpecializationFailed(i32, #[source] propolis::cpuid::SpecializeError), + #[error("xHC USB root hub port number invalid: {0}")] + UsbRootHubPortNumberInvalid(String), + #[cfg(feature = "falcon")] #[error("softnpu p9 device missing")] SoftNpuP9Missing, @@ -814,6 +818,45 @@ impl MachineInitializer<'_> { Ok(()) } + /// Initialize xHCI controllers, connect any USB devices given in the spec, + /// add them to the device map, and attach them to the chipset. + pub fn initialize_xhc_usb( + &mut self, + chipset: &RegisteredChipset, + ) -> Result<(), MachineInitError> { + for (xhc_id, xhc_spec) in &self.spec.xhcs { + info!( + self.log, + "Creating xHCI controller"; + "pci_path" => %xhc_spec.pci_path, + ); + + let log = self.log.new(slog::o!("dev" => "xhci")); + let bdf: pci::Bdf = xhc_spec.pci_path.into(); + let xhc = xhci::PciXhci::create(self.machine.hdl.clone(), log); + + for (usb_id, usb) in &self.spec.usbdevs { + if *xhc_id == usb.xhc_device { + info!( + self.log, + "Attaching USB device"; + "usb_id" => %usb_id, + "xhc_pci_path" => %xhc_spec.pci_path, + "usb_port" => %usb.root_hub_port_num, + ); + xhc.add_usb_device(usb.root_hub_port_num).map_err( + MachineInitError::UsbRootHubPortNumberInvalid, + )?; + } + } + + self.devices.insert(xhc_id.clone(), xhc.clone()); + chipset.pci_attach(bdf, xhc); + } + + Ok(()) + } + #[cfg(feature = "failure-injection")] pub fn initialize_test_devices(&mut self) { use propolis::hw::testdev::{ diff --git a/bin/propolis-server/src/lib/spec/api_spec_v0.rs b/bin/propolis-server/src/lib/spec/api_spec_v0.rs index 18011c97c..3c75da7b0 100644 --- a/bin/propolis-server/src/lib/spec/api_spec_v0.rs +++ b/bin/propolis-server/src/lib/spec/api_spec_v0.rs @@ -5,7 +5,7 @@ //! Conversions from version-0 instance specs in the [`propolis_api_types`] //! crate to the internal [`super::Spec`] representation. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; use propolis_api_types::instance_spec::{ components::{ @@ -44,6 +44,9 @@ pub(crate) enum ApiSpecError { #[error("network backend {backend} not found for device {device}")] NetworkBackendNotFound { backend: SpecKey, device: SpecKey }, + #[error("USB host controller {xhc} not found for device {device}")] + HostControllerNotFound { xhc: SpecKey, device: SpecKey }, + #[allow(dead_code)] #[error("support for component {component} compiled out via {feature}")] FeatureCompiledOut { component: SpecKey, feature: &'static str }, @@ -61,6 +64,8 @@ impl From for InstanceSpecV0 { cpuid, disks, nics, + xhcs, + usbdevs, boot_settings, serial, pci_pci_bridges, @@ -121,6 +126,14 @@ impl From for InstanceSpecV0 { ); } + for (id, xhc) in xhcs { + insert_component(&mut spec, id, ComponentV0::Xhci(xhc)); + } + + for (id, usb) in usbdevs { + insert_component(&mut spec, id, ComponentV0::UsbPlaceholder(usb)); + } + for (name, desc) in serial { if desc.device == SerialPortDevice::Uart { insert_component( @@ -230,6 +243,7 @@ impl TryFrom for Spec { BTreeMap::new(); let mut dlpi_backends: BTreeMap = BTreeMap::new(); + let mut xhci_controllers: BTreeSet = BTreeSet::new(); for (id, component) in value.components.into_iter() { match component { @@ -249,6 +263,10 @@ impl TryFrom for Spec { ComponentV0::DlpiNetworkBackend(dlpi) => { dlpi_backends.insert(id, dlpi); } + ComponentV0::Xhci(xhc) => { + xhci_controllers.insert(id.to_owned()); + builder.add_xhci_controller(id, xhc)?; + } device => { devices.push((id, device)); } @@ -365,6 +383,19 @@ impl TryFrom for Spec { ComponentV0::P9fs(p9fs) => { builder.set_p9fs(p9fs)?; } + ComponentV0::Xhci(xhci) => { + builder.add_xhci_controller(device_id, xhci)?; + } + ComponentV0::UsbPlaceholder(usbdev) => { + if xhci_controllers.contains(&usbdev.xhc_device) { + builder.add_usb_device(device_id, usbdev)?; + } else { + return Err(ApiSpecError::HostControllerNotFound { + xhc: usbdev.xhc_device.to_owned(), + device: device_id, + }); + } + } ComponentV0::CrucibleStorageBackend(_) | ComponentV0::FileStorageBackend(_) | ComponentV0::BlobStorageBackend(_) diff --git a/bin/propolis-server/src/lib/spec/builder.rs b/bin/propolis-server/src/lib/spec/builder.rs index ddd5c5e5b..809837aed 100644 --- a/bin/propolis-server/src/lib/spec/builder.rs +++ b/bin/propolis-server/src/lib/spec/builder.rs @@ -9,7 +9,7 @@ use std::collections::{BTreeSet, HashSet}; use propolis_api_types::instance_spec::{ components::{ board::Board as InstanceSpecBoard, - devices::{PciPciBridge, SerialPortNumber}, + devices::{PciPciBridge, SerialPortNumber, UsbDevice, XhciController}, }, PciPath, SpecKey, }; @@ -65,6 +65,9 @@ pub(crate) enum SpecBuilderError { #[error("failed to read default CPUID settings from the host")] DefaultCpuidReadFailed(#[from] cpuid_utils::host::GetHostCpuidError), + + #[error("a USB device is already attached to xHC {0} root hub port {1}")] + UsbPortInUse(SpecKey, u8), } #[derive(Debug, Default)] @@ -72,6 +75,7 @@ pub(crate) struct SpecBuilder { spec: super::Spec, pci_paths: BTreeSet, serial_ports: HashSet, + xhc_usb_ports: BTreeSet<(SpecKey, u8)>, component_names: BTreeSet, } @@ -154,6 +158,23 @@ impl SpecBuilder { } } + fn register_usb_device( + &mut self, + usbdev: &UsbDevice, + ) -> Result<(), SpecBuilderError> { + // slightly awkward: we have to take a ref of an owned tuple for + // .contains() below, and in either case we need an owned SpecKey, + // so we'll just clone it once here + let xhc_and_port = + (usbdev.xhc_device.to_owned(), usbdev.root_hub_port_num); + if self.xhc_usb_ports.contains(&xhc_and_port) { + Err(SpecBuilderError::UsbPortInUse(xhc_and_port.0, xhc_and_port.1)) + } else { + self.xhc_usb_ports.insert(xhc_and_port); + Ok(()) + } + } + /// Adds a storage device with an associated backend. pub(super) fn add_storage_device( &mut self, @@ -355,6 +376,26 @@ impl SpecBuilder { Ok(self) } + pub fn add_xhci_controller( + &mut self, + device_id: SpecKey, + xhc: XhciController, + ) -> Result<&Self, SpecBuilderError> { + self.register_pci_device(xhc.pci_path)?; + self.spec.xhcs.insert(device_id, xhc); + Ok(self) + } + + pub fn add_usb_device( + &mut self, + device_id: SpecKey, + usbdev: UsbDevice, + ) -> Result<&Self, SpecBuilderError> { + self.register_usb_device(&usbdev)?; + self.spec.usbdevs.insert(device_id, usbdev); + Ok(self) + } + /// Yields the completed spec, consuming the builder. pub fn finish(self) -> super::Spec { self.spec diff --git a/bin/propolis-server/src/lib/spec/mod.rs b/bin/propolis-server/src/lib/spec/mod.rs index 5bf7f8836..ae67eb635 100644 --- a/bin/propolis-server/src/lib/spec/mod.rs +++ b/bin/propolis-server/src/lib/spec/mod.rs @@ -26,7 +26,7 @@ use propolis_api_types::instance_spec::{ board::{Chipset, GuestHypervisorInterface, I440Fx}, devices::{ NvmeDisk, PciPciBridge, QemuPvpanic as QemuPvpanicDesc, - SerialPortNumber, VirtioDisk, VirtioNic, + SerialPortNumber, UsbDevice, VirtioDisk, VirtioNic, XhciController, }, }, v0::ComponentV0, @@ -66,6 +66,8 @@ pub(crate) struct Spec { pub cpuid: CpuidSet, pub disks: BTreeMap, pub nics: BTreeMap, + pub xhcs: BTreeMap, + pub usbdevs: BTreeMap, pub boot_settings: Option, pub serial: BTreeMap, diff --git a/bin/propolis-server/src/lib/vm/ensure.rs b/bin/propolis-server/src/lib/vm/ensure.rs index 20bc39ee5..afaa77bc4 100644 --- a/bin/propolis-server/src/lib/vm/ensure.rs +++ b/bin/propolis-server/src/lib/vm/ensure.rs @@ -563,6 +563,7 @@ async fn initialize_vm_objects( &properties, ))?; init.initialize_network_devices(&chipset).await?; + init.initialize_xhc_usb(&chipset)?; #[cfg(feature = "failure-injection")] init.initialize_test_devices(); diff --git a/bin/propolis-standalone/src/main.rs b/bin/propolis-standalone/src/main.rs index f3f52cad0..792340e42 100644 --- a/bin/propolis-standalone/src/main.rs +++ b/bin/propolis-standalone/src/main.rs @@ -1274,6 +1274,13 @@ fn setup_instance( block::attach(nvme.clone(), backend).unwrap(); chipset_pci_attach(bdf, nvme); } + "pci-xhci" => { + let log = log.new(slog::o!("dev" => "xhci")); + let bdf = bdf.unwrap(); + let xhci = hw::usb::xhci::PciXhci::create(hdl.clone(), log); + guard.inventory.register_instance(&xhci, &bdf.to_string()); + chipset_pci_attach(bdf, xhci); + } qemu::pvpanic::DEVICE_NAME => { let enable_isa = dev .options diff --git a/crates/propolis-api-types/src/instance_spec/components/devices.rs b/crates/propolis-api-types/src/instance_spec/components/devices.rs index 7bb2d9fc5..1a589ccdb 100644 --- a/crates/propolis-api-types/src/instance_spec/components/devices.rs +++ b/crates/propolis-api-types/src/instance_spec/components/devices.rs @@ -190,6 +190,33 @@ pub struct P9fs { pub pci_path: PciPath, } +/// Describes a PCI device implementing the eXtensible Host Controller Interface +/// for the purpose of attaching USB devices. +/// +/// (Note that at present no functional USB devices have yet been implemented.) +#[derive(Clone, Deserialize, Serialize, Debug, JsonSchema)] +#[serde(deny_unknown_fields)] +pub struct XhciController { + /// The PCI path at which to attach the guest to this xHC. + pub pci_path: PciPath, +} + +/// Describes a USB device, requires the presence of an XhciController. +/// +/// (Note that at present no USB devices have yet been implemented +/// outside of a null device for testing purposes.) +#[derive(Clone, Deserialize, Serialize, Debug, JsonSchema)] +#[serde(deny_unknown_fields)] +pub struct UsbDevice { + /// The name of the xHC to which this USB device shall be attached. + pub xhc_device: SpecKey, + /// The root hub port number to which this USB device shall be attached. + /// For USB 2.0 devices, valid values are 1-4, inclusive. + /// For USB 3.0 devices, valid values are 5-8, inclusive. + pub root_hub_port_num: u8, + // TODO(lif): a field for device type (e.g. HID tablet, mass storage...) +} + /// Describes a synthetic device that registers for VM lifecycle notifications /// and returns errors during attempts to migrate. /// diff --git a/crates/propolis-api-types/src/instance_spec/v0.rs b/crates/propolis-api-types/src/instance_spec/v0.rs index 8ebe45f8f..31ddfb64d 100644 --- a/crates/propolis-api-types/src/instance_spec/v0.rs +++ b/crates/propolis-api-types/src/instance_spec/v0.rs @@ -27,6 +27,8 @@ pub enum ComponentV0 { SoftNpuPort(components::devices::SoftNpuPort), SoftNpuP9(components::devices::SoftNpuP9), P9fs(components::devices::P9fs), + Xhci(components::devices::XhciController), + UsbPlaceholder(components::devices::UsbDevice), MigrationFailureInjector(components::devices::MigrationFailureInjector), CrucibleStorageBackend(components::backends::CrucibleStorageBackend), FileStorageBackend(components::backends::FileStorageBackend), diff --git a/crates/propolis-config-toml/src/lib.rs b/crates/propolis-config-toml/src/lib.rs index 43189a934..9acb7f6f1 100644 --- a/crates/propolis-config-toml/src/lib.rs +++ b/crates/propolis-config-toml/src/lib.rs @@ -97,6 +97,10 @@ impl Device { pub fn get>(&self, key: S) -> Option { self.get_string(key)?.parse().ok() } + + pub fn get_integer>(&self, key: S) -> Option { + self.options.get(key.as_ref())?.as_integer() + } } #[derive(Debug, Deserialize, Serialize, PartialEq)] diff --git a/crates/propolis-config-toml/src/spec.rs b/crates/propolis-config-toml/src/spec.rs index a4306c7d1..d1ad7ba7f 100644 --- a/crates/propolis-config-toml/src/spec.rs +++ b/crates/propolis-config-toml/src/spec.rs @@ -13,8 +13,8 @@ use propolis_client::{ instance_spec::{ ComponentV0, DlpiNetworkBackend, FileStorageBackend, MigrationFailureInjector, NvmeDisk, P9fs, PciPath, PciPciBridge, - SoftNpuP9, SoftNpuPciPort, SoftNpuPort, SpecKey, VirtioDisk, - VirtioNetworkBackend, VirtioNic, + SoftNpuP9, SoftNpuPciPort, SoftNpuPort, SpecKey, UsbDevice, VirtioDisk, + VirtioNetworkBackend, VirtioNic, XhciController, }, support::nvme_serial_from_str, }; @@ -33,6 +33,12 @@ pub enum TomlToSpecError { #[error("failed to get PCI path for device {0:?}")] InvalidPciPath(String), + #[error("failed to get USB root hub port for device {0:?}")] + InvalidUsbPort(String), + + #[error("no xHC name for USB device {0:?}")] + NoHostControllerNameForUsbDevice(String), + #[error("failed to parse PCI path string {0:?}")] PciPathParseFailed(String, #[source] std::io::Error), @@ -249,6 +255,44 @@ impl TryFrom<&super::Config> for SpecConfig { )?), )?; } + "pci-xhci" => { + let pci_path: PciPath = + device.get("pci-path").ok_or_else(|| { + TomlToSpecError::InvalidPciPath( + device_name.to_owned(), + ) + })?; + + spec.components.insert( + device_id, + ComponentV0::Xhci(XhciController { pci_path }), + ); + } + "usb-dummy" => { + let root_hub_port_num = device + .get_integer("root-hub-port") + .filter(|x| (1..=8).contains(x)) + .ok_or_else(|| { + TomlToSpecError::InvalidUsbPort( + device_name.to_owned(), + ) + })? as u8; + + let xhc_device: SpecKey = + device.get("xhc-device").ok_or_else(|| { + TomlToSpecError::NoHostControllerNameForUsbDevice( + device_name.to_owned(), + ) + })?; + + spec.components.insert( + device_id, + ComponentV0::UsbPlaceholder(UsbDevice { + root_hub_port_num, + xhc_device, + }), + ); + } _ => { return Err(TomlToSpecError::UnrecognizedDeviceType( driver.to_owned(), diff --git a/lib/propolis/Cargo.toml b/lib/propolis/Cargo.toml index 88a04558a..5bada4b1f 100644 --- a/lib/propolis/Cargo.toml +++ b/lib/propolis/Cargo.toml @@ -9,6 +9,7 @@ rust-version = "1.70" libc.workspace = true bitflags.workspace = true bitstruct.workspace = true +bitvec.workspace = true byteorder.workspace = true lazy_static.workspace = true thiserror.workspace = true diff --git a/lib/propolis/src/common.rs b/lib/propolis/src/common.rs index 7d118d611..768b4e99d 100644 --- a/lib/propolis/src/common.rs +++ b/lib/propolis/src/common.rs @@ -7,6 +7,8 @@ use std::ops::{Bound::*, RangeBounds}; use std::slice::SliceIndex; use std::sync::atomic::{AtomicBool, Ordering}; +use zerocopy::{FromBytes, Immutable}; + use crate::vmm::SubMapping; /// A vCPU number. @@ -170,6 +172,7 @@ fn numeric_bounds( } } +#[derive(Debug)] enum ROInner<'a> { Buf(&'a mut [u8]), Map(SubMapping<'a>), @@ -178,6 +181,7 @@ enum ROInner<'a> { /// Represents an abstract requested read operation. /// /// Exposes an API with various "write" methods, which fulfill the request. +#[derive(Debug)] pub struct ReadOp<'a> { inner: ROInner<'a>, offset: usize, @@ -310,6 +314,7 @@ impl<'a> ReadOp<'a> { } } +#[derive(Debug)] enum WOInner<'a> { Buf(&'a [u8]), Map(SubMapping<'a>), @@ -318,6 +323,7 @@ enum WOInner<'a> { /// Represents an abstract requested write operation. /// /// Exposes an API with various "read" methods, which fulfill the request. +#[derive(Debug)] pub struct WriteOp<'a> { inner: WOInner<'a>, offset: usize, @@ -467,7 +473,18 @@ impl RWOp<'_, '_> { } /// An address within a guest VM. -#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)] +#[derive( + Copy, + Clone, + Debug, + Eq, + PartialEq, + Ord, + PartialOrd, + Hash, + FromBytes, + Immutable, +)] pub struct GuestAddr(pub u64); impl GuestAddr { diff --git a/lib/propolis/src/hw/mod.rs b/lib/propolis/src/hw/mod.rs index 472a23e4a..e7bcfb1d3 100644 --- a/lib/propolis/src/hw/mod.rs +++ b/lib/propolis/src/hw/mod.rs @@ -12,4 +12,5 @@ pub mod ps2; pub mod qemu; pub mod testdev; pub mod uart; +pub mod usb; pub mod virtio; diff --git a/lib/propolis/src/hw/pci/bits.rs b/lib/propolis/src/hw/pci/bits.rs index e0b6f5c24..c19145a58 100644 --- a/lib/propolis/src/hw/pci/bits.rs +++ b/lib/propolis/src/hw/pci/bits.rs @@ -53,6 +53,7 @@ pub const CLASS_DISPLAY: u8 = 3; pub const CLASS_MULTIMEDIA: u8 = 4; pub const CLASS_MEMORY: u8 = 5; pub const CLASS_BRIDGE: u8 = 6; +pub const CLASS_SERIAL_BUS: u8 = 0xC; // Sub-classes under CLASS_STORAGE pub const SUBCLASS_STORAGE_NVM: u8 = 8; @@ -66,8 +67,12 @@ pub const HEADER_TYPE_DEVICE: u8 = 0b0; pub const HEADER_TYPE_BRIDGE: u8 = 0b1; pub const HEADER_TYPE_MULTIFUNC: u8 = 0b1000_0000; +pub const SUBCLASS_USB: u8 = 3; +pub const SUBCLASS_NVM: u8 = 8; + // Programming Interfaces for SUBCLASS_STORAGE_NVM pub const PROGIF_ENTERPRISE_NVME: u8 = 2; +pub const PROGIF_USB3: u8 = 0x30; pub(super) const MASK_FUNC: u8 = 0x07; pub(super) const MASK_DEV: u8 = 0x1f; diff --git a/lib/propolis/src/hw/pci/device.rs b/lib/propolis/src/hw/pci/device.rs index 4907aab70..0ca66a661 100644 --- a/lib/propolis/src/hw/pci/device.rs +++ b/lib/propolis/src/hw/pci/device.rs @@ -655,7 +655,7 @@ impl MigrateMulti for DeviceState { } } -#[derive(Copy, Clone, Eq, PartialEq)] +#[derive(Copy, Clone, Eq, PartialEq, Debug)] pub enum IntrMode { Disabled, INTxPin, diff --git a/lib/propolis/src/hw/usb/mod.rs b/lib/propolis/src/hw/usb/mod.rs new file mode 100644 index 000000000..c8bf7f9b1 --- /dev/null +++ b/lib/propolis/src/hw/usb/mod.rs @@ -0,0 +1,9 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +//! USB Emulation + +pub mod xhci; + +pub mod usbdev; diff --git a/lib/propolis/src/hw/usb/usbdev/demo_state_tracker.rs b/lib/propolis/src/hw/usb/usbdev/demo_state_tracker.rs new file mode 100644 index 000000000..294c4d98a --- /dev/null +++ b/lib/propolis/src/hw/usb/usbdev/demo_state_tracker.rs @@ -0,0 +1,269 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use crate::{ + common::{GuestData, GuestRegion}, + hw::usb::xhci::rings::consumer::transfer::PointerOrImmediate, + vmm::MemCtx, +}; + +use super::{ + descriptor::*, + probes, + requests::{Request, RequestDirection, SetupData, StandardRequest}, + {Error, Result}, +}; + +/// This is a hard-coded faux-device that purely exists to test the xHCI implementation. +#[derive(Default)] +pub struct NullUsbDevice { + current_setup: Option, + payload: Vec, + bytes_transferred: usize, +} + +impl NullUsbDevice { + const MANUFACTURER_NAME_INDEX: StringIndex = StringIndex(0); + const PRODUCT_NAME_INDEX: StringIndex = StringIndex(1); + const SERIAL_INDEX: StringIndex = StringIndex(2); + const CONFIG_NAME_INDEX: StringIndex = StringIndex(3); + const INTERFACE_NAME_INDEX: StringIndex = StringIndex(4); + + fn device_descriptor() -> DeviceDescriptor { + DeviceDescriptor { + usb_version: USB_VER_2_0, + device_class: ClassCode(0), + device_subclass: SubclassCode(0), + device_protocol: ProtocolCode(0), + max_packet_size_0: MaxSizeZeroEP::_64, + vendor_id: VendorId(0x1de), + product_id: ProductId(0xdead), + device_version: Bcd16(0), + manufacturer_name: Self::MANUFACTURER_NAME_INDEX, + product_name: Self::PRODUCT_NAME_INDEX, + serial: Self::SERIAL_INDEX, + configurations: vec![Self::config_descriptor()], + specific_augmentations: vec![], + } + } + fn config_descriptor() -> ConfigurationDescriptor { + ConfigurationDescriptor { + interfaces: vec![Self::interface_descriptor()], + config_value: ConfigurationValue(0), + configuration_name: Self::CONFIG_NAME_INDEX, + attributes: ConfigurationAttributes::default(), + specific_augmentations: vec![], + } + } + fn interface_descriptor() -> InterfaceDescriptor { + InterfaceDescriptor { + interface_num: 0, + alternate_setting: 0, + endpoints: vec![Self::endpoint_descriptor()], + class: InterfaceClass(0), + subclass: InterfaceSubclass(0), + protocol: InterfaceProtocol(0), + interface_name: Self::INTERFACE_NAME_INDEX, + specific_augmentations: vec![], + } + } + fn endpoint_descriptor() -> EndpointDescriptor { + EndpointDescriptor { + endpoint_addr: 0, + attributes: EndpointAttributes::default(), + max_packet_size: 64, + interval: 1, + specific_augmentations: vec![], + } + } + fn string_descriptor(idx: u8) -> StringDescriptor { + let s: &str = match StringIndex(idx) { + Self::MANUFACTURER_NAME_INDEX => "Oxide Computer Company", + Self::PRODUCT_NAME_INDEX => "Generic USB 2.0 Encabulator", + Self::SERIAL_INDEX => "9001", + Self::CONFIG_NAME_INDEX => "MyCoolConfiguration", + Self::INTERFACE_NAME_INDEX => "MyNotQuiteAsCoolInterface", + _ => "weird index but ok", + }; + StringDescriptor { string: s.to_string() } + } + fn device_qualifier_descriptor() -> DeviceQualifierDescriptor { + DeviceQualifierDescriptor { + usb_version: USB_VER_2_0, + device_class: ClassCode(0), + device_subclass: SubclassCode(0), + device_protocol: ProtocolCode(0), + max_packet_size_0: MaxSizeZeroEP::_64, + num_configurations: 0, + } + } + + pub fn setup_stage(&mut self, req: SetupData) -> Result<()> { + match req.direction() { + RequestDirection::DeviceToHost => { + self.payload = self.payload_for(&req)?; + } + RequestDirection::HostToDevice => { + self.payload.clear(); + } + } + self.bytes_transferred = 0; + self.current_setup = Some(req); + Ok(()) + } + + pub fn data_stage( + &mut self, + data_buffer: PointerOrImmediate, + data_direction: RequestDirection, + memctx: &MemCtx, + ) -> Result { + if let Some(setup_data) = self.current_setup.as_ref() { + if data_direction != setup_data.direction() { + return Err(Error::SetupVsDataDirectionMismatch( + setup_data.direction(), + data_direction, + )); + } + let count = match setup_data.direction() { + RequestDirection::DeviceToHost => { + let PointerOrImmediate::Pointer(region) = data_buffer + else { + return Err(Error::ImmediateParameterForOutDataStage); + }; + memctx + .write_from( + region.0, + &self.payload[self.bytes_transferred..], + region.1, + ) + .ok_or(Error::DataStageWriteFailed)? + } + RequestDirection::HostToDevice => match data_buffer { + PointerOrImmediate::Pointer(GuestRegion(ptr, len)) => { + self.payload.resize(self.bytes_transferred + len, 0u8); + memctx + .read_into( + ptr, + &mut GuestData::from( + &mut self.payload[self.bytes_transferred..], + ), + len, + ) + .ok_or(Error::DataStageReadFailed)? + } + PointerOrImmediate::Immediate(arr, len) => { + self.payload.extend_from_slice(&arr[..len]); + len + } + }, + }; + self.bytes_transferred += count; + Ok(count) + } else { + Err(Error::NoSetupStageBefore("Data Stage")) + } + } + + pub fn status_stage( + &mut self, + status_direction: RequestDirection, + ) -> Result<()> { + if let Some(setup) = self.current_setup.take() { + if status_direction == setup.direction() { + return Err(Error::SetupVsStatusDirectionMatch( + status_direction, + )); + } + + let result = match setup.direction() { + RequestDirection::HostToDevice => match setup.request() { + Request::Standard(StandardRequest::SetConfiguration) => { + self.set_configuration() + } + x => Err(Error::UnimplementedRequest(x)), + }, + RequestDirection::DeviceToHost => Ok(()), + }; + + self.payload.clear(); + self.bytes_transferred = 0; + result + } else { + Err(Error::NoSetupStageBefore("Status Stage")) + } + } + + fn set_configuration(&mut self) -> Result<()> { + if self.payload.is_empty() { + Ok(()) + } else { + Err(Error::InvalidPayloadForRequest( + Request::Standard(StandardRequest::SetConfiguration), + self.payload.clone(), + )) + } + } + + fn payload_for(&self, setup_data: &SetupData) -> Result> { + match setup_data.request() { + Request::Standard(StandardRequest::GetDescriptor) => { + let [desc, idx] = setup_data.value().to_be_bytes(); + let descriptor: Box = + match DescriptorType::from_repr(desc) { + Some(DescriptorType::Device) => { + Box::new(Self::device_descriptor()) + } + Some(DescriptorType::Configuration) => { + Box::new(Self::config_descriptor()) + } + Some(DescriptorType::String) => { + Box::new(Self::string_descriptor(idx)) + } + Some(DescriptorType::DeviceQualifier) => { + Box::new(Self::device_qualifier_descriptor()) + } + Some(x) => { + return Err(Error::UnimplementedDescriptor(x)) + } + None => return Err(Error::UnknownDescriptorType(desc)), + }; + probes::usb_get_descriptor!(|| (desc, idx)); + // slog::debug!(log, "usb: GET_DESCRIPTOR({descriptor:?})"); + Ok(descriptor.serialize().collect()) + } + Request::Standard(StandardRequest::GetStatus) => { + // USB 2.0 sect 9.4.5 - two-byte response where lowest-order + // bits are 'self powered' and 'remote wakeup' + let attrib = Self::config_descriptor().attributes; + Ok((attrib.self_powered() as u16 + | (attrib.remote_wakeup() as u16 * 2)) + .to_le_bytes() + .to_vec()) + } + x => Err(Error::UnimplementedRequest(x)), + } + } + + pub fn import( + &mut self, + value: &super::migrate::UsbDeviceV1, + ) -> core::result::Result<(), crate::migrate::MigrateStateError> { + let super::migrate::UsbDeviceV1 { device_type, current_setup } = value; + if *device_type != super::migrate::UsbDeviceTypeV1::Null { + return Err(crate::migrate::MigrateStateError::ImportFailed( + format!("USB device type mismatch {device_type:?} != Null"), + )); + } + self.current_setup = current_setup.map(|x| SetupData(x)); + Ok(()) + } + + pub fn export(&self) -> super::migrate::UsbDeviceV1 { + super::migrate::UsbDeviceV1 { + device_type: super::migrate::UsbDeviceTypeV1::Null, + current_setup: self.current_setup.as_ref().map(|x| x.0), + } + } +} diff --git a/lib/propolis/src/hw/usb/usbdev/descriptor.rs b/lib/propolis/src/hw/usb/usbdev/descriptor.rs new file mode 100644 index 000000000..e216d8e9c --- /dev/null +++ b/lib/propolis/src/hw/usb/usbdev/descriptor.rs @@ -0,0 +1,613 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use bitstruct::bitstruct; +use strum::FromRepr; + +#[repr(transparent)] +pub struct Bcd16(pub u16); +impl core::fmt::Debug for Bcd16 { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "Bcd16({:#x})", self.0) + } +} + +pub const USB_VER_2_0: Bcd16 = Bcd16(0x200); + +#[repr(transparent)] +#[derive(Debug)] +pub struct ClassCode(pub u8); +#[repr(transparent)] +#[derive(Debug)] +pub struct SubclassCode(pub u8); +#[repr(transparent)] +#[derive(Debug)] +pub struct ProtocolCode(pub u8); + +#[repr(u8)] +#[derive(Copy, Clone, Debug)] +pub enum MaxSizeZeroEP { + _8 = 8, + _16 = 16, + _32 = 32, + _64 = 64, +} + +#[repr(transparent)] +#[derive(Debug)] +pub struct VendorId(pub u16); +#[repr(transparent)] +#[derive(Debug)] +pub struct ProductId(pub u16); +#[repr(transparent)] +#[derive(Debug, PartialEq)] +pub struct StringIndex(pub u8); +#[repr(transparent)] +#[derive(Debug)] +pub struct ConfigurationValue(pub u8); + +bitstruct! { + #[derive(Debug)] + pub struct ConfigurationAttributes(pub u8) { + reserved: u8 = 0..5; + pub remote_wakeup: bool = 5; + pub self_powered: bool = 6; + /// Reserved, but set to 1 + pub one: bool = 7; + } +} +impl Default for ConfigurationAttributes { + fn default() -> Self { + Self(0).with_one(true) + } +} + +bitstruct! { + #[derive(Default, Debug)] + pub struct EndpointAttributes(pub u8) { + /// control, isoch, bulk, interrupt. TODO: enum + pub transfer_type: u8 = 0..2; + pub isoch_synch_type: u8 = 2..4; + pub isoch_usage_type: u8 = 4..6; + reserved: u8 = 6..8; + } +} + +/// USB 2.0 table 9-5 +#[repr(u8)] +#[derive(FromRepr, Debug)] +pub enum DescriptorType { + Device = 1, + Configuration = 2, + String = 3, + Interface = 4, + Endpoint = 5, + DeviceQualifier = 6, + OtherSpeedConfiguration = 7, + InterfacePower = 8, + // OTG and Embedded Host Supplement v1.1a + OnTheGo = 9, + Debug = 10, + InterfaceAssociation = 11, + HID = 33, +} + +#[repr(u8)] +#[derive(FromRepr, Debug)] +pub enum CountryCode { + // TODO + International = 13, + UnitedStates = 33, +} + +#[derive(Debug, Copy, Clone)] +pub enum LanguageId { + Known(KnownLanguageId), + Other(u16), +} +impl From<&LanguageId> for u16 { + fn from(value: &LanguageId) -> Self { + match value { + LanguageId::Known(langid) => *langid as u16, + LanguageId::Other(x) => *x, + } + } +} +impl From<&u16> for LanguageId { + fn from(value: &u16) -> Self { + KnownLanguageId::from_repr(*value) + .map(Self::Known) + .unwrap_or(Self::Other(*value)) + } +} + +#[repr(u16)] +#[derive(FromRepr, Copy, Clone, Debug)] +pub enum KnownLanguageId { + EnglishUS = 0x0409, + HIDUsageDataDescriptor = 0x04ff, + HIDVendorDefined1 = 0xf0ff, + HIDVendorDefined2 = 0xf4ff, + HIDVendorDefined3 = 0xf8ff, + HIDVendorDefined4 = 0xfcff, +} + +#[repr(transparent)] +#[derive(Debug)] +pub struct InterfaceClass(pub u8); +#[repr(transparent)] +#[derive(Debug)] +pub struct InterfaceSubclass(pub u8); +#[repr(transparent)] +#[derive(Debug)] +pub struct InterfaceProtocol(pub u8); + +pub trait Descriptor: core::fmt::Debug { + /// bLength. Size of serialized descriptor in bytes. + fn length(&self) -> u8; + + fn descriptor_type(&self) -> DescriptorType; + + fn header(&self) -> [u8; 2] { + [self.length(), self.descriptor_type() as u8] + } + + fn serialize(&self) -> Box + '_>; +} + +/// Used in Configuration Descriptor's computation of wTotalLength to give +/// the size of all the descriptors provided when GET_DESCRIPTOR(Configuration) +/// is requested, and to follow Configuration Descriptor's own serialization +/// with their own payloads. +pub trait NestedDescriptor: Descriptor { + fn total_length(&self) -> u16; + fn serialize_all(&self) -> Box + '_>; +} + +/// Device Descriptor. +#[derive(Debug)] +pub struct DeviceDescriptor { + /// bcdUSB. USB version in binary-coded decimal. + pub usb_version: Bcd16, + /// bDeviceClass. + pub device_class: ClassCode, + /// bDeviceSubClass. + pub device_subclass: SubclassCode, + /// bDeviceProtocol. + pub device_protocol: ProtocolCode, + /// bMaxPacketSize0. + pub max_packet_size_0: MaxSizeZeroEP, + /// idVendor. + pub vendor_id: VendorId, + /// idProduct. + pub product_id: ProductId, + /// bcdDevice. + pub device_version: Bcd16, + /// iManufacturer. + pub manufacturer_name: StringIndex, + /// iProduct. + pub product_name: StringIndex, + /// iSerial. + pub serial: StringIndex, + + /// bNumConfigurations (u8) is the length of: + pub configurations: Vec, + + /// Descriptors of class-specific or vendor-specific augmentations. + pub specific_augmentations: Vec, +} + +impl Descriptor for DeviceDescriptor { + /// bLength is 18 for Device Descriptor. + fn length(&self) -> u8 { + 18 + } + + /// bDescriptorType is 1 for Device Descriptor. + fn descriptor_type(&self) -> DescriptorType { + DescriptorType::Device + } + + /// USB 2.0 table 9-8 + fn serialize(&self) -> Box + '_> { + Box::new( + self.header() + .into_iter() // 0, 1 + .chain(self.usb_version.0.to_le_bytes()) // 2-3 + .chain([ + self.device_class.0, // 4 + self.device_subclass.0, // 5 + self.device_protocol.0, // 6 + self.max_packet_size_0 as u8, // 7 + ]) + .chain(self.vendor_id.0.to_le_bytes()) // 8-9 + .chain(self.product_id.0.to_le_bytes()) // 10-11 + .chain(self.device_version.0.to_le_bytes()) // 12-13 + .chain([ + self.manufacturer_name.0, // 14 + self.product_name.0, // 15 + self.serial.0, // 16 + self.configurations.len() as u8, // 17 + ]), + ) + } +} + +#[derive(Debug)] +pub struct ConfigurationDescriptor { + /// wTotalLength (u16) is calculated based on serialization of, + /// and bNumInterfaces (u8) is the length of: + pub interfaces: Vec, + + /// bConfigurationValue. + pub config_value: ConfigurationValue, + + /// iConfiguration. + pub configuration_name: StringIndex, + + /// bmAttributes. + pub attributes: ConfigurationAttributes, + + /// Descriptors of class-specific or vendor-specific augmentations. + pub specific_augmentations: Vec, +} + +impl Descriptor for ConfigurationDescriptor { + /// bLength is 9 for Configuration Descriptor. + /// (The combined length of other descriptors provided alongside it + /// are given in wTotalLength) + fn length(&self) -> u8 { + 9 + } + + /// bDescriptorType. 2 for Configuration Descriptor. + fn descriptor_type(&self) -> DescriptorType { + DescriptorType::Configuration + } + + /// USB 2.0 table 9-8 + fn serialize(&self) -> Box + '_> { + // wTotalLength. Total length of all data returned when requesting + // this descriptor, including interface and endpoint descriptors + // and descriptors of class- and vendor- augmentations (e.g. HID). + let total_length = self.length() as u16 + + self + .interfaces + .iter() + .map(NestedDescriptor::total_length) + .sum::(); + Box::new( + self.header() + .into_iter() // 0, 1 + .chain(total_length.to_le_bytes()) // 2-3 + .chain([ + self.interfaces.len() as u8, // 4 + self.config_value.0, // 5 + self.configuration_name.0, // 6 + self.attributes.0, // 7 + 0, // 8. max power in 2mA units, hardcoding to 0 + ]) + .chain( + self.interfaces + .iter() + .flat_map(NestedDescriptor::serialize_all), + ), + ) + } +} + +#[derive(Debug)] +pub struct InterfaceDescriptor { + /// bInterfaceNumber + pub interface_num: u8, + + /// bAlternateSetting. + pub alternate_setting: u8, + + /// bNumEndpoints is the length of: + pub endpoints: Vec, + + /// bInterfaceClass. + pub class: InterfaceClass, // u8 + + /// bInterfaceSubClass. + pub subclass: InterfaceSubclass, // u8 + + /// bInterfaceProtocol. + pub protocol: InterfaceProtocol, // u8, + + /// iInterface. + pub interface_name: StringIndex, // u8 + + /// Descriptors of class-specific or vendor-specific augmentations. + pub specific_augmentations: Vec, +} + +impl Descriptor for InterfaceDescriptor { + /// bLength is 9 for Interface Descriptor. + fn length(&self) -> u8 { + 9 + } + + /// bDescriptorType. 4 for Interface Descriptor. + fn descriptor_type(&self) -> DescriptorType { + DescriptorType::Interface + } + + /// USB 2.0 table 9-12 + fn serialize(&self) -> Box + '_> { + Box::new( + self.header() + .into_iter() // 0, 1 + .chain([ + self.interface_num, // 2 + self.alternate_setting, // 3 + self.endpoints.len() as u8, // 4 + self.class.0, // 5 + self.subclass.0, // 6 + self.protocol.0, // 7 + self.interface_name.0, // 8 + ]), + ) + } +} + +impl NestedDescriptor for InterfaceDescriptor { + fn total_length(&self) -> u16 { + self.length() as u16 + + self + .specific_augmentations + .iter() + .map(|aug| aug.total_length()) + .sum::() + + self + .endpoints + .iter() + .map(|endpoint| endpoint.total_length()) + .sum::() + } + + fn serialize_all(&self) -> Box + '_> { + Box::new( + self.serialize() + .chain( + self.specific_augmentations + .iter() + .flat_map(NestedDescriptor::serialize_all), + ) + .chain( + self.endpoints + .iter() + .flat_map(NestedDescriptor::serialize_all), + ), + ) + } +} + +#[derive(Debug)] +pub struct EndpointDescriptor { + /// bEndpointAddress. + pub endpoint_addr: u8, + + /// bmAttributes. + pub attributes: EndpointAttributes, + + /// wMaxPacketSize. Largest packet endpoint is capable of transmitting. + pub max_packet_size: u16, + + /// bInterval. Interval for polling transfers in frames on Interrupt and Isoch endpoints. + /// Always 1 for Isoch. Ignored for Bulk and Control endpoints. + pub interval: u8, + + /// Descriptors of class-specific or vendor-specific augmentations. + pub specific_augmentations: Vec, +} + +impl Descriptor for EndpointDescriptor { + /// bLength. 7 for Endpoint Descriptor. + fn length(&self) -> u8 { + 7 + } + + /// bDescriptorType. 5 for Endpoint Descriptor. + fn descriptor_type(&self) -> DescriptorType { + DescriptorType::Endpoint + } + + /// USB 2.0 table 9-13 + fn serialize(&self) -> Box + '_> { + Box::new( + self.header() + .into_iter() // 0, 1 + .chain([self.endpoint_addr, self.attributes.0]) // 2, 3 + .chain(self.max_packet_size.to_le_bytes()) // 4-5 + .chain([self.interval]), // 6 + ) + } +} + +impl NestedDescriptor for EndpointDescriptor { + fn total_length(&self) -> u16 { + self.length() as u16 + + self + .specific_augmentations + .iter() + .map(|aug| aug.total_length()) + .sum::() + } + + fn serialize_all(&self) -> Box + '_> { + Box::new( + self.serialize().chain( + self.specific_augmentations + .iter() + .flat_map(NestedDescriptor::serialize_all), + ), + ) + } +} + +#[derive(Debug)] +pub enum AugmentedDescriptor { + // HID(HidDescriptor) +} +impl Descriptor for AugmentedDescriptor { + fn length(&self) -> u8 { + todo!() + } + + fn descriptor_type(&self) -> DescriptorType { + todo!() + } + + fn serialize(&self) -> Box + '_> { + todo!() + } +} +impl NestedDescriptor for AugmentedDescriptor { + fn total_length(&self) -> u16 { + self.length() as u16 + } + + fn serialize_all(&self) -> Box + '_> { + self.serialize() + } +} + +#[derive(Debug)] +pub struct HidDescriptor { + /// bcdHID. HID standard version. + pub hid_version: Bcd16, + + /// bCountryCode. + pub country_code: CountryCode, + + /// bNumDescriptors is the length of this Vec, + /// which is followed by [bDescriptorType (u8), wDescriptorLength (u16)] + /// for each descriptor at serialization time. + pub class_descriptor: Vec, +} +impl Descriptor for HidDescriptor { + /// bLength. Dependent on bNumDescriptors. + fn length(&self) -> u8 { + todo!() + } + + /// bDescriptorType. 33 for HID Descriptor. + fn descriptor_type(&self) -> DescriptorType { + DescriptorType::HID + } + + fn serialize(&self) -> Box + '_> { + todo!() + } +} + +#[derive(Debug)] +pub struct StringDescriptor { + /// bString. Uses UTF-16 encoding in payloads. + pub string: String, +} + +impl Descriptor for StringDescriptor { + /// UNUSED, but provided for completeness. + /// To avoid doubling the calls to encode_utf16 in serialize, + /// this is computed inline. + // TODO: premature given that it costs an alloc and they're generally small? + // but also they're requested infrequently enough to not matter either way. + fn length(&self) -> u8 { + (self.header().len() + + (self.string.encode_utf16().count() * size_of::())) + as u8 + } + + /// bDescriptorType. 3 for String Descriptor. + fn descriptor_type(&self) -> DescriptorType { + DescriptorType::String + } + + fn serialize(&self) -> Box + '_> { + let utf16: Vec = self.string.encode_utf16().collect(); + let length = 2 + (utf16.len() * size_of::()) as u8; + Box::new( + [length, self.descriptor_type() as u8] + .into_iter() + .chain(utf16.into_iter().flat_map(|w| w.to_le_bytes())), + ) + } +} + +/// special-case for GET_DESCRIPTOR(String, 0) +#[derive(Debug)] +pub struct StringLanguageIdentifierDescriptor { + /// wLANGID. + pub language_ids: Vec, +} + +impl Descriptor for StringLanguageIdentifierDescriptor { + /// bLength. + fn length(&self) -> u8 { + (self.header().len() + (self.language_ids.len() * size_of::())) + as u8 + } + + /// bDescriptorType. 3, as it was with String Descriptor + fn descriptor_type(&self) -> DescriptorType { + DescriptorType::String + } + + fn serialize(&self) -> Box + '_> { + Box::new( + self.header().into_iter().chain( + self.language_ids + .iter() + .flat_map(|langid| u16::from(langid).to_le_bytes()), + ), + ) + } +} + +// USB 2.0 sect 11.23.1 +#[derive(Debug)] +pub struct DeviceQualifierDescriptor { + /// bcdUSB. USB version in binary-coded decimal. + pub usb_version: Bcd16, + /// bDeviceClass. + pub device_class: ClassCode, + /// bDeviceSubClass. + pub device_subclass: SubclassCode, + /// bDeviceProtocol. + pub device_protocol: ProtocolCode, + /// bMaxPacketSize0. + pub max_packet_size_0: MaxSizeZeroEP, + /// bNumConfigurations. Number of other-speed configurations + pub num_configurations: u8, +} + +impl Descriptor for DeviceQualifierDescriptor { + fn length(&self) -> u8 { + 10 + } + + /// bDescriptorType. 6 for Device_Qualifier Descriptor. + fn descriptor_type(&self) -> DescriptorType { + DescriptorType::DeviceQualifier + } + + fn serialize(&self) -> Box + '_> { + Box::new( + self.header() + .into_iter() + .chain(self.usb_version.0.to_le_bytes()) + .chain([ + self.device_class.0, + self.device_subclass.0, + self.device_protocol.0, + self.max_packet_size_0 as u8, + self.num_configurations, + 0, // bReserved + ]), + ) + } +} diff --git a/lib/propolis/src/hw/usb/usbdev/mod.rs b/lib/propolis/src/hw/usb/usbdev/mod.rs new file mode 100644 index 000000000..422d8c9c5 --- /dev/null +++ b/lib/propolis/src/hw/usb/usbdev/mod.rs @@ -0,0 +1,57 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use descriptor::DescriptorType; +use requests::{Request, RequestDirection}; + +pub mod descriptor; +pub mod requests; + +pub mod demo_state_tracker; + +#[derive(Debug, thiserror::Error)] +pub enum Error { + #[error("mismatched Setup and Data Stage transfer direction in transfer: {0:?} != {1:?}")] + SetupVsDataDirectionMismatch(RequestDirection, RequestDirection), + #[error("given an immediate for Out Data Stage")] + ImmediateParameterForOutDataStage, + #[error("In Data Stage memory write failed")] + DataStageWriteFailed, + #[error("Out Data Stage memory read failed")] + DataStageReadFailed, + #[error("expected Setup Stage before {0}")] + NoSetupStageBefore(&'static str), + #[error("matched Setup and Status Stage transfer direction {0:?}")] + SetupVsStatusDirectionMatch(RequestDirection), + #[error("unimplemented request {0:?}")] + UnimplementedRequest(Request), + #[error("invalid payload for {0:?} request: {1:#x?}")] + InvalidPayloadForRequest(Request, Vec), + #[error("unimplemented descriptor type: {0:?}")] + UnimplementedDescriptor(DescriptorType), + #[error("unknown descriptor type: {0:#x}")] + UnknownDescriptorType(u8), +} + +pub type Result = core::result::Result; + +#[usdt::provider(provider = "propolis")] +mod probes { + fn usb_get_descriptor(descriptor_type: u8, index: u8) {} +} + +pub mod migrate { + use serde::{Deserialize, Serialize}; + + #[derive(Serialize, Deserialize, Debug, PartialEq, Eq)] + pub enum UsbDeviceTypeV1 { + Null, + } + + #[derive(Serialize, Deserialize)] + pub struct UsbDeviceV1 { + pub device_type: UsbDeviceTypeV1, + pub current_setup: Option, + } +} diff --git a/lib/propolis/src/hw/usb/usbdev/requests.rs b/lib/propolis/src/hw/usb/usbdev/requests.rs new file mode 100644 index 000000000..9a5d53687 --- /dev/null +++ b/lib/propolis/src/hw/usb/usbdev/requests.rs @@ -0,0 +1,177 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use bitstruct::bitstruct; +use strum::FromRepr; + +#[repr(u8)] +#[derive(FromRepr, Debug, PartialEq, Eq)] +pub enum RequestDirection { + HostToDevice = 0, + DeviceToHost = 1, +} +impl From for RequestDirection { + fn from(value: bool) -> Self { + if value { + Self::DeviceToHost + } else { + Self::HostToDevice + } + } +} +impl Into for RequestDirection { + fn into(self) -> bool { + self as u8 != 0 + } +} + +#[repr(u8)] +#[derive(FromRepr, Debug)] +pub enum RequestType { + Standard = 0, + Class = 1, + Vendor = 2, + Reserved = 3, +} +impl From for RequestType { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("RequestType should only be converted from a 2-bit field in Request") + } +} +impl Into for RequestType { + fn into(self) -> u8 { + self as u8 + } +} + +#[repr(u8)] +#[derive(FromRepr, Debug)] +pub enum RequestRecipient { + Device = 0, + Interface = 1, + Endpoint = 2, + Other = 3, + Reserved4 = 4, + Reserved5 = 5, + Reserved6 = 6, + Reserved7 = 7, + Reserved8 = 8, + Reserved9 = 9, + Reserved10 = 10, + Reserved11 = 11, + Reserved12 = 12, + Reserved13 = 13, + Reserved14 = 14, + Reserved15 = 15, + Reserved16 = 16, + Reserved17 = 17, + Reserved18 = 18, + Reserved19 = 19, + Reserved20 = 20, + Reserved21 = 21, + Reserved22 = 22, + Reserved23 = 23, + Reserved24 = 24, + Reserved25 = 25, + Reserved26 = 26, + Reserved27 = 27, + Reserved28 = 28, + Reserved29 = 29, + Reserved30 = 30, + Reserved31 = 31, +} +impl From for RequestRecipient { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("RequestRecipient should only be converted from a 5-bit field in Request") + } +} +impl Into for RequestRecipient { + fn into(self) -> u8 { + self as u8 + } +} + +#[repr(u8)] +#[derive(FromRepr, Debug)] +pub enum StandardRequest { + GetStatus = 0, + ClearFeature = 1, + Reserved2 = 2, + SetFeature = 3, + Reserved4 = 4, + SetAddress = 5, + GetDescriptor = 6, + SetDescriptor = 7, + GetConfiguration = 8, + SetConfiguration = 9, + GetInterface = 10, + SetInterface = 11, + SynchFrame = 12, +} + +#[derive(Debug)] +pub enum Request { + Standard(StandardRequest), + Other(u8), +} +impl From for Request { + fn from(value: u8) -> Self { + StandardRequest::from_repr(value) + .map(Self::Standard) + .unwrap_or(Self::Other(value)) + } +} +impl Into for Request { + fn into(self) -> u8 { + match self { + Request::Standard(standard_request) => standard_request as u8, + Request::Other(x) => x, + } + } +} + +bitstruct! { + /// USB 2.0 table 9-2. + pub struct SetupData(pub u64) { + /// Part of bRequestType. Whether the request is addressed to the device, + /// one of its interfaces, one of its endpoints, or otherwise. + pub recipient: RequestRecipient = 0..5; + /// Part of bRequestType. Standard, Class, or Vendor. + pub request_type: RequestType = 5..7; + /// Part of bRequestType. Data transfer direction. + pub direction: RequestDirection = 7; + /// bRequest. Specific type of request (USB 2.0 table 9-3) + pub request: Request = 8..16; + /// wValue. Meaning varies according to bRequest. + pub value: u16 = 16..32; + /// wIndex. Meaning varies according to bRequest. + /// Typically used to pass an index or offset. + pub index: u16 = 32..48; + /// wLength. Number of bytes to transfer if there is a Data Stage. + pub length: u16 = 48..64; + } +} +impl core::fmt::Debug for SetupData { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "SetupData {{ \ + recipient: {:?}, \ + request_type: {:?}, \ + direction: {:?}, \ + request: {:?}, \ + value: {}, \ + index: {}, \ + length: {}, \ + }}", + self.recipient(), + self.request_type(), + self.direction(), + self.request(), + self.value(), + self.index(), + self.length() + ) + } +} diff --git a/lib/propolis/src/hw/usb/xhci/bits/device_context.rs b/lib/propolis/src/hw/usb/xhci/bits/device_context.rs new file mode 100644 index 000000000..79216f1a5 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/bits/device_context.rs @@ -0,0 +1,527 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use std::ops::{Deref, DerefMut}; + +use bitstruct::bitstruct; +use strum::FromRepr; +use zerocopy::{FromBytes, Immutable}; + +use crate::{ + common::GuestAddr, + hw::usb::xhci::{device_slots::SlotId, port::PortId}, +}; + +/// See xHCI 1.2 sect 4.5.3 & table 6-7 +#[derive(Copy, Clone, FromRepr, Debug, PartialEq, Eq)] +#[repr(u8)] +pub enum SlotState { + DisabledEnabled = 0, + Default = 1, + Addressed = 2, + Configured = 3, + Reserved4 = 4, + Reserved5 = 5, + Reserved6 = 6, + Reserved7 = 7, + Reserved8 = 8, + Reserved9 = 9, + Reserved10 = 10, + Reserved11 = 11, + Reserved12 = 12, + Reserved13 = 13, + Reserved14 = 14, + Reserved15 = 15, + Reserved16 = 16, + Reserved17 = 17, + Reserved18 = 18, + Reserved19 = 19, + Reserved20 = 20, + Reserved21 = 21, + Reserved22 = 22, + Reserved23 = 23, + Reserved24 = 24, + Reserved25 = 25, + Reserved26 = 26, + Reserved27 = 27, + Reserved28 = 28, + Reserved29 = 29, + Reserved30 = 30, + Reserved31 = 31, +} + +impl Into for SlotState { + fn into(self) -> u8 { + self as u8 + } +} +impl From for SlotState { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("SlotState should only be converted from a 5-bit field in SlotContext") + } +} + +#[derive(Copy, Clone, Immutable, FromBytes, Debug)] +#[repr(C)] +pub struct SlotContext { + first: SlotContextFirst, + reserved: u128, +} + +// HACK: the .with_* from bitstruct will only return First, +// but we ultimately only care about .set_* +impl Deref for SlotContext { + type Target = SlotContextFirst; + fn deref(&self) -> &Self::Target { + &self.first + } +} +impl DerefMut for SlotContext { + fn deref_mut(&mut self) -> &mut Self::Target { + &mut self.first + } +} + +bitstruct! { + /// Representation of the first half of a Slot Context. + /// (the second half is 128-bits of reserved.) + /// + /// See xHCI 1.2 Section 6.2.2 + #[derive(Clone, Copy, Default, Immutable, FromBytes)] + pub struct SlotContextFirst(pub u128) { + /// Used by hubs to route packets to the correct port. (USB3 section 8.9) + pub route_string: u32 = 0..20; + + /// Deprecated in xHCI. + speed: u8 = 20..24; + + reserved0: bool = 24; + + /// Set to 1 by software to enable a hub's Multi-TT (Transaction Translator) + /// interface, if supported by the hub. + pub multi_tt: bool = 25; + + pub hub: bool = 26; + + /// Index of the last valid endpoint context within the Device Context + /// that contains this Slot Context. Valid values are 1 through 31. + pub context_entries: u8 = 27..32; + + /// Indicates the worst-case time it takes to wake up all the links + /// in the path to the device, given the current USB link level power + /// management settings, in microseconds. + pub max_exit_latency_micros: u16 = 32..48; + + /// Indicates the root hub port number used to access this device. + /// Valid values are 1 through the controller's max number of ports. + /// (See xHCI 1.2 sect 4.19.7 for numbering info) + pub root_hub_port_number_: u8 = 48..56; + + /// If this device is a hub, guest sets this to the number of + /// downstream-facing ports supported by the hub. (USB2 table 11-13) + pub number_of_ports: u8 = 56..64; + + pub parent_hub_slot_id: SlotId = 64..72; + + pub parent_port_number: u8 = 72..80; + + pub tt_think_time: u8 = 80..82; + + reserved1: u8 = 82..86; + + /// The index of the interrupter to receive Bandwidth Request Events + /// and Device Notification Events generated by this slot, or when a + /// Ring Underrun or Ring Overrun condition is reported. Valid values + /// are 0..NUM_INTRS + pub interrupter_target: u16 = 86..96; + + /// The address assigned to the USB device in this slot by the xHC, + /// set during a successful Set Address command. + pub usb_device_address: u8 = 96..104; + + reserved2: u32 = 104..123; + + /// Updated by xHC when device slot transitions states. + pub slot_state: SlotState = 123..128 + } +} + +impl SlotContextFirst { + pub fn root_hub_port_number(&self) -> Result { + PortId::try_from(self.root_hub_port_number_()) + } + pub fn with_root_hub_port_number(self, port_id: PortId) -> Self { + self.with_root_hub_port_number_(port_id.as_raw_id()) + } + pub fn set_root_hub_port_number(&mut self, port_id: PortId) { + self.set_root_hub_port_number_(port_id.as_raw_id()); + } +} + +impl core::fmt::Debug for SlotContextFirst { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "SlotContextFirst {{ \ + route_string: {}, \ + multi_tt: {}, \ + hub: {}, \ + context_entries: {}, \ + max_exit_latency_micros: {}, \ + root_hub_port_number: {:?}, \ + number_of_ports: {}, \ + parent_hub_slot_id: {:?}, \ + parent_port_number: {}, \ + tt_think_time: {}, \ + interrupter_target: {}, \ + usb_device_address: {}, \ + slot_state: {:?} }}", + self.route_string(), + self.multi_tt(), + self.hub(), + self.context_entries(), + self.max_exit_latency_micros(), + self.root_hub_port_number(), + self.number_of_ports(), + self.parent_hub_slot_id(), + self.parent_port_number(), + self.tt_think_time(), + self.interrupter_target(), + self.usb_device_address(), + self.slot_state() + ) + } +} + +/// See xHCI 1.2 table 6-8 +#[derive(Copy, Clone, FromRepr, Debug, Eq, PartialEq)] +#[repr(u8)] +pub enum EndpointState { + Disabled = 0, + Running = 1, + Halted = 2, + Stopped = 3, + Error = 4, + Reserved5 = 5, + Reserved6 = 6, + Reserved7 = 7, +} + +impl Into for EndpointState { + fn into(self) -> u8 { + self as u8 + } +} +impl From for EndpointState { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("EndpointState should only be converted from a 3-bit field in EndpointContext") + } +} + +/// See xHCI 1.2 table 6-8 +#[derive(Copy, Clone, FromRepr, Debug)] +#[repr(u8)] +pub enum EndpointType { + NotValid = 0, + IsochOut = 1, + BulkOut = 2, + InterruptOut = 3, + Control = 4, + IsochIn = 5, + BulkIn = 6, + InterruptIn = 7, +} + +impl Into for EndpointType { + fn into(self) -> u8 { + self as u8 + } +} +impl From for EndpointType { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("EndpointType should only be converted from a 3-bit field in EndpointContext") + } +} + +#[derive(Copy, Clone, Immutable, FromBytes, Debug)] +#[repr(C)] +pub struct EndpointContext { + first: EndpointContextFirst, + second: EndpointContextSecond, +} + +impl Deref for EndpointContext { + type Target = EndpointContextFirst; + fn deref(&self) -> &Self::Target { + &self.first + } +} +impl DerefMut for EndpointContext { + fn deref_mut(&mut self) -> &mut Self::Target { + &mut self.first + } +} + +impl EndpointContext { + pub fn average_trb_length(&self) -> u16 { + self.second.average_trb_length() + } + pub fn set_average_trb_length(&mut self, value: u16) { + self.second.set_average_trb_length(value) + } + + /// xHCI 1.2 section 6.2.3.2 + pub fn valid_for_configure_endpoint(&self) -> bool { + // TODO: 1) the values of Max Packet Size, Max Burst Size, and Interval + // are within range for the endpoint type and speed of the device, + // TODO: 2) if MaxPStreams > 0, the TR Dequeue Pointer field points + // to a an array of valid Stream Contexts, or if MaxPStreams == 0, + // the TR Dequeue Pointer field points to a Transfer Ring, + // 3) the EP State field = Disabled, + self.endpoint_state() == EndpointState::Disabled + // 4) all other fields are within their valid range of values: + && self.mult() == 0 // SS Isoch not implemented + && self.max_primary_streams() == 0 // SS not implemented + && !self.linear_stream_array() // RsvdZ if MaxPStreams == 0 + + /* TODO: interval is valid per table 6-12 */ + /* TODO: other fields */ + } +} + +bitstruct! { + /// Representation of the first half of an Endpoint Context. + /// + /// See xHCI 1.2 Section 6.2.3 + #[derive(Clone, Copy, Default, Immutable, FromBytes)] + pub struct EndpointContextFirst(pub u128) { + /// EP State. Current operational state of the Endpoint. + pub endpoint_state: EndpointState = 0..3; + + reserved1: u8 = 3..8; + + /// If LEC=0, indicates one less than the maximum number of bursts + /// within an Interval that this endpoint supports. + /// Valid values are 0..=2 for SS Isoch, 0 for all other endpoints. + pub mult: u8 = 8..10; + + /// Maximum Primary Streams (MaxPStreams). + /// Maximum number of Primary Stream IDs this endpoint supports. + /// 0 indicates Streams are not supported by this endpoint, + /// and tr_dequeue_pointer points to a Transfer Ring. + /// Nonzero values indicate that the tr_dequeue_pointer points to a + /// Primrary Stream Context Array (see xHCI 1.2 sect 4.12). + /// Values of 1 to 15 indicate that the Primary Stream ID Width is + /// (MaxPStreams+1) and the Primary Stream Array contains + /// (2 << MaxPStreams) entries. + /// For SS Bulk endpoints, valid values are defined by MaxPSASize + /// in HCCPARAMS1. Must be 0 for all non-SS endpoints, and for + /// SS Control, SS Isoch and SS Interrupt endpoints. + pub max_primary_streams: u8 = 10..15; + + /// Linear Stream Array (LSA). + /// If MaxPStreams = 0, this field is RsvdZ. + pub linear_stream_array: bool = 15; + + /// Interval. The period between consecutive requests to USB endpoint + /// to send or receive data. The period is calculated as + /// (125 * (1 << interval)) microseconds. See xHCI 1.2 table 6-12. + pub interval: u8 = 16..24; + + pub max_endpoint_service_time_interval_payload_high: u8 = 24..32; + + reserved2: bool = 32; + + pub error_count: u8 = 33..35; + + pub endpoint_type: EndpointType = 35..38; + + reserved3: bool = 38; + + pub host_initiate_disable: bool = 39; + + pub max_burst_size: u8 = 40..48; + + pub max_packet_size: u16 = 48..64; + + pub dequeue_cycle_state: bool = 64; + + reserved4: u8 = 65..68; + + tr_dequeue_pointer_: u64 = 68..128; + } +} + +impl core::fmt::Debug for EndpointContextFirst { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "EndpointContextFirst {{ \ + endpoint_state: {:?}, \ + mult: {}, \ + max_primary_streams: {}, \ + linear_stream_array: {}, \ + interval: {}, \ + max_endpoint_service_time_interval_payload_high: {}, \ + error_count: {}, \ + endpoint_type: {:?}, \ + host_initiate_disable: {}, \ + max_burst_size: {}, \ + max_packet_size: {}, \ + dequeue_cycle_state: {}, \ + tr_dequeue_pointer: {:?} }}", + self.endpoint_state(), + self.mult(), + self.max_primary_streams(), + self.linear_stream_array(), + self.interval(), + self.max_endpoint_service_time_interval_payload_high(), + self.error_count(), + self.endpoint_type(), + self.host_initiate_disable(), + self.max_burst_size(), + self.max_packet_size(), + self.dequeue_cycle_state(), + self.tr_dequeue_pointer(), + ) + } +} + +impl EndpointContextFirst { + pub fn tr_dequeue_pointer(&self) -> GuestAddr { + GuestAddr(self.tr_dequeue_pointer_() << 4) + } + #[must_use] + pub const fn with_tr_dequeue_pointer(self, value: GuestAddr) -> Self { + self.with_tr_dequeue_pointer_(value.0 >> 4) + } + pub fn set_tr_dequeue_pointer(&mut self, value: GuestAddr) { + self.set_tr_dequeue_pointer_(value.0 >> 4); + } +} + +bitstruct! { + /// Representation of the second half of an Endpoint Context. + /// + /// See xHCI 1.2 Section 6.2.3 + #[derive(Clone, Copy, Default, Immutable, FromBytes)] + pub struct EndpointContextSecond(pub u128) { + pub average_trb_length: u16 = 0..16; + + pub max_endpoint_service_time_interval: u16 = 16..32; + + reserved0: u128 = 32..128; + } +} + +impl core::fmt::Debug for EndpointContextSecond { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "EndpointContextSecond {{ \ + average_trb_length: {}, \ + max_endpoint_service_time_interval: {} }}", + self.average_trb_length(), + self.max_endpoint_service_time_interval(), + ) + } +} + +#[repr(C)] +#[derive(Copy, Clone, Immutable, FromBytes, Debug)] +pub struct InputControlContext { + drop_context: AddDropContextFlags, + add_context: AddDropContextFlags, + reserved: [u32; 5], + last: InputControlContextLast, +} +// would love to use bitvec::BitArr!, but need FromBytes +// pub type AddDropContextFlags = bitvec::BitArr!(for 32, in u32); +#[repr(transparent)] +#[derive(Copy, Clone, Immutable, FromBytes, Debug)] +pub struct AddDropContextFlags(u32); + +bitstruct! { + /// Represrentation of the last 32-bits of an InputControlContext. + /// + /// See xHCI 1.2 table 6-17 + #[derive(Clone, Copy, Default, Immutable, FromBytes)] + pub struct InputControlContextLast(pub u32) { + pub configuration_value: u8 = 0..8; + pub interface_number: u8 = 8..16; + pub alternate_setting: u8 = 16..24; + reserved0: u8 = 24..32; + } +} + +impl core::fmt::Debug for InputControlContextLast { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "InputControlContextLast {{ \ + configuration_value: {}, \ + interface_number: {}, \ + alternate_setting: {} }}", + self.configuration_value(), + self.interface_number(), + self.alternate_setting(), + ) + } +} + +impl Deref for InputControlContext { + type Target = InputControlContextLast; + fn deref(&self) -> &Self::Target { + &self.last + } +} +impl DerefMut for InputControlContext { + fn deref_mut(&mut self) -> &mut Self::Target { + &mut self.last + } +} + +impl InputControlContext { + /// xHCI 1.2 table 6-15 + pub fn drop_context_bit(&self, index: u8) -> Option { + if index < 2 || index > 31 { + None + } else { + Some(self.drop_context.0 & (1 << index) != 0) + } + } + /// xHCI 1.2 table 6-15 + pub fn set_drop_context_bit(&mut self, index: u8, value: bool) { + // lower two bits reserved + if index > 2 && index <= 31 { + let mask = 1 << index; + if value { + self.drop_context.0 |= mask; + } else { + self.drop_context.0 &= !mask; + } + } + } + /// Returns whether the context corresponding to the given index + /// should be added in an Evaluate Context command. + /// See xHCI 1.2 table 6-16 + pub fn add_context_bit(&self, index: u8) -> Option { + if index > 31 { + None + } else { + Some(self.add_context.0 & (1 << index) != 0) + } + } + /// xHCI 1.2 table 6-16 + pub fn set_add_context_bit(&mut self, index: u8, value: bool) { + if index <= 31 { + let mask = 1 << index; + if value { + self.add_context.0 |= mask; + } else { + self.add_context.0 &= !mask; + } + } + } +} diff --git a/lib/propolis/src/hw/usb/xhci/bits/mod.rs b/lib/propolis/src/hw/usb/xhci/bits/mod.rs new file mode 100644 index 000000000..9d169154c --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/bits/mod.rs @@ -0,0 +1,1341 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +//! Constants and structures for xHCI. + +// Not all of these fields may be relevant to us, but they're here for completeness. +#![allow(dead_code)] + +use std::{sync::Arc, time::Duration}; + +use crate::{ + common::GuestAddr, + vmm::{time, VmmHdl}, +}; +use bitstruct::bitstruct; +use strum::FromRepr; + +pub mod device_context; +pub mod ring_data; + +/// Statically-known values for Operational/Capability register reads +pub mod values { + use super::*; + use crate::hw::usb::xhci::{ + registers::XHC_REGS, MAX_DEVICE_SLOTS, MAX_PORTS, NUM_INTRS, + }; + + pub const HCS_PARAMS1: HcStructuralParameters1 = HcStructuralParameters1(0) + .with_max_slots(MAX_DEVICE_SLOTS) + .with_max_intrs(NUM_INTRS) + .with_max_ports(MAX_PORTS); + + pub const HCS_PARAMS2: HcStructuralParameters2 = HcStructuralParameters2(0) + .with_ist_as_frame(true) + .with_iso_sched_threshold(0b111) + // We don't need any scratchpad buffers + .with_max_scratchpad_bufs(0) + .with_scratchpad_restore(false); + + // maximum values for each latency, unlikely as we may be to hit them + pub const HCS_PARAMS3: HcStructuralParameters3 = HcStructuralParameters3(0) + .with_u1_dev_exit_latency(10) + .with_u2_dev_exit_latency(2047); + + lazy_static::lazy_static! { + pub static ref HCC_PARAMS1: HcCapabilityParameters1 = { + let extcap_offset = XHC_REGS.extcap_offset(); + assert!(extcap_offset & 3 == 0); + assert!(extcap_offset / 4 < u16::MAX as usize); + HcCapabilityParameters1(0).with_ac64(true).with_xecp( + // xHCI 1.2 table 5-13: offset in 32-bit words from base + (extcap_offset / 4) as u16, + ) + }; + } + + pub const HCC_PARAMS2: HcCapabilityParameters2 = HcCapabilityParameters2(0); + + /// Operational register value for reporting supported page sizes. + /// bit n = 1, if 2^(n+12) is a supported page size. + /// (we only support 1, that being 4KiB, so this const evals to 1). + pub const PAGESIZE_XHCI: u32 = + 1 << (crate::common::PAGE_SIZE.trailing_zeros() - 12); +} + +/// Size of the USB-specific PCI configuration space. +/// +/// See xHCI 1.2 Section 5.2 PCI Configuration Registers (USB) +pub const USB_PCI_CFG_REG_SZ: u8 = 3; + +/// Offset of the USB-specific PCI configuration space. +/// +/// See xHCI 1.2 Section 5.2 PCI Configuration Registers (USB) +pub const USB_PCI_CFG_OFFSET: u8 = 0x60; + +/// Size of the Host Controller Capability Registers (excluding extended capabilities) +pub const XHC_CAP_BASE_REG_SZ: usize = 0x20; + +bitstruct! { + /// Representation of the Frame Length Adjustment Register (FLADJ). + /// + /// See xHCI 1.2 Section 5.2.4 + #[derive(Clone, Copy, Debug, Default)] + pub struct FrameLengthAdjustment(pub u8) { + /// Frame Length Timing Value (FLADJ) + /// + /// Used to select an SOF cycle time by adding 59488 to the value in this field. + /// Ignored if NFC is set to 1. + pub fladj: u8 = 0..6; + + /// No Frame Length Timing Capability (NFC) + /// + /// If set to 1, the controller does not support a Frame Length Timing Value. + pub nfc: bool = 6; + + reserved: u8 = 7..8; + } +} + +bitstruct! { + /// Representation of the Default Best Effort Service Latency \[Deep\] registers (DBESL / DBESLD). + /// + /// See xHCI 1.2 Section 5.2.5 & 5.2.6 + #[derive(Clone, Copy, Debug, Default)] + pub struct DefaultBestEffortServiceLatencies(pub u8) { + /// Default Best Effort Service Latency (DBESL) + pub dbesl: u8 = 0..4; + + /// Default Best Effort Service Latency Deep (DBESLD) + pub dbesld: u8 = 4..8; + } +} + +bitstruct! { + /// Representation of the Structural Parameters 1 (HCSPARAMS1) register. + /// + /// See xHCI 1.2 Section 5.3.3 + #[derive(Clone, Copy, Debug, Default)] + pub struct HcStructuralParameters1(pub u32) { + /// Number of Device Slots (MaxSlots) + /// + /// Indicates the number of device slots that the host controller supports + /// (max num of Device Context Structures and Doorbell Array entries). + /// + /// Valid values are 1-255, 0 is reserved. + pub max_slots: u8 = 0..8; + + /// Number of Interrupters (MaxIntrs) + /// + /// Indicates the number of interrupters that the host controller supports + /// (max addressable Interrupter Register Sets). + /// The value is 1 less than the actual number of interrupters. + /// + /// Valid values are 1-1024, 0 is undefined. + pub max_intrs: u16 = 8..19; + + reserved: u8 = 19..24; + + /// Number of Ports (MaxPorts) + /// + /// Indicates the max Port Number value. + /// + /// Valid values are 1-255. + pub max_ports: u8 = 24..32; + } +} + +bitstruct! { + /// Representation of the Structural Parameters 2 (HCSPARAMS2) register. + /// + /// See xHCI 1.2 Section 5.3.4 + #[derive(Clone, Copy, Debug, Default)] + pub struct HcStructuralParameters2(pub u32) { + /// Isochronous Scheduling Threshold (IST) + /// + /// Minimum distance (in time) required to stay ahead of the controller while adding TRBs. + pub iso_sched_threshold: u8 = 0..3; + + /// Indicates whether the IST value is in terms of frames (true) or microframes (false). + pub ist_as_frame: bool = 3; + + /// Event Ring Segment Table Max (ERST Max) + /// + /// Max num. of Event Ring Segment Table entries = 2^(ERST Max). + /// + /// Valid values are 0-15. + pub erst_max: u8 = 4..8; + + reserved: u16 = 8..21; + + /// Number of Scratchpad Buffers (Max Scratchpad Bufs Hi) + /// + /// High order 5 bits of the number of Scratchpad Buffers that shall be reserved for the + /// controller. + max_scratchpad_bufs_hi: u8 = 21..26; + + /// Scratchpad Restore (SPR) + /// + /// Whether Scratchpad Buffers should be maintained across power events. + pub scratchpad_restore: bool = 26; + + /// Number of Scratchpad Buffers (Max Scratchpad Bufs Lo) + /// + /// Low order 5 bits of the number of Scratchpad Buffers that shall be reserved for the + /// controller. + max_scratchpad_bufs_lo: u8 = 27..32; + } +} + +impl HcStructuralParameters2 { + #[inline] + pub const fn max_scratchpad_bufs(&self) -> u16 { + let lo = self.max_scratchpad_bufs_lo() as u16 | 0b11111; + let hi = self.max_scratchpad_bufs_hi() as u16 | 0b11111; + (hi << 5) | lo + } + + #[inline] + pub const fn with_max_scratchpad_bufs(self, max: u16) -> Self { + let lo = max & 0b11111; + let hi = (max >> 5) & 0b11111; + self.with_max_scratchpad_bufs_lo(lo as u8) + .with_max_scratchpad_bufs_hi(hi as u8) + } +} + +bitstruct! { + /// Representation of the Structural Parameters 3 (HCSPARAMS3) register. + /// + /// See xHCI 1.2 Section 5.3.5 + #[derive(Clone, Copy, Debug, Default)] + pub struct HcStructuralParameters3(pub u32) { + /// U1 Device Exit Latency + /// + /// Worst case latency to transition from U1 to U0. + /// + /// Valid values are 0-10 indicating microseconds. + pub u1_dev_exit_latency: u8 = 0..8; + + reserved: u8 = 8..16; + + /// U2 Device Exit Latency + /// + /// Worst case latency to transition from U2 to U0. + /// + /// Valid values are 0-2047 indicating microseconds. + pub u2_dev_exit_latency: u16 = 16..32; + } +} + +bitstruct! { + /// Representation of the Capability Parameters 1 (HCCPARAMS1) register. + /// + /// See xHCI 1.2 Section 5.3.6 + #[derive(Clone, Copy, Debug, Default)] + pub struct HcCapabilityParameters1(pub u32) { + /// 64-Bit Addressing Capability (AC64) + /// + /// Whether the controller supports 64-bit addressing. + pub ac64: bool = 0; + + /// BW Negotiation Capability (BNC) + /// + /// Whether the controller supports Bandwidth Negotiation. + pub bnc: bool = 1; + + /// Context Size (CSZ) + /// + /// Whether the controller uses the 64-byte Context data structures. + pub csz: bool = 2; + + /// Port Power Control (PPC) + /// + /// Whether the controller supports Port Power Control. + pub ppc: bool = 3; + + /// Port Indicators (PIND) + /// + /// Whether the xHC root hub supports port indicator control. + pub pind: bool = 4; + + /// Light HC Reset Capability (LHRC) + /// + /// Whether the controller supports a Light Host Controller Reset. + pub lhrc: bool = 5; + + /// Latency Tolerance Messaging Capability (LTC) + /// + /// Whether the controller supports Latency Tolerance Messaging. + pub ltc: bool = 6; + + /// No Secondary SID Support (NSS) + /// + /// Whether the controller supports Secondary Stream IDs. + pub nss: bool = 7; + + /// Parse All Event Data (PAE) + /// + /// Whether the controller parses all event data TRBs while advancing to the next TD + /// after a Short Packet, or it skips all but the first Event Data TRB. + pub pae: bool = 8; + + /// Stopped - Short Packet Capability (SPC) + /// + /// Whether the controller is capable of generating a Stopped - Short Packet + /// Completion Code. + pub spc: bool = 9; + + /// Stopped EDTLA Capability (SEC) + /// + /// Whether the controller's Stream Context supports a Stopped EDTLA field. + pub sec: bool = 10; + + /// Contiguous Frame ID Capability (CFC) + /// + /// Whether the controller is capable of matching the Frame ID of consecutive + /// isochronous TDs. + pub cfc: bool = 11; + + /// Maximum Primary Stream Array Size (MaxPSASize) + /// + /// The maximum number of Primary Stream Array entries supported by the controller. + /// + /// Primary Stream Array size = 2^(MaxPSASize + 1) + /// Valid values are 0-15, 0 indicates that Streams are not supported. + pub max_primary_streams: u8 = 12..16; + + /// xHCI Extended Capabilities Pointer (xECP) + /// + /// Offset of the first Extended Capability (in 32-bit words). + pub xecp: u16 = 16..32; + } +} + +bitstruct! { + /// Representation of the Capability Parameters 2 (HCCPARAMS2) register. + /// + /// See xHCI 1.2 Section 5.3.9 + #[derive(Clone, Copy, Debug, Default)] + pub struct HcCapabilityParameters2(pub u32) { + /// U3 Entry Capability (U3C) + /// + /// Whether the controller root hub ports support port Suspend Complete + /// notification. + pub u3c: bool = 0; + + /// Configure Endpoint Command Max Exit Latency Too Large Capability (CMC) + /// + /// Indicates whether a Configure Endpoint Command is capable of generating + /// a Max Exit Latency Too Large Capability Error. + pub cmc: bool = 1; + + /// Force Save Context Capability (FSC) + /// + /// Whether the controller supports the Force Save Context Capability. + pub fsc: bool = 2; + + /// Compliance Transition Capability (CTC) + /// + /// Inidcates whether the xHC USB3 root hub ports support the Compliance Transition + /// Enabled (CTE) flag. + pub ctc: bool = 3; + + /// Large ESIT Payload Capability (LEC) + /// + /// Indicates whether the controller supports ESIT Payloads larger than 48K bytes. + pub lec: bool = 4; + + /// Configuration Information Capability (CIC) + /// + /// Indicates whether the controller supports extended Configuration Information. + pub cic: bool = 5; + + /// Extended TBC Capability (ETC) + /// + /// Indicates if the TBC field in an isochronous TRB supports the definition of + /// Burst Counts greater than 65535 bytes. + pub etc: bool = 6; + + /// Extended TBC TRB Status Capability (ETC_TSC) + /// + /// Indicates if the TBC/TRBSts field in an isochronous TRB has additional + /// information regarding TRB in the TD. + pub etc_tsc: bool = 7; + + /// Get/Set Extended Property Capability (GSC) + /// + /// Indicates if the controller supports the Get/Set Extended Property commands. + pub gsc: bool = 8; + + /// Virtualization Based Trusted I/O Capability (VTC) + /// + /// Whether the controller supports the Virtualization-based Trusted I/O Capability. + pub vtc: bool = 9; + + reserved: u32 = 10..32; + } +} + +bitstruct! { + /// Representation of the USB Command (USBCMD) register. + /// + /// See xHCI 1.2 Section 5.4.1 + #[derive(Clone, Copy, Debug, Default)] + pub struct UsbCommand(pub u32) { + /// Run/Stop (R/S) + /// + /// The controller continues execution as long as this bit is set to 1. + pub run_stop: bool = 0; + + /// Host Controller Reset (HCRST) + /// + /// This control bit is used to reset the controller. + pub host_controller_reset: bool = 1; + + /// Interrupter Enable (INTE) + /// + /// Enables or disables interrupts generated by Interrupters. + pub interrupter_enable: bool = 2; + + /// Host System Error Enable (HSEE) + /// + /// Whether the controller shall assert out-of-band error signaling to the host. + /// See xHCI 1.2 Section 4.10.2.6 + pub host_system_error_enable: bool = 3; + + reserved: u8 = 4..7; + + /// Light Host Controller Reset (LHCRST) + /// + /// This control bit is used to initiate a soft reset of the controller. + /// (If the LHRC bit in HCCPARAMS is set to 1.) + pub light_host_controller_reset: bool = 7; + + /// Controller Save State (CSS) + /// + /// When set to 1, the controller shall save any internal state. + /// Always returns 0 when read. + /// See xHCI 1.2 Section 4.23.2 + pub controller_save_state: bool = 8; + + /// Controller Restore State (CRS) + /// + /// When set to 1, the controller shall perform a Restore State operation. + /// Always returns 0 when read. + /// See xHCI 1.2 Section 4.23.2 + pub controller_restore_state: bool = 9; + + /// Enable Wrap Event (EWE) + /// + /// When set to 1, the controller shall generate an MFINDEX Wrap Event + /// every time the MFINDEX register transitions from 0x3FFF to 0. + /// See xHCI 1.2 Section 4.14.2 + pub enable_wrap_event: bool = 10; + + /// Enable U3 MFINDEX Stop (EU3S) + /// + /// When set to 1, the controller may stop incrementing MFINDEX if all + /// Root Hub ports are in the U3, Disconnected, Disabled or Powered-off states. + /// See xHCI 1.2 Section 4.14.2 + pub enable_u3_mfindex_stop: bool = 11; + + reserved2: u32 = 12; + + /// CEM Enable (CME) + /// + /// When set to 1, a Max Exit Latency Too Large Capability Error may be + /// returned by a Configure Endpoint Command. + /// See xHCI 1.2 Section 4.23.5.2.2 + pub cem_enable: bool = 13; + + /// Extended TBC Enable (ETE) + /// + /// Indicates whether the controller supports Transfer Burst Count (TBC) + /// values greate than 4 in isochronous TDs. + /// See xHCI 1.2 Section 4.11.2.3 + pub ete: bool = 14; + + /// Extended TBC TRB Status Enable (TSC_EN) + /// + /// Indicates whether the controller supports the ETC_TSC capability. + /// See xHCI 1.2 Section 4.11.2.3 + pub tsc_enable: bool = 15; + + /// VTIO Enable (VTIOE) + /// + /// When set to 1, the controller shall enable the VTIO capability. + pub vtio_enable: bool = 16; + + reserved3: u32 = 17..32; + } +} + +bitstruct! { + /// Representation of the USB Status (USBSTS) register. + /// + /// See xHCI 1.2 Section 5.4.2 + #[derive(Clone, Copy, Debug, Default)] + pub struct UsbStatus(pub u32) { + /// Host Controller Halted (HCH) + /// + /// This bit is set to 0 whenever the R/S bit is set to 1. It is set to 1 + /// when the controller has stopped executing due to the R/S bit being cleared. + pub host_controller_halted: bool = 0; + + reserved: u8 = 1; + + /// Host System Error (HSE) + /// + /// Indicates an error condition preventing continuing normal operation. + pub host_system_error: bool = 2; + + /// Event Interrupt (EINT) + /// + /// The controller sets this bit to 1 when the IP bit of any interrupter + /// goes from 0 to 1. + pub event_interrupt: bool = 3; + + /// Port Change Detect (PCD) + /// + /// The controller sets this bit to 1 when any port has a change bit flip + /// from 0 to 1. + pub port_change_detect: bool = 4; + + reserved2: u8 = 5..8; + + /// Save State Status (SSS) + /// + /// A write to the CSS bit in the USBCMD register causes this bit to flip to + /// 1. The controller shall clear this bit to 0 when the Save State operation + /// has completed. + pub save_state_status: bool = 8; + + /// Restore State Status (RSS) + /// + /// A write to the CRS bit in the USBCMD register causes this bit to flip to + /// 1. The controller shall clear this bit to 0 when the Restore State operation + /// has completed. + pub restore_state_status: bool = 9; + + /// Save/Restore Error (SRE) + /// + /// Indicates that the controller has detected an error condition + /// during a Save or Restore State operation. + pub save_restore_error: bool = 10; + + /// Controller Not Ready (CNR) + /// + /// Indicates that the controller is not ready to accept doorbell + /// or runtime register writes. + pub controller_not_ready: bool = 11; + + /// Host Controller Error (HCE) + /// + /// Indicates if the controller has encountered an internal error + /// that requires a reset to recover. + pub host_controller_error: bool = 12; + + reserved3: u32 = 13..32; + } +} + +/// Representation of the Device Notification Control (DNCTRL) register. +/// +/// Bits: 0-15 Notification Enable (N0-N15) +/// +/// When set to 1, the controller shall generate a Device Notification Event +/// when a Device Notification Transaction Packet matching the set bit is received. +/// +/// See xHCI 1.2 Sections 5.4.4, 6.4.2.7 +pub type DeviceNotificationControl = bitvec::BitArr!(for 16, in u32); + +bitstruct! { + /// Representation of the Command Ring Control (CRCR) register. + /// + /// See xHCI 1.2 Section 5.4.5 + #[derive(Clone, Copy, Debug, Default)] + pub struct CommandRingControl(pub u64) { + /// Ring Cycle State (RCS) + /// + /// Indicates the Consumer Cycle State (CCS) flag for the TRB + /// referenced by the Command Ring Pointer (CRP). + pub ring_cycle_state: bool = 0; + + /// Command Stop (CS) + /// + /// When set to 1, the controller shall stop the Command Ring operation + /// after the currently executing command has completed. + pub command_stop: bool = 1; + + /// Command Abort (CA) + /// + /// When set to 1, the controller shall abort the currently executing + /// command and stop the Command Ring operation. + pub command_abort: bool = 2; + + /// Command Ring Running (CRR) + /// + /// This bit is set to 1 if the R/S bit is 1 and software submitted + /// a Host Controller Command. + pub command_ring_running: bool = 3; + + reserved: u8 = 4..6; + + /// Command Ring Pointer (CRP) + /// + /// The high order bits of the initial value of the Command Ring Dequeue Pointer. + command_ring_pointer_: u64 = 6..64; + } +} + +impl CommandRingControl { + /// The Command Ring Dequeue Pointer. + #[inline] + pub fn command_ring_pointer(&self) -> GuestAddr { + GuestAddr(self.command_ring_pointer_() << 6) + } + + /// Build register value with the given Command Ring Dequeue Pointer. + #[inline] + pub fn with_command_ring_pointer(self, addr: GuestAddr) -> Self { + self.with_command_ring_pointer_(addr.0 >> 6) + } + + /// Set the Command Ring Dequeue Pointer. + #[inline] + pub fn set_command_ring_pointer(&mut self, addr: GuestAddr) { + self.set_command_ring_pointer_(addr.0 >> 6) + } +} + +bitstruct! { + /// Representation of the Configure (CONFIG) register. + /// + /// See xHCI 1.2 Section 5.4.7 + #[derive(Clone, Copy, Debug, Default)] + pub struct Configure(pub u32) { + /// Max Device Slots Enabled (MaxSlotsEn) + /// + /// The maximum number of enabled device slots. + /// Valid values are 0 to MaxSlots. + pub max_device_slots_enabled: u8 = 0..8; + + /// U3 Entry Enable (U3E) + /// + /// When set to 1, the controller shall assert the PLC flag + /// when a Root hub port enters U3 state. + pub u3_entry_enable: bool = 8; + + /// Configuration Information Enable (CIE) + /// + /// When set to 1, the software shall initialize the + /// Configuration Value, Interface Number, and Alternate Setting + /// fields in the Input Control Context. + pub configuration_information_enable: bool = 9; + + reserved: u32 = 10..32; + } +} + +/// xHCI 1.2 table 5-27; section 4.19 +#[derive(FromRepr, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug)] +#[repr(u8)] +pub enum PortLinkState { + U0 = 0, + U1 = 1, + U2 = 2, + U3Suspended = 3, + Disabled = 4, + RxDetect = 5, + Inactive = 6, + Polling = 7, + Recovery = 8, + HotReset = 9, + ComplianceMode = 10, + TestMode = 11, + Reserved12 = 12, + Reserved13 = 13, + Reserved14 = 14, + Resume = 15, +} + +impl From for PortLinkState { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("PortLinkState should only be converted from a 4-bit field in PortStatusControl") + } +} +impl Into for PortLinkState { + fn into(self) -> u8 { + self as u8 + } +} + +bitstruct! { + /// Representation of the Port Status and Control (PORTSC) operational register. + /// + /// See xHCI 1.2 section 5.4.8 + #[derive(Clone, Copy, Debug, Default, PartialEq)] + pub struct PortStatusControl(pub u32) { + /// Current Connect Status (CCS) + /// + /// Read-only to software. True iff a device is connected to the port. + /// Reflects current state of the port, may not correspond directly + /// to the event that caused the Connect Status Change bit to be set. + pub current_connect_status: bool = 0; + + /// Port Enabled/Disabled (PED) + /// + /// 1 = Enabled, 0 = Disabled. + /// Software may *disable* a port by writing a '1' to this register, + /// but only the xHC may enable a port. Automatically cleared to '0' + /// by disconnects or faults. + pub port_enabled_disabled: bool = 1; + + reserved0: bool = 2; + + /// Overcurrent Active (OCA) + /// + /// Read-only by software, true iff the port has an over-current condition. + pub overcurrent_active: bool = 3; + + /// Port Reset (PR) + /// + /// When read, true iff port is in reset. + /// Software may write a '1' to set this register to '1', which + /// is done to transition a USB2 port from Polling to Enabled. + pub port_reset: bool = 4; + + /// Port Link State (PLS) + /// + /// May only be written when LWS is true. + pub port_link_state: PortLinkState = 5..9; + + /// Port Power (PP) + /// + /// False iff the port is in powered-off state. + pub port_power: bool = 9; + + /// Port Speed + /// + /// Read-only to software. + /// 0 = Undefined, 1-15 = Protocol Speed ID (PSI). + /// See xHCI 1.2 section 7.2.1 + pub port_speed: u8 = 10..14; + + /// Port Indicator Control (PIC) + /// + /// What color to light the port indicator, if PIND in HCCPARAM1 is set. + /// (0 = off, 1 = amber, 2 = green, 3 = undefined.) Not used by us. + pub port_indicator_control: u8 = 14..16; + + /// Port Link State Write Strobe (LWS) + /// + /// When true, writes to the Port Link State (PLS) field are enabled. + pub port_link_state_write_strobe: bool = 16; + + /// Connect Status Change (CSC) + /// + /// Indicates a change has occurred in CCS or CAS. + /// Software clears this bit by writing a '1' to it. + /// xHC sets to '1' for all changes to the port device connect status, + /// even if software has not cleared an existing CSC. + pub connect_status_change: bool = 17; + + /// Port Enabled/Disabled Change (PEC) + /// + /// For USB2 ports only, '1' indicates a change in PED. + /// Software clears flag by writing '1' to it. + pub port_enabled_disabled_change: bool = 18; + + /// Warm Port Reset Change (WRC) + /// + /// For USB3 ports only. + /// xHC sets to '1' when warm reset processing completes. + /// Software clears flag by writing '1' to it. + pub warm_port_reset_change: bool = 19; + + /// Over-current Change (OCC) + /// + /// xHC sets to '1' when the value of OCA has changed. + /// Software clears flag by writing '1' to it. + pub overcurrent_change: bool = 20; + + /// Port Reset Change (PRC) + /// + /// xHC sets to '1' when PR transitions from '1' to '0', + /// as long as the reset processing was not forced to terminate + /// due to software clearing PP or PED. + /// Software clears flag by writing '1' to it. + pub port_reset_change: bool = 21; + + /// Port Link State Change (PLSC) + /// + /// xHC sets to '1' according to conditions described in + /// sub-table of xHCI 1.2 table 5-27 (bit 22). + /// Software clears flag by writing '1' to it. + pub port_link_state_change: bool = 22; + + /// Port Config Error Change (CEC) + /// + /// For USB3 ports only, xHC sets to '1' if Port Config Error detected. + /// Software clears flag by writing '1' to it. + pub port_config_error_change: bool = 23; + + /// Cold Attach Status (CAS) + /// + /// For USB3 only. See xHCI 1.2 sect 4.19.8 + pub cold_attach_status: bool = 24; + + /// Wake on Connect Enable (WCE) + /// + /// Software writes '1' to enable sensitivity to device connects + /// as system wake-up events. See xHCI 1.2 sect 4.15 + pub wake_on_connect_enable: bool = 25; + + /// Wake on Disconnect Enable (WDE) + /// + /// Software writes '1' to enable sensitivity to device disconnects + /// as system wake-up events. See xHCI 1.2 sect 4.15 + pub wake_on_disconnect_enable: bool = 26; + + /// Wake on Overcurrent Enable (WOE) + /// + /// Software writes '1' to enable sensitivity to over-current conditions + /// as system wake-up events. See xHCI 1.2 sect 4.15 + pub wake_on_overcurrent_enable: bool = 27; + + reserved1: u8 = 28..30; + + /// Device Removable (DR) \[sic\] + /// + /// True iff the attached device is *non-*removable. + pub device_nonremovable: bool = 30; + + /// Warm Port Reset (WPR) + /// + /// For USB3 only. See xHCI 1.2 sect 4.19.5.1 + pub warm_port_reset: bool = 31; + } +} + +impl PortStatusControl { + /// xHCI 1.2 sect 4.19.2, figure 4-34: The PSCEG signal that determines + /// whether an update to PORTSC produces a Port Status Change Event + pub fn port_status_change_event_generation(&self) -> bool { + self.connect_status_change() + || self.port_enabled_disabled_change() + || self.warm_port_reset_change() + || self.overcurrent_change() + || self.port_reset_change() + || self.port_link_state_change() + || self.port_config_error_change() + } +} + +/// State of USB2 Link Power Management (LPM). See xHCI 1.2 table 5-30. +#[derive(FromRepr, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug)] +#[repr(u8)] +pub enum L1Status { + /// Ignored by software. + Invalid = 0, + /// Port successfully transitioned to L1 (ACK) + Success = 1, + /// Device unable to enter L1 at this time (NYET) + NotYet = 2, + /// Device does not support L1 transitions (STALL) + NotSupported = 3, + /// Device failed to respond to the LPM Transaction or an error occurred + TimeoutError = 4, + Reserved5 = 5, + Reserved6 = 6, + Reserved7 = 7, +} + +impl From for L1Status { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("L1Status should only be converted from a 3-bit field in PortPowerManagementStatusControlUsb2") + } +} +impl Into for L1Status { + fn into(self) -> u8 { + self as u8 + } +} + +/// See xHCI 1.2 table 5-30; USB 2.0 sects 7.1.20, 11.24.2.13. +#[derive(FromRepr, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug)] +#[repr(u8)] +pub enum PortTestControl { + /// Port is not operating in a test mode. + TestModeNotEnabled = 0, + /// Test J_STATE + JState = 1, + /// Test K_STATE + KState = 2, + /// Test SE0_NAK + SE0Nak = 3, + /// Test Packet + Packet = 4, + /// Test FORCE_ENABLE + ForceEnable = 5, + Reserved6 = 6, + Reserved7 = 7, + Reserved8 = 8, + Reserved9 = 9, + Reserved10 = 10, + Reserved11 = 11, + Reserved12 = 12, + Reserved13 = 13, + Reserved14 = 14, + /// Port Test Control Error + Error = 15, +} + +impl From for PortTestControl { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("PortTestControl should only be converted from a 4-bit field in PortPowerManagementStatusControlUsb2") + } +} +impl Into for PortTestControl { + fn into(self) -> u8 { + self as u8 + } +} + +bitstruct! { + /// xHCI 1.2 table 5-30 + #[derive(Clone, Copy, Debug, Default)] + pub struct PortPowerManagementStatusControlUsb2(pub u32) { + /// L1 Status (L1S) + /// + /// Read-only to software. Determines whether an L1 suspend request + /// (LPM transaction) was successful. + pub l1_status: L1Status = 0..3; + + /// Remote Wake Enable (RWE) + /// + /// Read/write. Software sets this flag to enable/disable the device + /// for remote wake from L1. While in L1, this flag overrides the + /// current setting of the Remote Wake feature set by the standard + /// Set/ClearFeature() commands defined in USB 2.0 chapter 9. + pub remote_wake_enable: bool = 3; + + /// Best Effort Service Latency (BESL) + /// + /// Read/write. Software sets this field to indicate how long the xHC + /// will drive resume if the xHC initiates an exit from L1. + /// See xHCI 1.2 section 4.23.5.1.1 and table 4-13. + pub best_effort_service_latency: u8 = 4..8; + + /// L1 Device Slot + /// + /// Read/write. Software sets this to the ID of the Device Slot + /// associated with the device directly attached to the Root Hub port. + /// 0 indicates no device is present. xHC uses this to look up info + /// necessary to generate the LPM token packet. + pub l1_device_slot: u8 = 8..16; + + /// Hardware LPM Enable (HLE) + /// + /// Read/write. If true, enable hardware controlled LPM on this port. + /// See xHCI 1.2 sect 4.23.5.1.1.1. + pub hardware_lpm_enable: bool = 16; + + reserved: u16 = 17..28; + + /// Port Test Control (Test Mode) + /// + /// Read/write. Indicates whether the port is operating in test mode, + /// and if so which specific test mode is used. + pub port_test_control: PortTestControl = 28..32; + } +} + +bitstruct! { + /// xHCI 1.2 table 5-29 + #[derive(Clone, Copy, Debug, Default)] + pub struct PortPowerManagementStatusControlUsb3(pub u32) { + pub u1_timeout: u8 = 0..8; + pub u2_timeout: u8 = 8..16; + pub force_link_pm_accept: bool = 16; + reserved: u16 = 17..32; + } +} + +bitstruct! { + /// xHCI 1.2 table 5-31 + #[derive(Clone, Copy, Debug, Default)] + pub struct PortLinkInfoUsb3(pub u32) { + /// Link Error Count + /// + /// Number of link errors detected by the port, saturating at u16::MAX. + pub link_error_count: u16 = 0..16; + + /// Rx Lane Count (RLC) + /// + /// One less than the number of Receive Lanes negotiated by the port. + /// Values from 0 to 15 represent Lane Counts of 1 to 16. Read-only. + pub rx_lane_count: u8 = 16..20; + + /// Tx Lane Count (TLC) + /// + /// One less than the number of Transmit Lanes negotiated by the port. + /// Values from 0 to 15 represent Lane Counts of 1 to 16. Read-only. + pub tx_lane_count: u8 = 20..24; + + reserved: u8 = 24..32; + } +} + +bitstruct! { + /// xHCI 1.2 sect 5.4.10.2: The USB2 PORTLI is reserved. + #[derive(Clone, Copy, Debug, Default)] + pub struct PortLinkInfoUsb2(pub u32) { + reserved: u32 = 0..32; + } +} + +/// xHCI 1.2 table 5-34 +#[derive(FromRepr, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug)] +#[repr(u8)] +pub enum HostInitiatedResumeDurationMode { + /// Initiate L1 using BESL only on timeout + BESL = 0, + /// Initiate L1 using BESLD on timeout. If rejected by device, initiate L1 using BESL. + BESLD = 1, + Reserved2 = 2, + Reserved3 = 3, +} + +impl From for HostInitiatedResumeDurationMode { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("HostInitiatedResumeDurationMode should only be converted from a 2-bit field in PortHardwareLpmControlUsb2") + } +} +impl Into for HostInitiatedResumeDurationMode { + fn into(self) -> u8 { + self as u8 + } +} + +bitstruct! { + /// xHCI 1.2 table 5-34 + #[derive(Clone, Copy, Debug, Default)] + pub struct PortHardwareLpmControlUsb2(pub u32) { + pub host_initiated_resume_duration_mode: HostInitiatedResumeDurationMode = 0..2; + pub l1_timeout: u8 = 2..10; + pub best_effort_service_latency_deep: u8 = 10..14; + reserved: u32 = 14..32; + } +} + +bitstruct! { + /// xHCI 1.2 section 5.4.11.3: The USB3 PORTHLPMC register is reserved. + #[derive(Clone, Copy, Debug, Default)] + pub struct PortHardwareLpmControlUsb3(pub u32) { + reserved: u32 = 0..32; + } +} + +bitstruct! { + /// Representation of the Microframe Index (MFINDEX) runtime register. + /// + /// See xHCI 1.2 Section 5.5.1 + #[derive(Clone, Copy, Debug, Default)] + pub struct MicroframeIndex(pub u32) { + /// The number of 125-millisecond microframes that have passed, + /// only incrementing while [UsbCommand::run_stop] has been set to 1. + /// Read-only. + pub microframe: u16 = 0..14; + + reserved: u32 = 14..32; + } +} + +/// Minimum Interval Time (MIT) = 125 microseconds. xHCI 1.2 sect 4.14.2 +pub const MINIMUM_INTERVAL_TIME: Duration = Duration::from_micros(125); +pub const MFINDEX_WRAP_POINT: u32 = 0x4000; + +impl MicroframeIndex { + /// As MFINDEX is incremented by 1 every 125 microsections while the + /// controller is running, we compute its value based on the instant + /// RS was set to 1 in USBCMD. + pub fn microframe_ongoing( + &self, + run_start: &Option>, + vmm_hdl: &VmmHdl, + ) -> u16 { + let elapsed_microframes = if let Some(then) = run_start { + let now = time::VmGuestInstant::now(vmm_hdl).unwrap(); + // NOTE: div_duration_f32 not available until 1.80, MSRV is 1.70 + (now.saturating_duration_since(**then).as_secs_f64() + / MINIMUM_INTERVAL_TIME.as_secs_f64()) as u16 + } else { + 0 + }; + + self.microframe().wrapping_add(elapsed_microframes) + } +} + +bitstruct! { + /// Representation of the Interrupter Management Register (IMAN). + /// + /// See xHCI 1.2 Section 5.5.2.1 + #[derive(Clone, Copy, Debug, Default)] + pub struct InterrupterManagement(pub u32) { + /// Interrupt Pending (IP) + /// + /// True iff an interrupt is pending for this interrupter. RW1C. + /// See xHCI 1.2 Section 4.17.3 for modification rules. + pub pending: bool = 0; + + /// Interrupt Enable (IE) + /// + /// True if this interrupter is capable of generating an interrupt. + /// When both this and [Self::pending] are true, the interrupter + /// shall generate an interrupt when [InterrupterModeration::counter] + /// reaches 0. + pub enable: bool = 1; + + reserved: u32 = 2..32; + } +} + +bitstruct! { + /// Representation of the Interrupter Moderation Register (IMOD). + /// + /// See xHCI 1.2 Section 5.5.2.2 + #[derive(Clone, Copy, Debug)] + pub struct InterrupterModeration(pub u32) { + /// Interrupt Moderation Interval (IMODI) + /// + /// Minimum inter-interrupt interval, specified in 250 nanosecond increments. + /// 0 disables throttling logic altogether. Default 0x4000 (1 millisecond). + pub interval: u16 = 0..16; + + /// Interrupt Moderation Counter (IMODC) + /// + /// Loaded with the value of [Self::interval] whenever + /// [InterrupterManagement::pending] is cleared to 0, then counts down + /// to 0, and then stops. The associated interrupt is signaled when + /// this counter is 0, the Event Ring is not empty, both flags in + /// [InterrupterManagement] are true, and + /// [EventRingDequeuePointer::handler_busy] is false. + pub counter: u16 = 16..32; + } +} + +pub const IMOD_TICK: Duration = Duration::from_nanos(250); + +impl Default for InterrupterModeration { + fn default() -> Self { + Self(0).with_interval(0x4000) + } +} + +impl InterrupterModeration { + pub fn interval_duration(&self) -> std::time::Duration { + self.interval() as u32 * IMOD_TICK + } +} + +bitstruct! { + /// Representation of the Event Ring Segment Table Size Register (ERSTSZ) + /// + /// See xHCI 1.2 Section 5.5.2.3.1 + // (Note: ERSTSZ, not ersatz -- this is the real deal.) + #[derive(Clone, Copy, Debug, Default)] + pub struct EventRingSegmentTableSize(pub u32) { + /// Number of valid entries in the Event Ring Segment Table pointed to + /// by [EventRingSegmentTableBaseAddress]. The maximum value is defined + /// in [HcStructuralParameters2::erst_max]. + /// + /// For secondary interrupters, writing 0 to this field disables the + /// Event Ring. Events targeted at this Event Ring while disabled result + /// in undefined behavior. + /// + /// Primary interrupters writing 0 to this field is undefined behavior, + /// as the Primary Event Ring cannot be disabled. + pub size: u16 = 0..16; + + reserved: u16 = 16..32; + } +} + +bitstruct! { + /// Representation of the Event Ring Segment Table Base Address Register (ERSTBA). + /// + /// Writing this register starts the Event Ring State Machine. + /// Secondary interrupters may modify the field at any time. + /// Primary interrupters shall not modify this if + /// [UsbStatus::host_controller_halted] is true. + /// + /// See xHCI 1.2 Section 5.5.2.3.2 + #[derive(Clone, Copy, Debug, Default)] + pub struct EventRingSegmentTableBaseAddress(pub u64) { + reserved: u8 = 0..6; + /// Default 0. Defines the high-order bits (6..=63) of the start address + /// of the Event Ring Segment Table. + address_: u64 = 6..64; + } +} + +impl EventRingSegmentTableBaseAddress { + pub fn address(&self) -> GuestAddr { + GuestAddr(self.address_() << 6) + } + #[must_use] + pub const fn with_address(self, value: GuestAddr) -> Self { + self.with_address_(value.0 >> 6) + } + pub fn set_address(&mut self, value: GuestAddr) { + self.set_address_(value.0 >> 6); + } +} + +bitstruct! { + /// Representation of the Event Ring Dequeue Pointer Register (ERDP) + /// + /// See xHCI 1.2 Section 5.5.2.3.2 + #[derive(Clone, Copy, Debug, Default)] + pub struct EventRingDequeuePointer(pub u64) { + /// Dequeue ERST Segment Index (DESI). Default 0. + /// Intended for use by some xHCs to accelerate checking if the + /// Event Ring is full. (Not used by Propolis at time of writing.) + pub dequeue_erst_segment_index: u8 = 0..3; + + /// Event Handler Busy (EHB). Default false. + /// Set to true when Interrupt Pending (IP) is set, cleared by software + /// (by writing true) when the Dequeue Pointer is written. + pub handler_busy: bool = 3; + + /// Default 0. Defines the high-order bits (4..=63) of the address of + /// the current Event Ring Dequeue Pointer. + pointer_: u64 = 4..64; + } +} +impl EventRingDequeuePointer { + pub fn pointer(&self) -> GuestAddr { + GuestAddr(self.pointer_() << 4) + } + #[must_use] + pub const fn with_pointer(self, value: GuestAddr) -> Self { + self.with_pointer_(value.0 >> 4) + } + pub fn set_pointer(&mut self, value: GuestAddr) { + self.set_pointer_(value.0 >> 4); + } +} + +bitstruct! { + /// Representation of a Doorbell Register. + /// + /// Software uses this to notify xHC of work to be done for a Device Slot. + /// From the software's perspective, this should be write-only (reads 0). + /// See xHCI 1.2 Section 5.6 + #[derive(Clone, Copy, Debug, Default)] + pub struct DoorbellRegister(pub u32) { + /// Doorbell Target + /// + /// Written value corresponds to a specific xHC notification. + /// + /// Values 1..=31 correspond to enqueue pointer updates (see spec). + /// Values 0 and 32..=247 are reserved. + /// Values 248..=255 are vendor-defined (and we're the vendor). + pub db_target: u8 = 0..8; + + reserved: u8 = 8..16; + + /// Doorbell Stream ID + /// + /// If the endpoint defines Streams: + /// - This identifies which the doorbell reference is targeting, and + /// - 0, 65535 (No Stream), and 65534 (Prime) are reserved values that + /// software shall not write to this field. + /// + /// If the endpoint does not define Streams, and a nonzero value is + /// written by software, the doorbell reference is ignored. + /// + /// If this is a doorbell is a Host Controller Command Doorbell rather + /// than a Device Context Doorbell, this field shall be cleared to 0. + pub db_stream_id: u16 = 16..32; + } +} + +bitstruct! { + /// Representation of the first 32-bits of the Supported Protocol + /// Extended Capability field. + /// + /// See xHCI 1.2 Section 7.2 + #[derive(Clone, Copy, Debug, Default)] + pub struct SupportedProtocol1(pub u32) { + /// Capability ID. For Supported Protocol, this must be 2 + pub capability_id: u8 = 0..8; + + /// Offset in DWORDs from the start of this register to that of the next. + /// If there is an Extended Capability after this in the list, + /// then set to the number of 32-bit DWORDs in the full register. + /// If this is the last Extended Capability at the XECP, it may be set to 0. + pub next_capability_pointer: u8 = 8..16; + + /// Minor Revision - binary-coded decimal minor release number of + /// a USB specification version supported by the xHC. + pub minor_revision: u8 = 16..24; + + /// Major Revision - binary-coded decimal major release number of + /// a USB specification version supported by the xHC. + pub major_revision: u8 = 24..32; + } +} + +/// The second part of the Supported Protocol Extended Capability field +/// See xHCI 1.2 sect 7.2.2 and table 7-7 +pub const SUPPORTED_PROTOCOL_2: u32 = u32::from_ne_bytes(*b"USB "); + +bitstruct! { + /// Representation of the third 32-bits of the Supported Protocol + /// Extended Capability field. + /// + /// See xHCI 1.2 Section 7.2 + #[derive(Clone, Copy, Debug, Default)] + pub struct SupportedProtocol3(pub u32) { + /// The starting Port Number of Root Hub ports that support this protocol. + /// Valid values are 1 to MAX_PORTS. + pub compatible_port_offset: u8 = 0..8; + + /// The number of consecutive Root Hub ports that support this protocol. + /// Valid values are 1 to MAX_PORTS. + pub compatible_port_count: u8 = 8..16; + + /// Protocol-defined definitions. + /// See xHCI 1.2 section 7.2.2.1.3, tables 7-14 and 7-15 + pub protocol_defined: u16 = 16..28; + + /// Protocol Speed ID Count (PSIC) + /// Indicates number of Protocol Speed ID (PSI) DWORDs that follow + /// the SupportedProtocol4 field. May be set to zero, in which case + /// defaults appropriate for the spec version are used. + pub protocol_speed_id_count: u8 = 28..32; + } +} + +bitstruct! { + /// Representation of the fourth 32-bits of the Supported Protocol + /// Extended Capability field. + /// + /// See xHCI 1.2 Section 7.2 + #[derive(Clone, Copy, Debug, Default)] + pub struct SupportedProtocol4(pub u32) { + /// Protocol Slot Type - the Slot Type value which may be specified + /// when allocating Device Slots that support this protocol. + /// Valid values are ostensibly 0 to 31, but it is also specified that + /// it "shall be set to 0". + /// See xHCI 1.2 sect 4.6.3 and 7.2.2.1.4 and table 7-9. + pub protocol_slot_type: u8 = 0..5; + + reserved: u32 = 5..32; + } +} diff --git a/lib/propolis/src/hw/usb/xhci/bits/ring_data.rs b/lib/propolis/src/hw/usb/xhci/bits/ring_data.rs new file mode 100644 index 000000000..2ad642878 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/bits/ring_data.rs @@ -0,0 +1,900 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use crate::{common::GuestAddr, hw::usb::xhci::device_slots::SlotId}; +use bitstruct::bitstruct; +use strum::FromRepr; +use zerocopy::{FromBytes, Immutable}; + +/// xHCI 1.2 sect 6.5 +#[repr(C)] +#[derive(Copy, Clone, Debug, FromBytes, Immutable)] +pub struct EventRingSegment { + /// Ring Segment Base Address. Lower 6 bits are reserved (addresses are 64-byte aligned). + pub base_address: GuestAddr, + /// Ring Segment Size. Valid values are between 16 and 4096. + pub segment_trb_count: usize, +} + +#[repr(C)] +#[derive(Copy, Clone, FromBytes, Immutable)] +pub struct Trb { + /// may be an address or immediate data + pub parameter: u64, + pub status: TrbStatusField, + pub control: TrbControlField, +} + +impl core::fmt::Debug for Trb { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!( + f, + "Trb {{ parameter: 0x{:x}, control.trb_type: {:?} }}", + self.parameter, + self.control.trb_type() + )?; + Ok(()) + } +} + +impl Default for Trb { + fn default() -> Self { + Self { + parameter: 0, + status: Default::default(), + control: TrbControlField { normal: Default::default() }, + } + } +} + +/// Representations of the 'control' field of Transfer Request Block (TRB). +/// The field definitions differ depending on the TrbType. +/// See xHCI 1.2 Section 6.4.1 (Comments are paraphrases thereof) +#[derive(Copy, Clone, FromBytes, Immutable)] +pub union TrbControlField { + pub normal: TrbControlFieldNormal, + pub setup_stage: TrbControlFieldSetupStage, + pub data_stage: TrbControlFieldDataStage, + pub status_stage: TrbControlFieldStatusStage, + pub link: TrbControlFieldLink, + pub event: TrbControlFieldEvent, + pub transfer_event: TrbControlFieldTransferEvent, + pub slot_cmd: TrbControlFieldSlotCmd, + pub endpoint_cmd: TrbControlFieldEndpointCmd, + pub get_port_bw_cmd: TrbControlFieldGetPortBandwidthCmd, + pub ext_props_cmd: TrbControlFieldExtendedPropsCmd, +} + +impl TrbControlField { + pub fn trb_type(&self) -> TrbType { + // all variants are alike in TRB type location + unsafe { self.normal.trb_type() } + } + + pub fn cycle(&self) -> bool { + // all variants are alike in cycle bit location + unsafe { self.normal.cycle() } + } + + pub fn set_cycle(&mut self, cycle_state: bool) { + // all variants are alike in cycle bit location + unsafe { self.normal.set_cycle(cycle_state) } + } + + pub fn chain_bit(&self) -> Option { + Some(match self.trb_type() { + TrbType::Normal => unsafe { self.normal.chain_bit() }, + TrbType::DataStage => unsafe { self.data_stage.chain_bit() }, + TrbType::StatusStage => unsafe { self.status_stage.chain_bit() }, + TrbType::Link => unsafe { self.link.chain_bit() }, + _ => return None, + }) + } +} + +bitstruct! { + /// Normal TRB control fields (xHCI 1.2 table 6-22) + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldNormal(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer Ring. + pub cycle: bool = 0; + + /// Or "ENT". If set, the xHC shall fetch and evaluate the next TRB + /// before saving the endpoint state (see xHCI 1.2 Section 4.12.3) + pub evaluate_next_trb: bool = 1; + + /// Or "ISP". If set, and a Short Packet is encountered for this TRB + /// (less than the amount specified in the TRB Transfer Length), + /// then a Transfer Event TRB shall be generated with its + /// Completion Code set to Short Packet and its TRB Transfer Length + /// field set to the residual number of bytes not transfered into + /// the associated data buffer. + pub interrupt_on_short_packet: bool = 2; + + /// Or "NS". If set, xHC is permitted to set the No Snoop bit in the + /// Requester attributes of the PCIe transactions it initiates if the + /// PCIe Enable No Snoop flag is also set. (see xHCI 1.2 sect 4.18.1) + pub no_snoop: bool = 3; + + /// Or "CH". If set, this TRB is associated with the next TRB on the + /// ring. The last TRB of a Transfer Descriptor is always unset (0). + pub chain_bit: bool = 4; + + /// Or "IOC". If set, when this TRB completes, the xHC shall notify + /// the system of completion by enqueueing a Transfer Event TRB on the + /// Event ring and triggering an interrupt as appropriate. + /// (see xHCI 1.2 sect 4.10.4, 4.17.5) + pub interrupt_on_completion: bool = 5; + + /// Or "IDT". If set, the Data Buffer Pointer field ([Trb::parameter]) + /// is not a pointer, but an array of between 0 and 8 bytes (specified + /// by the TRB Transfer Length field). Never set on IN endpoints or + /// endpoints that define a Max Packet Size less than 8 bytes. + pub immediate_data: bool = 6; + + reserved1: u8 = 7..9; + + /// Or "BEI". If this and `interrupt_on_completion` are set, the + /// Transfer Event generated shall not interrupt when enqueued. + pub block_event_interrupt: bool = 9; + + /// Set to [TrbType::Normal] for Normal TRBs. + pub trb_type: TrbType = 10..16; + + reserved2: u16 = 16..32; + } +} + +bitstruct! { + /// Setup Stage TRB control fields (xHCI 1.2 table 6-26) + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldSetupStage(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer Ring. + pub cycle: bool = 0; + + reserved1: u8 = 1..5; + + /// Or "IOC". See [TrbControlFieldNormal::interrupt_on_completion] + pub interrupt_on_completion: bool = 5; + + /// Or "IDT". See [TrbControlFieldNormal::immediate_data] + pub immediate_data: bool = 6; + + reserved2: u8 = 7..10; + + /// Set to [TrbType::SetupStage] for Setup Stage TRBs. + pub trb_type: TrbType = 10..16; + + /// Or "TRT". Indicates the type and direction of the control transfer. + pub transfer_type: TrbTransferType = 16..18; + + reserved3: u16 = 18..32; + } +} + +bitstruct! { + /// Data Stage TRB control fields (xHCI 1.2 table 6-29) + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldDataStage(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer Ring. + pub cycle: bool = 0; + + /// Or "ENT". See [TrbControlFieldNormal::evaluate_next_trb] + pub evaluate_next_trb: bool = 1; + + /// Or "ISP". See [TrbControlFieldNormal::interrupt_on_short_packet] + pub interrupt_on_short_packet: bool = 2; + + /// Or "NS". See [TrbControlFieldNormal::no_snoop] + pub no_snoop: bool = 3; + + /// Or "CH". See [TrbControlFieldNormal::chain_bit] + pub chain_bit: bool = 4; + + /// Or "IOC". See [TrbControlFieldNormal::interrupt_on_completion] + pub interrupt_on_completion: bool = 5; + + /// Or "IDT". See [TrbControlFieldNormal::immediate_data] + pub immediate_data: bool = 6; + + reserved1: u8 = 7..10; + + /// Set to [TrbType::DataStage] for Data Stage TRBs. + pub trb_type: TrbType = 10..16; + + /// Or "DIR". Indicates the direction of data transfer, where + /// OUT (0) is toward the device and IN (1) is toward the host. + /// (see xHCI 1.2 sect 4.11.2.2) + pub direction: TrbDirection = 16; + + reserved2: u16 = 17..32; + } +} + +bitstruct! { + /// Status Stage TRB control fields (xHCI 1.2 table 6-31) + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldStatusStage(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer Ring. + pub cycle: bool = 0; + + /// Or "ENT". If set, the xHC shall fetch and evaluate the next TRB + /// before saving the endpoint state (see xHCI 1.2 Section 4.12.3) + pub evaluate_next_trb: bool = 1; + + reserved1: u8 = 2..4; + + /// Or "CH". See [TrbControlFieldNormal::chain_bit] + pub chain_bit: bool = 4; + + /// Or "IOC". See [TrbControlFieldNormal::interrupt_on_completion] + pub interrupt_on_completion: bool = 5; + + reserved2: u8 = 6..10; + + /// Set to [TrbType::StatusStage] for Status Stage TRBs. + pub trb_type: TrbType = 10..16; + + /// Or "DIR". See [TrbControlFieldDataStage::direction] + pub direction: TrbDirection = 16; + + reserved3: u16 = 17..32; + } +} + +bitstruct! { + /// Status Stage TRB control fields (xHCI 1.2 table 6-31) + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldLink(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer or Command Ring. + pub cycle: bool = 0; + + /// Or "TC". If set, the xHC shall toggle its interpretation of the + /// cycle bit. If claered, the xHC shall continue to the next segment + /// using its current cycle bit interpretation. + pub toggle_cycle: bool = 1; + + reserved1: u8 = 2..4; + + /// Or "CH". See [TrbControlFieldNormal::chain_bit] + pub chain_bit: bool = 4; + + /// Or "IOC". See [TrbControlFieldNormal::interrupt_on_completion] + pub interrupt_on_completion: bool = 5; + + reserved2: u8 = 6..10; + + /// Set to [TrbType::Link] for Link TRBs. + pub trb_type: TrbType = 10..16; + + reserved3: u16 = 16..32; + } +} + +bitstruct! { + /// Common control fields in Event TRBs + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldEvent(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer or Command Ring. + pub cycle: bool = 0; + + reserved1: u16 = 1..10; + + // Set to the [TrbType] corresponding to the Event. + pub trb_type: TrbType = 10..16; + + pub virtual_function_id: u8 = 16..24; + + /// ID of the Device Slot corresponding to this event. + pub slot_id: SlotId = 24..32; + } +} + +bitstruct! { + /// Common control fields in Transfer Event TRBs + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldTransferEvent(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer or Command Ring. + pub cycle: bool = 0; + + reserved0: bool = 1; + + /// Or "ED". If set, event was generated by an Event Data TRB and the + /// parameter is a 64-bit value provided by such. If cleared (0), the + /// parameter is a pointer to the TRB that generated this event. + /// (See xHCI 1.2 sect 4.11.5.2) + pub event_data: bool = 2; + + reserved1: u16 = 3..10; + + /// Set to [TrbType::TransferEvent] for Transfer Event TRBs. + pub trb_type: TrbType = 10..16; + + /// ID of the Endpoint that generated the event. Used as an index in + /// the Device Context to select the Endpoint Context associated with + /// this Event. + pub endpoint_id: u8 = 16..21; + + reserved2: u16 = 21..24; + + /// ID of the Device Slot corresponding to this event. + pub slot_id: SlotId = 24..32; + } +} + +bitstruct! { + /// Common control fields in Command TRBs to do with slot enablement + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldSlotCmd(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer or Command Ring. + pub cycle: bool = 0; + + reserved1: u16 = 1..9; + + /// In an Address Device Command TRB, this is BSR (Block SetAddress Request). + /// When true, the Address Device Command shall not generate a USB + /// SET_ADDRESS request. (xHCI 1.2 section 4.6.5, table 6-62) + /// + /// In a Configure Endpoint Command TRB, this is DC (Deconfigure). + pub bit9: bool = 9; + + /// Set to either [TrbType::EnableSlotCmd] or [TrbType::DisableSlotCmd] + pub trb_type: TrbType = 10..16; + + /// Type of Slot to be enabled by this command. (See xHCI 1.2 table 7-9) + pub slot_type: u8 = 16..21; + + reserved2: u8 = 21..24; + + /// ID of the Device Slot corresponding to this event. + pub slot_id: SlotId = 24..32; + } +} + +bitstruct! { + /// Common control fields in Command TRBs to do with endpoint start/stop/reset + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldEndpointCmd(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer or Command Ring. + pub cycle: bool = 0; + + reserved1: u16 = 1..9; + + /// Only in Reset Endpoint Command TRB. + /// If true, the Reset operation doesn't affect the current transfer + /// state of the endpoint. (See also xHCI 1.2 sect 4.6.8.1) + pub transfer_state_preserve: bool = 9; + + /// [TrbType::ConfigureEndpointCmd], [TrbType::ResetEndpointCmd], + /// or [TrbType::StopEndpointCmd]. + pub trb_type: TrbType = 10..16; + + /// The Device Context Index (xHCI 1.2 section 4.8.1) of the EP Context. + /// Valid values are 1..=31. + pub endpoint_id: u8 = 16..21; + + reserved2: u8 = 21..23; + + /// Only in Stop Endpoint Command TRB. + /// If true, we're stopping activity on an endpoint that's about to be + /// suspended, and the endpoint shall be stopped for at least 10ms. + pub suspend: bool = 23; + + /// ID of the Device Slot corresponding to this event. + pub slot_id: SlotId = 24..32; + } +} + +bitstruct! { + /// Control fields of Get Port Bandwidth Command TRB + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldGetPortBandwidthCmd(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer or Command Ring. + pub cycle: bool = 0; + + reserved1: u16 = 1..9; + + /// Only in Reset Endpoint Command TRB. + /// If true, the Reset operation doesn't affect the current transfer + /// state of the endpoint. (See also xHCI 1.2 sect 4.6.8.1) + pub transfer_state_preserve: bool = 9; + + /// Set to [TrbType::GetPortBandwidthCmd] + pub trb_type: TrbType = 10..16; + + /// The bus speed of interest (See 'Port Speed' in xHCI 1.2 table 5-27, + /// but no Undefined or Reserved speeds allowed here) + pub dev_speed: u8 = 16..20; + + reserved2: u8 = 20..24; + + /// ID of the Hub Slot of which the bandwidth shall be returned. + pub hub_slot_id: SlotId = 24..32; + } +} + +bitstruct! { + /// Common control fields in Command TRBs to do with extended properties + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbControlFieldExtendedPropsCmd(pub u32) { + /// Used to mark the Enqueue Pointer of the Transfer or Command Ring. + pub cycle: bool = 0; + + reserved1: u16 = 1..10; + + /// Set to [TrbType::GetExtendedPropertyCmd] or + /// [TrbType::SetExtendedPropertyCmd] + pub trb_type: TrbType = 10..16; + + /// Indicates the specific extended capability specific action the xHC + /// is required to perform. Software sets to 0 when the ECI is 0 + pub subtype: u8 = 16..19; + + /// ID of the Endpoint whose extended properties we're interested in. + /// If nonzero, `slot_id` shall be valid. + pub endpoint_id: u8 = 19..24; + + /// ID of the Device Slot whose extended properties we're interested in. + pub slot_id: SlotId = 24..32; + } +} + +#[derive(Copy, Clone, FromBytes, Immutable)] +pub union TrbStatusField { + pub transfer: TrbStatusFieldTransfer, + pub event: TrbStatusFieldEvent, + pub command_ext: TrbStatusFieldCommandExtProp, +} +impl Default for TrbStatusField { + fn default() -> Self { + Self { transfer: TrbStatusFieldTransfer(0) } + } +} + +bitstruct! { + /// Representation of the 'status' field of Transfer Request Block (TRB). + /// + /// See xHCI 1.2 Section 6.4.1 (Comments are paraphrases thereof) + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbStatusFieldTransfer(pub u32) { + /// For OUT, this field defines the number of data bytes the xHC shall + /// send during the execution of this TRB. If this field is 0 when the + /// xHC fetches this TRB, xHC shall execute a zero-length transaction. + /// (See xHCI 1.2 section 4.9.1 for zero-length TRB handling) + /// + /// For IN, this field indicates the size of the data buffer referenced + /// by the Data Buffer Pointer, i.e. the number of bytes the host + /// expects the endpoint to deliver. + /// + /// "Valid values are 0 to 64K." + pub trb_transfer_length: u32 = 0..17; + + /// Indicates number of packets remaining in the Transfer Descriptor. + /// (See xHCI 1.2 section 4.10.2.4) + pub td_size: u8 = 17..22; + + /// The index of the Interrupter that will receive events generated + /// by this TRB. "Valid values are between 0 and MaxIntrs-1." + pub interrupter_target: u16 = 22..32; + } +} + +bitstruct! { + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbStatusFieldEvent(pub u32) { + /// Optionally set by a command, see xHCI 1.2 sect 4.6.6.1. + pub completion_parameter: u32 = 0..24; + + /// The completion status of the command that generated the event. + /// See xHCI 1.2 section 6.4.5, as well as the specifications for each + /// individual command's behavior in section 4.6. + pub completion_code: TrbCompletionCode = 24..32; + } +} + +bitstruct! { + #[derive(Clone, Copy, Debug, Default, FromBytes, Immutable)] + pub struct TrbStatusFieldCommandExtProp(pub u32) { + /// ECI. Specifies the Extended Capability Identifier associated + /// with the Get/Set Extended Property Command (See xHCI 1.2 table 4-3) + pub extended_capability_id: u16 = 0..16; + + /// In *Set* Extended Property Command TRB, specifies a parameter to be + /// interpreted by the xHC based on the given ECI. + pub capability_parameter: u8 = 16..24; + + reserved: u8 = 24..32; + } +} + +/// xHCI 1.2 Section 6.4.6 +#[derive(FromRepr, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug)] +#[repr(u8)] +pub enum TrbType { + Reserved0 = 0, + Normal = 1, + SetupStage = 2, + DataStage = 3, + StatusStage = 4, + Isoch = 5, + Link = 6, + EventData = 7, + NoOp = 8, + EnableSlotCmd = 9, + DisableSlotCmd = 10, + AddressDeviceCmd = 11, + ConfigureEndpointCmd = 12, + EvaluateContextCmd = 13, + ResetEndpointCmd = 14, + StopEndpointCmd = 15, + SetTRDequeuePointerCmd = 16, + ResetDeviceCmd = 17, + ForceEventCmd = 18, + NegotiateBandwidthCmd = 19, + SetLatencyToleranceValueCmd = 20, + GetPortBandwidthCmd = 21, + ForceHeaderCmd = 22, + NoOpCmd = 23, + GetExtendedPropertyCmd = 24, + SetExtendedPropertyCmd = 25, + Reserved26 = 26, + Reserved27 = 27, + Reserved28 = 28, + Reserved29 = 29, + Reserved30 = 30, + Reserved31 = 31, + TransferEvent = 32, + CommandCompletionEvent = 33, + PortStatusChangeEvent = 34, + BandwidthRequestEvent = 35, + DoorbellEvent = 36, + HostControllerEvent = 37, + DeviceNotificationEvent = 38, + MfIndexWrapEvent = 39, + Reserved40 = 40, + Reserved41 = 41, + Reserved42 = 42, + Reserved43 = 43, + Reserved44 = 44, + Reserved45 = 45, + Reserved46 = 46, + Reserved47 = 47, + Vendor48 = 48, + Vendor49 = 49, + Vendor50 = 50, + Vendor51 = 51, + Vendor52 = 52, + Vendor53 = 53, + Vendor54 = 54, + Vendor55 = 55, + Vendor56 = 56, + Vendor57 = 57, + Vendor58 = 58, + Vendor59 = 59, + Vendor60 = 60, + Vendor61 = 61, + Vendor62 = 62, + Vendor63 = 63, +} + +impl From for TrbType { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("TrbType should only be converted from a 6-bit field in TrbControlField") + } +} +impl Into for TrbType { + fn into(self) -> u8 { + self as u8 + } +} + +/// Or "TRT". See xHCI 1.2 Table 6-26 and Section 4.11.2.2 +#[derive(FromRepr, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug)] +#[repr(u8)] +pub enum TrbTransferType { + NoDataStage = 0, + Reserved = 1, + OutDataStage = 2, + InDataStage = 3, +} +impl From for TrbTransferType { + fn from(value: u8) -> Self { + Self::from_repr(value).expect("TrbTransferType should only be converted from a 2-bit field in TrbControlField") + } +} +impl Into for TrbTransferType { + fn into(self) -> u8 { + self as u8 + } +} + +#[derive(FromRepr, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug)] +#[repr(u8)] +pub enum TrbDirection { + Out = 0, + In = 1, +} +impl From for TrbDirection { + fn from(value: bool) -> Self { + unsafe { core::mem::transmute(value as u8) } + } +} +impl Into for TrbDirection { + fn into(self) -> bool { + self == Self::In + } +} + +#[derive(FromRepr, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug)] +#[repr(u8)] +pub enum TrbCompletionCode { + Invalid = 0, + Success = 1, + DataBufferError = 2, + BabbleDetectedError = 3, + UsbTransactionError = 4, + TrbError = 5, + StallError = 6, + ResourceError = 7, + BandwidthError = 8, + NoSlotsAvailableError = 9, + InvalidStreamTypeError = 10, + SlotNotEnabledError = 11, + EndpointNotEnabledError = 12, + ShortPacket = 13, + RingUnderrun = 14, + RingOverrun = 15, + VfEventRingFullError = 16, + ParameterError = 17, + BandwidthOverrunError = 18, + ContextStateError = 19, + NoPingResponseError = 20, + EventRingFullError = 21, + IncompatibleDeviceError = 22, + MissedServiceError = 23, + CommandRingStopped = 24, + CommandAborted = 25, + Stopped = 26, + StoppedLengthInvalid = 27, + StoppedShortPacket = 28, + MaxExitLatencyTooLarge = 29, + Reserved30 = 30, + IsochBufferOverrun = 31, + EventLostError = 32, + UndefinedError = 33, + InvalidStreamIdError = 34, + SecondaryBandwidthError = 35, + SplitTransactionError = 36, + Reserved37 = 37, + Reserved38 = 38, + Reserved39 = 39, + Reserved40 = 40, + Reserved41 = 41, + Reserved42 = 42, + Reserved43 = 43, + Reserved44 = 44, + Reserved45 = 45, + Reserved46 = 46, + Reserved47 = 47, + Reserved48 = 48, + Reserved49 = 49, + Reserved50 = 50, + Reserved51 = 51, + Reserved52 = 52, + Reserved53 = 53, + Reserved54 = 54, + Reserved55 = 55, + Reserved56 = 56, + Reserved57 = 57, + Reserved58 = 58, + Reserved59 = 59, + Reserved60 = 60, + Reserved61 = 61, + Reserved62 = 62, + Reserved63 = 63, + Reserved64 = 64, + Reserved65 = 65, + Reserved66 = 66, + Reserved67 = 67, + Reserved68 = 68, + Reserved69 = 69, + Reserved70 = 70, + Reserved71 = 71, + Reserved72 = 72, + Reserved73 = 73, + Reserved74 = 74, + Reserved75 = 75, + Reserved76 = 76, + Reserved77 = 77, + Reserved78 = 78, + Reserved79 = 79, + Reserved80 = 80, + Reserved81 = 81, + Reserved82 = 82, + Reserved83 = 83, + Reserved84 = 84, + Reserved85 = 85, + Reserved86 = 86, + Reserved87 = 87, + Reserved88 = 88, + Reserved89 = 89, + Reserved90 = 90, + Reserved91 = 91, + Reserved92 = 92, + Reserved93 = 93, + Reserved94 = 94, + Reserved95 = 95, + Reserved96 = 96, + Reserved97 = 97, + Reserved98 = 98, + Reserved99 = 99, + Reserved100 = 100, + Reserved101 = 101, + Reserved102 = 102, + Reserved103 = 103, + Reserved104 = 104, + Reserved105 = 105, + Reserved106 = 106, + Reserved107 = 107, + Reserved108 = 108, + Reserved109 = 109, + Reserved110 = 110, + Reserved111 = 111, + Reserved112 = 112, + Reserved113 = 113, + Reserved114 = 114, + Reserved115 = 115, + Reserved116 = 116, + Reserved117 = 117, + Reserved118 = 118, + Reserved119 = 119, + Reserved120 = 120, + Reserved121 = 121, + Reserved122 = 122, + Reserved123 = 123, + Reserved124 = 124, + Reserved125 = 125, + Reserved126 = 126, + Reserved127 = 127, + Reserved128 = 128, + Reserved129 = 129, + Reserved130 = 130, + Reserved131 = 131, + Reserved132 = 132, + Reserved133 = 133, + Reserved134 = 134, + Reserved135 = 135, + Reserved136 = 136, + Reserved137 = 137, + Reserved138 = 138, + Reserved139 = 139, + Reserved140 = 140, + Reserved141 = 141, + Reserved142 = 142, + Reserved143 = 143, + Reserved144 = 144, + Reserved145 = 145, + Reserved146 = 146, + Reserved147 = 147, + Reserved148 = 148, + Reserved149 = 149, + Reserved150 = 150, + Reserved151 = 151, + Reserved152 = 152, + Reserved153 = 153, + Reserved154 = 154, + Reserved155 = 155, + Reserved156 = 156, + Reserved157 = 157, + Reserved158 = 158, + Reserved159 = 159, + Reserved160 = 160, + Reserved161 = 161, + Reserved162 = 162, + Reserved163 = 163, + Reserved164 = 164, + Reserved165 = 165, + Reserved166 = 166, + Reserved167 = 167, + Reserved168 = 168, + Reserved169 = 169, + Reserved170 = 170, + Reserved171 = 171, + Reserved172 = 172, + Reserved173 = 173, + Reserved174 = 174, + Reserved175 = 175, + Reserved176 = 176, + Reserved177 = 177, + Reserved178 = 178, + Reserved179 = 179, + Reserved180 = 180, + Reserved181 = 181, + Reserved182 = 182, + Reserved183 = 183, + Reserved184 = 184, + Reserved185 = 185, + Reserved186 = 186, + Reserved187 = 187, + Reserved188 = 188, + Reserved189 = 189, + Reserved190 = 190, + Reserved191 = 191, + VendorDefinedError192 = 192, + VendorDefinedError193 = 193, + VendorDefinedError194 = 194, + VendorDefinedError195 = 195, + VendorDefinedError196 = 196, + VendorDefinedError197 = 197, + VendorDefinedError198 = 198, + VendorDefinedError199 = 199, + VendorDefinedError200 = 200, + VendorDefinedError201 = 201, + VendorDefinedError202 = 202, + VendorDefinedError203 = 203, + VendorDefinedError204 = 204, + VendorDefinedError205 = 205, + VendorDefinedError206 = 206, + VendorDefinedError207 = 207, + VendorDefinedError208 = 208, + VendorDefinedError209 = 209, + VendorDefinedError210 = 210, + VendorDefinedError211 = 211, + VendorDefinedError212 = 212, + VendorDefinedError213 = 213, + VendorDefinedError214 = 214, + VendorDefinedError215 = 215, + VendorDefinedError216 = 216, + VendorDefinedError217 = 217, + VendorDefinedError218 = 218, + VendorDefinedError219 = 219, + VendorDefinedError220 = 220, + VendorDefinedError221 = 221, + VendorDefinedError222 = 222, + VendorDefinedError223 = 223, + VendorDefinedInfo224 = 224, + VendorDefinedInfo225 = 225, + VendorDefinedInfo226 = 226, + VendorDefinedInfo227 = 227, + VendorDefinedInfo228 = 228, + VendorDefinedInfo229 = 229, + VendorDefinedInfo230 = 230, + VendorDefinedInfo231 = 231, + VendorDefinedInfo232 = 232, + VendorDefinedInfo233 = 233, + VendorDefinedInfo234 = 234, + VendorDefinedInfo235 = 235, + VendorDefinedInfo236 = 236, + VendorDefinedInfo237 = 237, + VendorDefinedInfo238 = 238, + VendorDefinedInfo239 = 239, + VendorDefinedInfo240 = 240, + VendorDefinedInfo241 = 241, + VendorDefinedInfo242 = 242, + VendorDefinedInfo243 = 243, + VendorDefinedInfo244 = 244, + VendorDefinedInfo245 = 245, + VendorDefinedInfo246 = 246, + VendorDefinedInfo247 = 247, + VendorDefinedInfo248 = 248, + VendorDefinedInfo249 = 249, + VendorDefinedInfo250 = 250, + VendorDefinedInfo251 = 251, + VendorDefinedInfo252 = 252, + VendorDefinedInfo253 = 253, + VendorDefinedInfo254 = 254, + VendorDefinedInfo255 = 255, +} + +impl From for TrbCompletionCode { + fn from(value: u8) -> Self { + // the field is 8-bits and the entire range is defined in the enum + unsafe { core::mem::transmute(value) } + } +} +impl Into for TrbCompletionCode { + fn into(self) -> u8 { + self as u8 + } +} diff --git a/lib/propolis/src/hw/usb/xhci/controller.rs b/lib/propolis/src/hw/usb/xhci/controller.rs new file mode 100644 index 000000000..8a0498da9 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/controller.rs @@ -0,0 +1,1140 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +//! Emulated USB Host Controller + +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use bitvec::field::BitField; +use device_slots::SlotId; + +use crate::common::{GuestAddr, Lifecycle, RWOp, ReadOp, WriteOp}; +use crate::hw::ids::pci::{PROPOLIS_XHCI_DEV_ID, VENDOR_OXIDE}; +use crate::hw::pci::{self, Device}; +use crate::hw::usb::usbdev::demo_state_tracker::NullUsbDevice; +use crate::hw::usb::xhci::bits::ring_data::TrbCompletionCode; +use crate::hw::usb::xhci::port::PortId; +use crate::hw::usb::xhci::rings::consumer::doorbell; +use crate::migrate::{MigrateMulti, Migrator}; +use crate::vmm::{time, VmmHdl}; + +use super::device_slots::DeviceSlotTable; +use super::rings::consumer::command::CommandRing; +use super::{bits::values::*, registers::*, *}; + +#[usdt::provider(provider = "propolis")] +mod probes { + fn xhci_reset() {} + fn xhci_reg_read(reg_name: &str, value: u64, index: i16) {} + fn xhci_reg_write(reg_name: &str, value: u64, index: i16) {} +} + +pub struct XhciState { + /// USB Command Register + usbcmd: bits::UsbCommand, + + /// USB Status Register + pub(super) usbsts: bits::UsbStatus, + + /// Device Notification Control Register + dnctrl: bits::DeviceNotificationControl, + + /// Microframe counter (125 ms per tick while running) + mfindex: bits::MicroframeIndex, + + /// Used for computing MFINDEX while Run/Stop (RS) is set + run_start: Option>, + + /// If USBCMD EWE is enabled, generates MFINDEX Wrap Events periodically while USBCMD RS=1 + mfindex_wrap_thread: Option>, + + /// Make stale threads self-destruct after changes to Run/Stop or EWE. + mfindex_wrap_thread_generation: u32, + + /// Interrupters, including registers and the Event Ring + pub(super) interrupters: [interrupter::XhciInterrupter; NUM_INTRS as usize], + + /// EINT in USBSTS is set when any interrupters IP changes from 0 to 1. + /// we give a weak reference to this to our interrupters, and set the flag + /// in USBSTS before reading it when it's true. + any_interrupt_pending_raised: Arc, + + pub(super) command_ring: Option, + + /// Command Ring Control Register (CRCR). + pub(super) crcr: bits::CommandRingControl, + + pub(super) dev_slots: DeviceSlotTable, + + /// Configure Register + config: bits::Configure, + + port_regs: [Box; MAX_PORTS as usize], + + /// Event Data Transfer Length Accumulator (EDTLA). + pub(super) evt_data_xfer_len_accum: u32, + + /// USB devices to attach (currently only supports a proof-of-concept + /// "device" used for testing basic xHC functionality) + queued_device_connections: Vec<(PortId, NullUsbDevice)>, + vmm_hdl: Arc, +} + +impl XhciState { + fn new( + pci_state: &pci::DeviceState, + vmm_hdl: Arc, + log: slog::Logger, + ) -> Self { + // The controller is initially halted and asserts CNR (controller not ready) + let usb_sts = bits::UsbStatus(0) + .with_host_controller_halted(true) + .with_controller_not_ready(true); + + let any_interrupt_pending_raised = Arc::new(AtomicBool::new(false)); + + let pci_intr = interrupter::XhciPciIntr::new(&pci_state, log.clone()); + let interrupters = [interrupter::XhciInterrupter::new( + 0, + pci_intr, + vmm_hdl.clone(), + Arc::downgrade(&any_interrupt_pending_raised), + log.clone(), + )]; + + Self { + vmm_hdl, + usbcmd: bits::UsbCommand(0), + usbsts: usb_sts, + dnctrl: bits::DeviceNotificationControl::new([0]), + dev_slots: DeviceSlotTable::new(log.clone()), + config: bits::Configure(0), + mfindex: bits::MicroframeIndex(0), + run_start: None, + mfindex_wrap_thread: None, + mfindex_wrap_thread_generation: 0, + interrupters, + any_interrupt_pending_raised, + command_ring: None, + crcr: bits::CommandRingControl(0), + port_regs: [ + // NUM_USB2_PORTS = 4 + Box::new(port::Usb2Port::default()), + Box::new(port::Usb2Port::default()), + Box::new(port::Usb2Port::default()), + Box::new(port::Usb2Port::default()), + // NUM_USB3_PORTS = 4 + Box::new(port::Usb3Port::default()), + Box::new(port::Usb3Port::default()), + Box::new(port::Usb3Port::default()), + Box::new(port::Usb3Port::default()), + ], + evt_data_xfer_len_accum: 0, + queued_device_connections: vec![], + } + } + + fn apply_ip_raise_to_usbsts_eint(&mut self) { + if self + .any_interrupt_pending_raised + .compare_exchange(true, false, Ordering::Acquire, Ordering::Relaxed) + .is_ok() + { + self.usbsts.set_event_interrupt(true); + } + } +} + +/// An emulated USB Host Controller attached over PCI +pub struct PciXhci { + /// PCI device state + pci_state: pci::DeviceState, + + /// Controller state + state: Arc>, + + log: slog::Logger, +} + +impl PciXhci { + /// Create a new pci-xhci device + pub fn create(hdl: Arc, log: slog::Logger) -> Arc { + let pci_builder = pci::Builder::new(pci::Ident { + vendor_id: VENDOR_OXIDE, + device_id: PROPOLIS_XHCI_DEV_ID, + sub_vendor_id: VENDOR_OXIDE, + sub_device_id: PROPOLIS_XHCI_DEV_ID, + class: pci::bits::CLASS_SERIAL_BUS, + subclass: pci::bits::SUBCLASS_USB, + prog_if: pci::bits::PROGIF_USB3, + ..Default::default() + }); + + let pci_state = pci_builder + .add_bar_mmio64(pci::BarN::BAR0, 0x2000) + // Place MSI-X in BAR4 + .add_cap_msix(pci::BarN::BAR4, NUM_INTRS) + .add_custom_cfg(bits::USB_PCI_CFG_OFFSET, bits::USB_PCI_CFG_REG_SZ) + .finish(); + + let state = + Arc::new(Mutex::new(XhciState::new(&pci_state, hdl, log.clone()))); + + Arc::new(Self { pci_state, state, log }) + } + + pub fn add_usb_device( + &self, + raw_port: u8, + // TODO: pass the device when real ones exist + ) -> Result<(), String> { + let mut state = self.state.lock().unwrap(); + let port_id = PortId::try_from(raw_port)?; + let dev = NullUsbDevice::default(); + state.queued_device_connections.push((port_id, dev)); + Ok(()) + } + + /// Handle read of register in USB-specific PCI configuration space + fn usb_cfg_read(&self, id: UsbPciCfgReg, ro: &mut ReadOp) { + match id { + UsbPciCfgReg::SerialBusReleaseNumber => { + // USB 3.0 + ro.write_u8(0x30); + } + UsbPciCfgReg::FrameLengthAdjustment => { + // We don't support adjusting the SOF cycle + let fladj = bits::FrameLengthAdjustment(0).with_nfc(true); + ro.write_u8(fladj.0); + } + UsbPciCfgReg::DefaultBestEffortServiceLatencies => { + // We don't support link power management so return 0 + ro.write_u8(bits::DefaultBestEffortServiceLatencies(0).0); + } + } + } + + /// Handle write to register in USB-specific PCI configuration space + fn usb_cfg_write(&self, id: UsbPciCfgReg, _wo: &mut WriteOp) { + match id { + // Ignore writes to read-only register + UsbPciCfgReg::SerialBusReleaseNumber => {} + + // We don't support adjusting the SOF cycle + UsbPciCfgReg::FrameLengthAdjustment => {} + + // We don't support link power management + UsbPciCfgReg::DefaultBestEffortServiceLatencies => {} + } + } + + /// Handle read of memory-mapped host controller register + fn reg_read(&self, id: Registers, ro: &mut ReadOp) { + use CapabilityRegisters::*; + use OperationalRegisters::*; + use Registers::*; + use RuntimeRegisters::*; + + use RegRWOpValue::*; + let mut reg_index = -1; + let value = match id { + Reserved => RegRWOpValue::Fill(0), + + // Capability registers + Cap(CapabilityLength) => { + // xHCI 1.2 Section 5.3.1: Used to find the beginning of + // operational registers. + U8(XHC_REGS.operational_offset() as u8) + } + Cap(HciVersion) => { + // xHCI 1.2 Section 5.3.2: xHCI Version 1.2.0 + U16(0x0120) + } + Cap(HcStructuralParameters1) => U32(HCS_PARAMS1.0), + Cap(HcStructuralParameters2) => U32(HCS_PARAMS2.0), + Cap(HcStructuralParameters3) => U32(HCS_PARAMS3.0), + Cap(HcCapabilityParameters1) => U32(HCC_PARAMS1.0), + Cap(HcCapabilityParameters2) => U32(HCC_PARAMS2.0), + // Per layout defined in XhcRegMap. + Cap(DoorbellOffset) => U32(XHC_REGS.doorbell_offset() as u32), + Cap(RuntimeRegisterSpaceOffset) => { + U32(XHC_REGS.runtime_offset() as u32) + } + + // Operational registers + Op(UsbCommand) => U32(self.state.lock().unwrap().usbcmd.0), + Op(UsbStatus) => { + let mut state = self.state.lock().unwrap(); + state.apply_ip_raise_to_usbsts_eint(); + U32(state.usbsts.0) + } + + Op(PageSize) => U32(PAGESIZE_XHCI), + + Op(DeviceNotificationControl) => { + U32(self.state.lock().unwrap().dnctrl.data[0]) + } + + Op(CommandRingControlRegister1) => { + // xHCI 1.2 table 5-24: Most of these fields read as 0, except for CRR + let state = self.state.lock().unwrap(); + let crcr = bits::CommandRingControl(0) + .with_command_ring_running( + state.crcr.command_ring_running(), + ); + U32(crcr.0 as u32) + } + // xHCI 1.2 table 5-24: The upper region of this register is all + // upper bits of the command ring pointer, which returns 0 for reads. + Op(CommandRingControlRegister2) => U32(0), + + Op(DeviceContextBaseAddressArrayPointerRegister) => { + let state = self.state.lock().unwrap(); + let addr = state.dev_slots.dcbaap().map(|x| x.0).unwrap_or(0); + U64(addr) + } + Op(Configure) => U32(self.state.lock().unwrap().config.0), + Op(Port(port_id, regs)) => { + reg_index = port_id.as_raw_id() as i16; + let state = self.state.lock().unwrap(); + state.port_regs[port_id.as_index()].reg_read(regs) + } + + // Runtime registers + Runtime(MicroframeIndex) => { + let state = self.state.lock().unwrap(); + let mf_adjusted = state + .mfindex + .microframe_ongoing(&state.run_start, &state.vmm_hdl); + U32(state.mfindex.with_microframe(mf_adjusted).0) + } + Runtime(Interrupter(i, intr_regs)) => { + self.state.lock().unwrap().interrupters[i as usize] + .reg_read(intr_regs) + } + + // Only for software to write, returns 0 when read. + Doorbell(i) => { + reg_index = i as i16; + U32(0) + } + + ExtCap(ExtendedCapabilityRegisters::SupportedProtocol1(i)) => { + reg_index = i as i16; + const CAP: bits::SupportedProtocol1 = + bits::SupportedProtocol1(0) + .with_capability_id(2) + .with_next_capability_pointer(4); + + U32(match i { + 0 => CAP.with_minor_revision(0).with_major_revision(2).0, + 1 => CAP.with_minor_revision(0).with_major_revision(3).0, + // possible values of i defined in registers.rs + _ => unreachable!("unsupported SupportedProtocol1 {i}"), + }) + } + ExtCap(ExtendedCapabilityRegisters::SupportedProtocol2(i)) => { + reg_index = i as i16; + U32(bits::SUPPORTED_PROTOCOL_2) + } + ExtCap(ExtendedCapabilityRegisters::SupportedProtocol3(i)) => { + reg_index = i as i16; + let cap = bits::SupportedProtocol3::default() + .with_protocol_defined(0) + .with_protocol_speed_id_count(0); + U32(match i { + 0 => { + cap.with_compatible_port_offset(1) + .with_compatible_port_count(NUM_USB2_PORTS) + .0 + } + 1 => { + cap.with_compatible_port_offset(1 + NUM_USB2_PORTS) + .with_compatible_port_count(NUM_USB3_PORTS) + .0 + } + // possible values of i defined in registers.rs + _ => unreachable!("unsupported SupportedProtocol3 {i}"), + }) + } + ExtCap(ExtendedCapabilityRegisters::SupportedProtocol4(i)) => { + reg_index = i as i16; + U32(bits::SupportedProtocol4::default() + .with_protocol_slot_type(0) + .0) + } + // end of list of ext caps + ExtCap(ExtendedCapabilityRegisters::Reserved) => U32(0), + }; + + match value { + RegRWOpValue::NoOp => {} + RegRWOpValue::U8(x) => ro.write_u8(x), + RegRWOpValue::U16(x) => ro.write_u16(x), + RegRWOpValue::U32(x) => ro.write_u32(x), + RegRWOpValue::U64(x) => ro.write_u64(x), + RegRWOpValue::Fill(x) => ro.fill(x), + } + + let reg_name = id.reg_name(); + let reg_value = value.as_u64(); + probes::xhci_reg_read!(|| (reg_name, reg_value, reg_index)); + } + + /// Handle write to memory-mapped host controller register + fn reg_write(&self, id: Registers, wo: &mut WriteOp) { + use OperationalRegisters::*; + use RegRWOpValue::*; + use Registers::*; + use RuntimeRegisters::*; + + let mut reg_index = -1; + let written_value = match id { + // Ignore writes to reserved bits + Reserved => NoOp, + + // Capability registers are all read-only; ignore any writes + Cap(_) => NoOp, + + // Operational registers + Op(UsbCommand) => { + let mut state = self.state.lock().unwrap(); + let cmd = bits::UsbCommand(wo.read_u32()); + + // xHCI 1.2 Section 5.4.1.1 + if cmd.run_stop() && !state.usbcmd.run_stop() { + if !state.usbsts.host_controller_halted() { + slog::error!( + self.log, + "USBCMD Run while not Halted: undefined behavior!" + ); + } + state.usbsts.set_host_controller_halted(false); + + // xHCI 1.2 sect 4.3 + let mut queued_conns = Vec::new(); + core::mem::swap( + &mut queued_conns, + &mut state.queued_device_connections, + ); + for (port_id, usb_dev) in queued_conns { + let memctx = self.pci_state.acc_mem.access().unwrap(); + if let Some(evt) = state.port_regs[port_id.as_index()] + .xhc_update_portsc( + &|portsc| { + *portsc = portsc + .with_current_connect_status(true) + .with_port_enabled_disabled(false) + .with_port_reset(false) + .with_port_link_state( + bits::PortLinkState::Polling, + ); + }, + port_id, + ) + { + if let Err(e) = state.interrupters[0] + .enqueue_event(evt, &memctx, false) + { + slog::error!(&self.log, "unable to signal Port Status Change for device attach: {e}"); + } + } + state.usbsts.set_port_change_detect(true); + if let Err(_) = state + .dev_slots + .attach_to_root_hub_port_address(port_id, usb_dev) + { + slog::error!(&self.log, "root hub port {port_id:?} already had a device attached"); + } + } + + slog::debug!( + self.log, + "command ring at {:#x}", + state.crcr.command_ring_pointer().0 + ); + // unwrap: crcr.command_ring_pointer() can only return 64-aligned values + state.command_ring = Some( + CommandRing::new( + state.crcr.command_ring_pointer(), + state.crcr.ring_cycle_state(), + ) + .unwrap(), + ); + + // for MFINDEX computation + state.run_start = Some(Arc::new( + time::VmGuestInstant::now(&state.vmm_hdl).unwrap(), + )); + if state.usbcmd.enable_wrap_event() { + self.start_mfindex_wrap_thread(&mut state); + } + } else if !cmd.run_stop() && state.usbcmd.run_stop() { + // stop running/queued commands and transfers on all device slots. + + // apply new MFINDEX value based on time elapsed running + let run_start = state.run_start.take(); + let mf_index = state + .mfindex + .microframe_ongoing(&run_start, &state.vmm_hdl); + state.mfindex.set_microframe(mf_index); + self.stop_mfindex_wrap_thread(&mut state); + + state.usbsts.set_host_controller_halted(true); + // xHCI 1.2 table 5-24: cleared to 0 when R/S is. + state.crcr.set_command_ring_running(false); + } + + // xHCI 1.2 table 5-20: Any transactions in progress are + // immediately terminated; all internal pipelines, registers, + // timers, counters, state machines, etc. are reset to their + // initial value. + if cmd.host_controller_reset() { + let mut devices = Vec::new(); + core::mem::swap( + &mut devices, + &mut state.queued_device_connections, + ); + devices.extend(state.dev_slots.detach_all_for_reset()); + + *state = XhciState::new( + &self.pci_state, + state.vmm_hdl.clone(), + self.log.clone(), + ); + state.queued_device_connections = devices; + + state.usbsts.set_controller_not_ready(false); + slog::debug!(self.log, "xHC reset"); + probes::xhci_reset!(|| ()); + return; + } + + let usbcmd_inte = cmd.interrupter_enable(); + if usbcmd_inte != state.usbcmd.interrupter_enable() { + for interrupter in &mut state.interrupters { + interrupter.set_usbcmd_inte(usbcmd_inte); + } + slog::debug!( + self.log, + "Interrupter Enabled: {usbcmd_inte}", + ); + } + + // xHCI 1.2 Section 4.10.2.6 + if cmd.host_system_error_enable() { + slog::debug!( + self.log, + "USBCMD HSEE unused (USBSTS HSE unimplemented)" + ); + } + + // xHCI 1.2 Section 4.23.2.1 + if cmd.controller_save_state() { + if state.usbsts.save_state_status() { + slog::error!( + self.log, + "save state while saving: undefined behavior!" + ); + } + if state.usbsts.host_controller_halted() { + slog::error!( + self.log, + "unimplemented USBCMD: Save State" + ); + } + } + // xHCI 1.2 Section 4.23.2 + if cmd.controller_restore_state() { + if state.usbsts.save_state_status() { + slog::error!( + self.log, + "restore state while saving: undefined behavior!" + ); + } + if state.usbsts.host_controller_halted() { + slog::error!( + self.log, + "unimplemented USBCMD: Restore State" + ); + } + } + + // xHCI 1.2 Section 4.14.2 + if cmd.enable_wrap_event() && !state.usbcmd.enable_wrap_event() + { + self.start_mfindex_wrap_thread(&mut state); + } else if !cmd.enable_wrap_event() + && state.usbcmd.enable_wrap_event() + { + self.stop_mfindex_wrap_thread(&mut state); + } + + // xHCI 1.2 Section 4.14.2 + if cmd.enable_u3_mfindex_stop() { + slog::error!( + self.log, + "unimplemented USBCMD: Enable U3 MFINDEX Stop" + ); + } + + // xHCI 1.2 Section 4.23.5.2.2 + if cmd.cem_enable() { + slog::error!(self.log, "unimplemented USBCMD: CEM Enable"); + } + + // xHCI 1.2 Section 4.11.2.3 + if cmd.ete() { + slog::error!( + self.log, + "unimplemented USBCMD: ETE (Extended TBC Enable)" + ); + } + + // xHCI 1.2 Section 4.11.2.3 + if cmd.tsc_enable() { + slog::error!( + self.log, + "unimplemented USBCMD: Extended TSC TRB Status Enable" + ); + } + + // LHCRST is optional, and when it is not implemented + // (HCCPARAMS1), it must always return 0 when read. + // CSS and CRS also must always return 0 when read. + state.usbcmd = cmd + .with_host_controller_reset(false) + .with_controller_save_state(false) + .with_controller_restore_state(false) + .with_light_host_controller_reset(false); + + U32(cmd.0) + } + // xHCI 1.2 Section 5.4.2 + Op(UsbStatus) => { + let mut state = self.state.lock().unwrap(); + // HCH, SSS, RSS, CNR, and HCE are read-only (ignored here). + // HSE, EINT, PCD, and SRE are RW1C (guest writes a 1 to + // clear a field to 0, e.g. to ack an interrupt we gave it). + let sts = bits::UsbStatus(wo.read_u32()); + if sts.host_system_error() { + state.usbsts.set_host_system_error(false); + } + if sts.event_interrupt() { + state.usbsts.set_event_interrupt(false); + } + if sts.port_change_detect() { + state.usbsts.set_port_change_detect(false); + } + if sts.save_restore_error() { + state.usbsts.set_save_restore_error(false); + } + U32(sts.0) + } + // Page size is read-only. + Op(PageSize) => RegRWOpValue::NoOp, + // xHCI 1.2 sections 5.4.4, 6.4.2.7. + // Bitfield enabling/disabling Device Notification Events + // when Device Notification Transaction Packets are received + // for each of 16 possible notification types + Op(DeviceNotificationControl) => { + let mut state = self.state.lock().unwrap(); + let val = wo.read_u32(); + state.dnctrl.data[0] = val & 0xFFFFu32; + U32(val) + } + Op(CommandRingControlRegister1) => { + let crcr = bits::CommandRingControl(wo.read_u32() as u64); + let mut state = self.state.lock().unwrap(); + // xHCI 1.2 sections 4.9.3, 5.4.5 + if state.crcr.command_ring_running() { + // xHCI 1.2 table 5-24 + if crcr.command_stop() { + // wait for command ring idle, generate command completion event + let memctx = self.pci_state.acc_mem.access().unwrap(); + doorbell::command_ring_stop( + &mut state, + TrbCompletionCode::CommandRingStopped, + &memctx, + &self.log, + ); + } else if crcr.command_abort() { + // XXX: this doesn't actually abort ongoing processing + let memctx = self.pci_state.acc_mem.access().unwrap(); + doorbell::command_ring_stop( + &mut state, + TrbCompletionCode::CommandAborted, + &memctx, + &self.log, + ); + } else { + slog::error!( + self.log, + "wrote CRCR while running: {crcr:?}" + ); + } + } else { + state.crcr = crcr; + } + U32(crcr.0 as u32) + } + // xHCI 5.1 - 64-bit registers can be written as {lower dword, upper dword}, + // and in CRCR's case this matters, because read-modify-write for each half + // doesn't work when reads are defined to return 0. + Op(CommandRingControlRegister2) => { + let mut state = self.state.lock().unwrap(); + let val = wo.read_u32(); + // xHCI 1.2 sections 4.9.3, 5.4.5 + if !state.crcr.command_ring_running() { + state.crcr.0 &= 0xFFFFFFFFu64; + state.crcr.0 |= (val as u64) << 32; + } + U32(val) + } + Op(DeviceContextBaseAddressArrayPointerRegister) => { + let mut state = self.state.lock().unwrap(); + let dcbaap = GuestAddr(wo.read_u64()); + state.dev_slots.set_dcbaap(dcbaap); + U64(dcbaap.0) + } + Op(Configure) => { + let mut state = self.state.lock().unwrap(); + state.config = bits::Configure(wo.read_u32()); + U32(state.config.0) + } + Op(Port(port_id, regs)) => { + reg_index = port_id.as_raw_id() as i16; + let mut state = self.state.lock().unwrap(); + let port = &mut state.port_regs[port_id.as_index()]; + // all implemented port regs are 32-bit + let value = wo.read_u32(); + match port.reg_write(value, regs, &self.log) { + // xHCI 1.2 sect 4.19.5 + port::PortWrite::BusReset => { + // NOTE: do USB bus reset seq with device in port here. + // USB2 ports are specified as being unable + // to fail the bus reset sequence. + + let memctx = self.pci_state.acc_mem.access().unwrap(); + if let Some(evt) = port.xhc_update_portsc( + &|portsc| { + *portsc = portsc + .with_port_link_state( + bits::PortLinkState::U0, + ) + .with_port_reset(false) + .with_port_enabled_disabled(true) + .with_port_reset_change(true) + .with_port_speed(0); + }, + port_id, + ) { + if let Err(e) = state.interrupters[0] + .enqueue_event(evt, &memctx, false) + { + slog::error!(&self.log, "unable to signal Port Status Change for bus reset: {e}"); + } + } + } + _ => {} + } + U32(value) + } + + // Runtime registers + Runtime(MicroframeIndex) => NoOp, // Read-only + Runtime(Interrupter(i, intr_regs)) => { + reg_index = i as i16; + let mut state = self.state.lock().unwrap(); + let memctx = self.pci_state.acc_mem.access().unwrap(); + state.interrupters[i as usize].reg_write(wo, intr_regs, &memctx) + } + + Doorbell(0) => { + reg_index = 0; + slog::debug!(self.log, "doorbell 0"); + // xHCI 1.2 section 4.9.3, table 5-43 + let doorbell_register = bits::DoorbellRegister(wo.read_u32()); + if doorbell_register.db_target() == 0 { + let mut state = self.state.lock().unwrap(); + // xHCI 1.2 table 5-24: only set to 1 if R/S is 1 + if state.usbcmd.run_stop() { + state.crcr.set_command_ring_running(true); + } + let memctx = self.pci_state.acc_mem.access().unwrap(); + doorbell::process_command_ring( + &mut state, &memctx, &self.log, + ); + } + U32(doorbell_register.0) + } + // xHCI 1.2 section 4.7 + Doorbell(slot_id) => { + reg_index = slot_id as i16; + // TODO: care about DoorbellRegister::db_stream_id for USB3 + let doorbell_register = bits::DoorbellRegister(wo.read_u32()); + let endpoint_id = doorbell_register.db_target(); + slog::debug!( + self.log, + "doorbell slot {slot_id} ep {endpoint_id}" + ); + let mut state = self.state.lock().unwrap(); + let memctx = self.pci_state.acc_mem.access().unwrap(); + doorbell::process_transfer_ring( + &mut state, + // safe: only valid slot ids in reg map + SlotId::from(slot_id), + endpoint_id, + &memctx, + &self.log, + ); + U32(doorbell_register.0) + } + + // read-only + ExtCap(_) => NoOp, + }; + + let reg_value = written_value.as_u64(); + let reg_name = id.reg_name(); + probes::xhci_reg_write!(|| (reg_name, reg_value, reg_index)); + } + + // xHCI 1.2 sect 4.14.2 - generate a MFINDEX Wrap Event every time MFINDEX's + // microframe value wraps from 0x3FFF to 0. + // + // state.mfindex_wrap_thread_generation controls when the thread will + // terminate its loop (as will losing its Weak>). + // Both this method and stop_mfindex_wrap_thread shall increment + // the generation number. The latter is called in conditions where we should + // no longer generate events - i.e. 0 written to R/S or EWE - so we do not + // check those here. + // + // The XhciState mutex is necessarily held through this method call, and is + // acquired by the started thread each time it wishes to enqueue an event. + fn start_mfindex_wrap_thread(&self, state: &mut XhciState) { + if let Some(run_start_arc) = state.run_start.as_ref() { + let initial_value = state.mfindex.microframe() as u32; + let hdl = state.vmm_hdl.clone(); + let first_wrap_time = run_start_arc + .checked_add( + (bits::MFINDEX_WRAP_POINT - initial_value) + * bits::MINIMUM_INTERVAL_TIME, + ) + .unwrap_or_else(|| { + slog::error!( + self.log, + "unrepresentable instant in mfindex wrap" + ); + // fudge the numbers a bit and maintain periodicity + time::VmGuestInstant::now(&hdl).unwrap() + }); + + let generation = + state.mfindex_wrap_thread_generation.wrapping_add(1); + state.mfindex_wrap_thread_generation = generation; + + let state_weak = Arc::downgrade(&self.state); + let acc_mem = self + .pci_state + .acc_mem + .child(Some("MFINDEX Wrap Event thread".to_string())); + + state.mfindex_wrap_thread = Some(std::thread::spawn(move || { + use rings::producer::event::EventInfo; + let mut wraps = 0; + loop { + const WRAP_INTERVAL: Duration = bits::MINIMUM_INTERVAL_TIME + .checked_mul(bits::MFINDEX_WRAP_POINT) + .unwrap(); + if let Some(deadline) = + first_wrap_time.checked_add(WRAP_INTERVAL * wraps) + { + wraps += 1; + let now = time::VmGuestInstant::now(&hdl).unwrap(); + if let Some(delay) = + deadline.checked_duration_since(now) + { + std::thread::sleep(delay); + } + } + // enqueue event + let Some(state_arc) = state_weak.upgrade() else { + break; + }; + let Ok(mut state) = state_arc.lock() else { + break; + }; + if state.mfindex_wrap_thread_generation == generation { + let memctx = acc_mem.access().unwrap(); + state.interrupters[0] + .enqueue_event( + EventInfo::MfIndexWrap, + &memctx, + false, + ) + .ok(); // shall be dropped by the xHC if Event Ring full + } else { + break; + } + } + })); + } + } + + fn stop_mfindex_wrap_thread(&self, state: &mut XhciState) { + state.mfindex_wrap_thread_generation = + state.mfindex_wrap_thread_generation.wrapping_add(1); + state.mfindex_wrap_thread = None; + } +} + +impl Lifecycle for PciXhci { + fn type_name(&self) -> &'static str { + "pci-xhci" + } + fn migrate(&self) -> Migrator { + Migrator::Multi(self) + } +} + +impl pci::Device for PciXhci { + fn device_state(&self) -> &pci::DeviceState { + &self.pci_state + } + + fn cfg_rw(&self, region: u8, mut rwo: RWOp) { + assert_eq!(region, bits::USB_PCI_CFG_OFFSET); + + USB_PCI_CFG_REGS.process( + &mut rwo, + |id: &UsbPciCfgReg, rwo: RWOp<'_, '_>| match rwo { + RWOp::Read(ro) => self.usb_cfg_read(*id, ro), + RWOp::Write(wo) => self.usb_cfg_write(*id, wo), + }, + ) + } + + fn bar_rw(&self, bar: pci::BarN, mut rwo: RWOp) { + assert_eq!(bar, pci::BarN::BAR0); + XHC_REGS.map.process(&mut rwo, |id: &Registers, rwo: RWOp<'_, '_>| { + match rwo { + RWOp::Read(ro) => self.reg_read(*id, ro), + RWOp::Write(wo) => self.reg_write(*id, wo), + } + }) + } + + fn interrupt_mode_change(&self, mode: pci::IntrMode) { + let mut state = self.state.lock().unwrap(); + for interrupter in &mut state.interrupters { + interrupter.set_pci_intr_mode(mode, &self.pci_state); + } + } +} + +impl MigrateMulti for PciXhci { + fn export( + &self, + output: &mut crate::migrate::PayloadOutputs, + ctx: &crate::migrate::MigrateCtx, + ) -> Result<(), crate::migrate::MigrateStateError> { + let mut state = self.state.lock().unwrap(); + state.apply_ip_raise_to_usbsts_eint(); + + let XhciState { + usbcmd, + usbsts, + dnctrl, + mfindex, + run_start, + mfindex_wrap_thread, + mfindex_wrap_thread_generation, + interrupters, + any_interrupt_pending_raised: _, + command_ring, + crcr, + dev_slots, + config, + port_regs, + evt_data_xfer_len_accum, + queued_device_connections, + vmm_hdl: _, + } = &*state; + + let mstate = migrate::XhciStateV1 { + usbcmd: usbcmd.0, + usbsts: usbsts.0, + dnctrl: dnctrl.load_le(), + crcr: crcr.0, + mfindex: mfindex.0, + config: config.0, + evt_data_xfer_len_accum: *evt_data_xfer_len_accum, + run_start: run_start.as_ref().map(|x| **x), + mfindex_wrap_thread: mfindex_wrap_thread.is_some(), + mfindex_wrap_thread_generation: *mfindex_wrap_thread_generation, + interrupters: interrupters + .iter() + .map(|xi| xi.export()) + .collect::, _>>()?, + command_ring: command_ring.as_ref().map(|cr| cr.export()), + dev_slots: dev_slots.export(), + port_regs: port_regs.iter().map(|port| port.export()).collect(), + queued_device_connections: queued_device_connections + .iter() + .map(|(port_id, usbdev)| (port_id.as_raw_id(), usbdev.export())) + .collect(), + }; + + output.push(mstate.into())?; + MigrateMulti::export(pci::Device::device_state(self), output, ctx)?; + Ok(()) + } + + fn import( + &self, + offer: &mut crate::migrate::PayloadOffers, + ctx: &crate::migrate::MigrateCtx, + ) -> Result<(), crate::migrate::MigrateStateError> { + let migrate::XhciStateV1 { + usbcmd, + usbsts, + dnctrl, + crcr, + mfindex, + config, + evt_data_xfer_len_accum, + run_start, + mfindex_wrap_thread, + mfindex_wrap_thread_generation, + interrupters, + command_ring, + dev_slots, + queued_device_connections, + port_regs, + } = offer.take()?; + + let mut state = self.state.lock().unwrap(); + + if state.port_regs.len() != port_regs.len() { + return Err(crate::migrate::MigrateStateError::ImportFailed( + format!( + "port_regs count mismatch: {} != {}", + state.port_regs.len(), + port_regs.len() + ), + )); + } + if state.interrupters.len() != interrupters.len() { + return Err(crate::migrate::MigrateStateError::ImportFailed( + format!( + "interrupters count mismatch: {} != {}", + state.interrupters.len(), + interrupters.len() + ), + )); + } + + state.usbcmd = bits::UsbCommand(usbcmd); + state.usbsts = bits::UsbStatus(usbsts); + state.dnctrl.store_le(dnctrl); + state.crcr = bits::CommandRingControl(crcr); + state.mfindex = bits::MicroframeIndex(mfindex); + state.config = bits::Configure(config); + state.evt_data_xfer_len_accum = evt_data_xfer_len_accum; + state.run_start = run_start.map(|x| Arc::new(x)); + + state.command_ring = command_ring + .map(|cr| { + CommandRing::try_from(&cr).map_err(|e| { + crate::migrate::MigrateStateError::ImportFailed(format!( + "{e:?}" + )) + }) + }) + .transpose()?; + state.dev_slots.import(&dev_slots)?; + state.queued_device_connections = queued_device_connections + .into_iter() + .map(|(port_id, dev_data)| { + let mut dev = NullUsbDevice::default(); + dev.import(&dev_data)?; + let port_id = PortId::try_from(port_id) + .map_err(crate::migrate::MigrateStateError::ImportFailed)?; + Ok::<_, crate::migrate::MigrateStateError>((port_id, dev)) + }) + .collect::, _>>()?; + + for (pr, pr_data) in state.port_regs.iter_mut().zip(port_regs) { + pr.import(&pr_data)?; + } + for (intr, intr_data) in state.interrupters.iter_mut().zip(interrupters) + { + intr.import(intr_data)?; + } + + if state.mfindex_wrap_thread.is_some() && !mfindex_wrap_thread { + // stop_mfindex_wrap_thread will increment generation; decrement it here + state.mfindex_wrap_thread_generation = + mfindex_wrap_thread_generation.wrapping_sub(1); + self.stop_mfindex_wrap_thread(&mut state); + } else if mfindex_wrap_thread + && (state.mfindex_wrap_thread.is_none() // thread has not been started + || state.mfindex_wrap_thread_generation // running thread is stale + != mfindex_wrap_thread_generation) + { + // start_mfindex_wrap_thread will increment generation; decrement it here + state.mfindex_wrap_thread_generation = + mfindex_wrap_thread_generation.wrapping_sub(1); + self.start_mfindex_wrap_thread(&mut state); + } else { + // thread is not running, or generation already matches + state.mfindex_wrap_thread_generation = + mfindex_wrap_thread_generation; + } + + drop(state); + + MigrateMulti::import(&self.pci_state, offer, ctx)?; + self.interrupt_mode_change(self.pci_state.get_intr_mode()); + Ok(()) + } +} + +pub mod migrate { + use crate::migrate::*; + use serde::{Deserialize, Serialize}; + + use super::interrupter::migrate::XhciInterrupterV1; + use super::port::migrate::XhcUsbPortV1; + use super::rings::consumer::migrate::ConsumerRingV1; + use crate::hw::usb::usbdev::migrate::UsbDeviceV1; + + #[derive(Deserialize, Serialize)] + pub struct XhciStateV1 { + pub usbcmd: u32, + pub usbsts: u32, + pub dnctrl: u32, + pub crcr: u64, + pub mfindex: u32, + pub config: u32, + pub evt_data_xfer_len_accum: u32, + pub run_start: Option, + pub mfindex_wrap_thread: bool, + pub mfindex_wrap_thread_generation: u32, + pub interrupters: Vec, + pub command_ring: Option, + pub dev_slots: super::device_slots::migrate::DeviceSlotTableV1, + pub queued_device_connections: Vec<(u8, UsbDeviceV1)>, + pub port_regs: Vec, + } + + impl Schema<'_> for XhciStateV1 { + fn id() -> SchemaId { + ("pci-xhci", 1) + } + } +} diff --git a/lib/propolis/src/hw/usb/xhci/device_slots.rs b/lib/propolis/src/hw/usb/xhci/device_slots.rs new file mode 100644 index 000000000..59378d715 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/device_slots.rs @@ -0,0 +1,1020 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use std::collections::HashMap; +use std::ops::Deref; +use zerocopy::{FromBytes, FromZeros}; + +use crate::common::GuestAddr; +use crate::hw::usb::usbdev::demo_state_tracker::NullUsbDevice; +use crate::vmm::MemCtx; + +use super::bits::device_context::{ + EndpointContext, EndpointState, InputControlContext, SlotContext, + SlotContextFirst, SlotState, +}; +use super::bits::ring_data::TrbCompletionCode; +use super::port::PortId; +use super::rings::consumer::transfer::TransferRing; +use super::{MAX_DEVICE_SLOTS, MAX_PORTS}; + +#[derive(Debug, thiserror::Error)] +pub enum Error { + #[error("Cannot use 0-value SlotId as array index")] + SlotIdZero, + #[error("{0:?} greater than max device slots ({MAX_DEVICE_SLOTS})")] + SlotIdBounds(SlotId), + #[error("USB device not found in {0:?}")] + NoUsbDevInSlot(SlotId), + #[error("{0:?} not enabled")] + SlotNotEnabled(SlotId), + #[error("No port associated with {0:?}")] + NoPortAssociatedWithSlot(SlotId), +} + +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +#[repr(transparent)] +pub struct SlotId(u8); + +impl From for SlotId { + fn from(value: u8) -> Self { + Self(value) + } +} + +impl From for u8 { + fn from(value: SlotId) -> Self { + value.0 + } +} + +impl SlotId { + pub fn as_index(&self) -> Result { + if self.0 <= MAX_DEVICE_SLOTS { + (self.0 as usize).checked_sub(1).ok_or(Error::SlotIdZero) + } else { + Err(Error::SlotIdBounds(*self)) + } + } +} + +struct MemCtxValue<'a, T: Copy + FromBytes> { + value: T, + addr: GuestAddr, + memctx: &'a MemCtx, +} + +impl core::fmt::Debug + for MemCtxValue<'_, T> +{ + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + core::fmt::Debug::fmt(&self.value, f) + } +} + +impl<'a, T: Copy + FromBytes> MemCtxValue<'a, T> { + pub fn new(addr: GuestAddr, memctx: &'a MemCtx) -> Option { + memctx.read(addr).map(move |value| Self { value: *value, addr, memctx }) + } + pub fn mutate(&mut self, mut f: impl FnMut(&mut T)) -> bool { + f(&mut self.value); + self.memctx.write(self.addr, &self.value) + } +} + +impl Deref for MemCtxValue<'_, T> { + type Target = T; + fn deref(&self) -> &Self::Target { + &self.value + } +} + +struct DeviceSlot { + endpoints: HashMap, + port_address: Option, +} +impl DeviceSlot { + fn new() -> Self { + Self { endpoints: HashMap::new(), port_address: None } + } + + fn set_endpoint_tr(&mut self, endpoint_id: u8, ep_ctx: EndpointContext) { + self.endpoints.insert( + endpoint_id, + // unwrap: tr_dequeue_pointer's lower 4 bits are 0, will always be TRB-aligned + TransferRing::new( + ep_ctx.tr_dequeue_pointer(), + ep_ctx.dequeue_cycle_state(), + ) + .unwrap(), + ); + } + + fn unset_endpoint_tr(&mut self, endpoint_id: u8) { + self.endpoints.remove(&endpoint_id); + } + + fn import( + &mut self, + value: &migrate::DeviceSlotV1, + ) -> Result<(), crate::migrate::MigrateStateError> { + let migrate::DeviceSlotV1 { endpoints, port_address } = value; + self.port_address = port_address + .map(PortId::try_from) + .transpose() + .map_err(crate::migrate::MigrateStateError::ImportFailed)?; + for ep_id in self.endpoints.keys().copied().collect::>() { + if !endpoints.contains_key(&ep_id) { + self.endpoints.remove(&ep_id); + } + } + for (ep_id, ep) in endpoints { + self.endpoints.insert( + *ep_id, + TransferRing::try_from(ep).map_err(|e| { + crate::migrate::MigrateStateError::ImportFailed(format!( + "{e:?}" + )) + })?, + ); + } + Ok(()) + } + + fn export(&self) -> migrate::DeviceSlotV1 { + let Self { endpoints, port_address } = self; + migrate::DeviceSlotV1 { + endpoints: endpoints + .iter() + .map(|(ep_id, ep)| (*ep_id, ep.export())) + .collect(), + port_address: port_address.map(|port_id| port_id.as_raw_id()), + } + } +} + +pub struct DeviceSlotTable { + /// Device Context Base Address Array Pointer (DCBAAP) + /// + /// Points to an array of address pointers referencing the device context + /// structures for each attached device. + /// + /// See xHCI 1.2 Section 5.4.6 + dcbaap: Option, + slots: Vec>, + port_devs: [Option; MAX_PORTS as usize], + log: slog::Logger, +} + +impl DeviceSlotTable { + pub fn new(log: slog::Logger) -> Self { + Self { + dcbaap: None, + slots: vec![], + // seemingly asinine, but NullUsbDevice: !Copy, + // which invalidates Option<> + port_devs: [None, None, None, None, None, None, None, None], + log, + } + } +} + +impl DeviceSlotTable { + pub fn set_dcbaap(&mut self, addr: GuestAddr) { + self.dcbaap = Some(addr) + } + pub fn dcbaap(&self) -> Option<&GuestAddr> { + self.dcbaap.as_ref() + } + + pub fn attach_to_root_hub_port_address( + &mut self, + port_id: PortId, + usb_dev: NullUsbDevice, + ) -> Result<(), NullUsbDevice> { + if let Some(dev) = self.port_devs[port_id.as_index()].replace(usb_dev) { + Err(dev) + } else { + Ok(()) + } + } + + pub fn detach_all_for_reset( + &mut self, + ) -> impl Iterator + '_ { + self.port_devs.iter_mut().enumerate().flat_map(|(i, opt_dev)| { + opt_dev + .take() + .map(|dev| (PortId::try_from(i as u8 + 1).unwrap(), dev)) + }) + } + + pub fn usbdev_for_slot( + &mut self, + slot_id: SlotId, + ) -> Result<&mut NullUsbDevice, Error> { + let slot = self.slot_mut(slot_id)?; + let idx = slot + .port_address + .ok_or(Error::NoPortAssociatedWithSlot(slot_id))?; + self.port_devs[idx.as_index()] + .as_mut() + .ok_or(Error::NoUsbDevInSlot(slot_id)) + } + + fn slot(&self, slot_id: SlotId) -> Result<&DeviceSlot, Error> { + self.slots + .get(slot_id.as_index()?) + .and_then(|res| res.as_ref()) + .ok_or(Error::SlotNotEnabled(slot_id)) + } + + fn slot_mut(&mut self, slot_id: SlotId) -> Result<&mut DeviceSlot, Error> { + self.slots + .get_mut(slot_id.as_index()?) + .and_then(|res| res.as_mut()) + .ok_or(Error::SlotNotEnabled(slot_id)) + } + + fn endpoint_context( + slot_addr: GuestAddr, + endpoint_id: u8, + memctx: &MemCtx, + ) -> Option> { + const { assert!(size_of::() == size_of::()) }; + MemCtxValue::new( + slot_addr + .offset::(1) + .offset::(endpoint_id.checked_sub(1)? as usize), + memctx, + ) + } + + pub fn enable_slot(&mut self, slot_type: u8) -> Option { + // USB protocol slot type is 0 (xHCI 1.2 section 7.2.2.1.4) + if slot_type != 0 { + return None; + } + let slot_id_opt = self + .slots + .iter() + .position(Option::is_none) + .map(|i| SlotId::from(i as u8 + 1)) + .or_else(|| { + if self.slots.len() < MAX_DEVICE_SLOTS as usize { + self.slots.push(None); + Some(SlotId::from(self.slots.len() as u8)) + } else { + None + } + }); + if let Some(slot_id) = slot_id_opt { + self.slots[slot_id.as_index().unwrap()] = Some(DeviceSlot::new()); + } + slot_id_opt + } + + pub fn disable_slot( + &mut self, + slot_id: SlotId, + memctx: &MemCtx, + ) -> TrbCompletionCode { + if self.slot(slot_id).is_ok() { + // terminate any transfers on the slot + + // slot ctx is first element of dev context table + if let Some(slot_ptr) = self.dev_context_addr(slot_id, memctx) { + if let Some(mut slot_ctx) = + MemCtxValue::::new(slot_ptr, memctx) + { + slot_ctx.mutate(|ctx| { + ctx.set_slot_state(SlotState::DisabledEnabled) + }); + } + } + // unwrap: self.slot(slot_id).is_ok(), above + self.slots[slot_id.as_index().unwrap()] = None; + TrbCompletionCode::Success + } else { + TrbCompletionCode::SlotNotEnabledError + } + } + + fn dev_context_addr( + &self, + slot_id: SlotId, + memctx: &MemCtx, + ) -> Option { + self.dcbaap.as_ref().and_then(|base_ptr| { + memctx + .read::( + // lower 6 bits are reserved (xHCI 1.2 table 6-2) + // software is supposed to clear them to 0, but + // let's double-tap for safety's sake. + (*base_ptr & !0b11_1111) + // NOTE: index 0 *is* reserved in DCBAA, so we don't SlotId::as_index() + .offset::(u8::from(slot_id) as usize), + ) + .map(|x| GuestAddr(*x)) + }) + } + + /// xHCI 1.2 sect 4.6.5 + pub fn address_device( + &mut self, + slot_id: SlotId, + input_context_ptr: GuestAddr, + block_set_address_request: bool, + memctx: &MemCtx, + ) -> Option { + if self.slot(slot_id).is_err() { + return Some(TrbCompletionCode::SlotNotEnabledError); + } + + // xHCI 1.2 Figure 6-5: differences between input context indeces + // and device context indeces. + + let out_slot_addr = self.dev_context_addr(slot_id, memctx)?; + let out_ep0_addr = out_slot_addr.offset::(1); + + let in_slot_addr = input_context_ptr.offset::(1); + let in_ep0_addr = in_slot_addr.offset::(1); + + let mut slot_ctx = memctx.read::(in_slot_addr)?; + let mut ep0_ctx = memctx.read::(in_ep0_addr)?; + + // we'll just use the root hub port number as the USB address for + // whatever's in this slot + if let Ok(port_id) = slot_ctx.root_hub_port_number() { + // unwrap: we've checked is_none already + self.slot_mut(slot_id).unwrap().port_address = Some(port_id); + } + + let usb_addr = if block_set_address_request { + if matches!(slot_ctx.slot_state(), SlotState::DisabledEnabled) { + // set output slot context state to default + slot_ctx.set_slot_state(SlotState::Default); + + 0 // because BSR, just set field to 0 in slot ctx + } else { + return Some(TrbCompletionCode::ContextStateError); + } + } else if matches!( + slot_ctx.slot_state(), + SlotState::DisabledEnabled | SlotState::Default + ) { + // we'll just use the root hub port number as our USB address + if let Some(port_id) = + self.slot(slot_id).ok().and_then(|slot| slot.port_address) + { + // TODO: issue 'set address' to USB device itself + + // set output slot context state to addressed + slot_ctx.set_slot_state(SlotState::Addressed); + + port_id.as_raw_id() + } else { + // root hub port ID in slot ctx not in range of our ports + return Some(TrbCompletionCode::ContextStateError); + } + } else { + return Some(TrbCompletionCode::ContextStateError); + }; + + // set usb device address in output slot ctx to chosen addr (or 0 if BSR) + slot_ctx.set_usb_device_address(usb_addr); + // copy input slot ctx to output slot ctx + memctx.write(out_slot_addr, &slot_ctx); + + // set output ep0 state to running + ep0_ctx.set_endpoint_state(EndpointState::Running); + // copy input ep0 ctx to output ep0 ctx + memctx.write(out_ep0_addr, &ep0_ctx); + + slog::debug!( + self.log, + "slot_ctx: in@{:#x} out@{:#x} {slot_ctx:?}", + in_slot_addr.0, + out_slot_addr.0 + ); + slog::debug!( + self.log, + "ep0_ctx: in@{:#x} out@{:#x} {ep0_ctx:?}", + in_ep0_addr.0, + out_ep0_addr.0 + ); + + // unwrap: we've checked self.slot() at function begin, + // we just can't hold a &mut for the whole duration + let device_slot = self.slot_mut(slot_id).unwrap(); + + // add default control endpoint to scheduling list + device_slot.set_endpoint_tr(1, *ep0_ctx); + + Some(TrbCompletionCode::Success) + } + + /// See xHCI 1.2 sect 3.3.5, 4.6.6, 6.2.3.2, and figure 4-3. + // NOTE: if we were concerned with bandwidth/resource limits + // as real hardware is, this function would + // - add/subtract resources allocated to the endpoint from a + // Resource Required variable + // - if endpoint is periodic, add/subtract bandwidth allocated to the + // endpoint from a Bandwidth Required variable + // in various points throughout the function where contexts are added + // and dropped. these steps (as outlined in xHCI 1.2 sect 4.6.6) + // are elided, since we're a virtual machine and can both bend reality + // to our will and (at time of writing) control what devices are + // allowed to be connected to us in the first place. + pub fn configure_endpoint( + &mut self, + input_context_ptr: GuestAddr, + slot_id: SlotId, + deconfigure: bool, + memctx: &MemCtx, + ) -> Option { + // following xHC behavior described in xHCI 1.2 sect 4.6.6: + // if not previously enabled by an Enable Slot command + if self.slot(slot_id).is_err() { + return Some(TrbCompletionCode::SlotNotEnabledError); + } + + // retrieve the output device context of the selected device slot + let out_slot_addr = self.dev_context_addr(slot_id, memctx)?; + let mut out_slot_ctx = + MemCtxValue::::new(out_slot_addr, memctx)?; + + // if output dev context slot state is not addressed or configured + if !matches!( + out_slot_ctx.slot_state(), + SlotState::Addressed | SlotState::Configured + ) { + return Some(TrbCompletionCode::ContextStateError); + } + + // setting deconfigure (DC) is equivalent to setting Input Context + // Drop Context flags 2..=31 to 1 and Add Context flags 2..=31 to 0. + // if DC=1, Input Context Pointer field shall be *ignored* by the xHC, + // the Output Slot Context 'Context Entries' field shall be set to 1. + let input_ctx = if deconfigure { + let mut in_ctx = InputControlContext::new_zeroed(); + for i in 2..=31 { + in_ctx.set_add_context_bit(i, false); + in_ctx.set_drop_context_bit(i, true); + } + out_slot_ctx.mutate(|out_ctx| { + out_ctx.set_context_entries(1); + }); + in_ctx + } else { + *memctx.read::(input_context_ptr)? + }; + + // if the output slot state is configured + if deconfigure { + if out_slot_ctx.slot_state() == SlotState::Configured { + // for each endpoint context not in disabled state: + for i in 1..=31 { + let mut out_ep_ctx = + Self::endpoint_context(out_slot_addr, i, memctx)?; + if out_ep_ctx.endpoint_state() != EndpointState::Disabled { + // set output EP State field to disabled + out_ep_ctx.mutate(|ctx| { + ctx.set_endpoint_state(EndpointState::Disabled) + }); + // XXX: is this right? + self.slot_mut(slot_id).unwrap().unset_endpoint_tr(i); + } + } + // set Slot State in output slot context to Addressed + out_slot_ctx + .mutate(|ctx| ctx.set_slot_state(SlotState::Addressed)); + } + } + // if output slot state is addressed or configured and DC=0 + // (for slot state, we've already returned ContextStateError otherwise) + else { + let in_slot_addr = + input_context_ptr.offset::(1); + + // for each endpoint context designated by a Drop Context flag = 2 + for i in 2..=31 { + // unwrap: only None when index not in 2..=31 + if input_ctx.drop_context_bit(i).unwrap() { + let mut out_ep_ctx = + Self::endpoint_context(out_slot_addr, i, memctx)?; + // set output EP State field to disabled + out_ep_ctx.mutate(|ctx| { + ctx.set_endpoint_state(EndpointState::Disabled) + }); + // XXX: is this right? + self.slot_mut(slot_id).unwrap().unset_endpoint_tr(i); + } + } + + // if all input endpoint contexts with Add Context = 1 are valid + for i in 1..=31 { + if input_ctx.add_context_bit(i).unwrap() { + let in_ep_ctx = + Self::endpoint_context(in_slot_addr, i, memctx)?; + // not all input ep contexts valid -> Parameter Error + if !in_ep_ctx.valid_for_configure_endpoint() { + return Some(TrbCompletionCode::ParameterError); + } + } + } + + let mut any_endpoint_enabled = false; + // for each endpoint context designated by an Add Context flag = 1 + for i in 1..=31 { + let mut out_ep_ctx = + Self::endpoint_context(out_slot_addr, i, memctx)?; + + // unwrap: only None when index > 31 + if input_ctx.add_context_bit(i).unwrap() { + // copy all fields of input ep context to output ep context; + // set output EP state field to running. + let in_ep_ctx = + Self::endpoint_context(in_slot_addr, i, memctx)?; + out_ep_ctx.mutate(|ctx| { + *ctx = *in_ep_ctx; + ctx.set_endpoint_state(EndpointState::Running); + }); + // load the xHC enqueue and dequeue pointers with the + // value of TR Dequeue Pointer field from Endpoint Context + let device_slot = self.slot_mut(slot_id).unwrap(); + device_slot.set_endpoint_tr(i, *out_ep_ctx); + } + + if out_ep_ctx.endpoint_state() != EndpointState::Disabled { + any_endpoint_enabled = true; + } + } + + // if all Endpoints are Disabled + if !any_endpoint_enabled { + out_slot_ctx.mutate(|ctx| { + // set Slot State in Output Slot Context to Addressed + ctx.set_slot_state(SlotState::Addressed); + // set Context Entries in Output Slot Context to 1 + ctx.set_context_entries(1); + }); + } else { + out_slot_ctx.mutate(|ctx| { + // set Slot State in Output Slot Context to Configured + ctx.set_slot_state(SlotState::Configured); + // set Context Entries to the index of the last valid + // Endpoint Context in the Output Device Context + }); + } + } + + Some(TrbCompletionCode::Success) + } + + /// xHCI 1.2 sect 4.6.7, 6.2.2.3, 6.2.3.3 + pub fn evaluate_context( + &self, + slot_id: SlotId, + input_context_ptr: GuestAddr, + memctx: &MemCtx, + ) -> Option { + if self.slot(slot_id).is_err() { + return Some(TrbCompletionCode::SlotNotEnabledError); + } + + let input_ctx = + memctx.read::(input_context_ptr)?; + + // retrieve the output device context of the selected device slot + let out_slot_addr = self.dev_context_addr(slot_id, memctx)?; + + let mut out_slot_ctx = + MemCtxValue::::new(out_slot_addr, memctx)?; + Some(match out_slot_ctx.slot_state() { + // if the output slot state is default, addressed, or configured: + SlotState::Default + | SlotState::Addressed + | SlotState::Configured => { + slog::debug!( + self.log, + "input_ctx: {:#x} {input_ctx:?}", + input_context_ptr.0 + ); + + let in_slot_addr = + input_context_ptr.offset::(1); + + // for each context designated by an add context flag = 1, + // evaluate the parameter settings defined by the selected contexts. + // (limited to context indeces 0 and 1, per xHCI 1.2 sect 6.2.3.3) + + // xHCI 1.2 sect 6.2.2.3: interrupter target & max exit latency + if input_ctx.add_context_bit(0).unwrap() { + let in_slot_ctx = + memctx.read::(in_slot_addr)?; + out_slot_ctx.mutate(|ctx| { + ctx.set_interrupter_target( + in_slot_ctx.interrupter_target(), + ); + ctx.set_max_exit_latency_micros( + in_slot_ctx.max_exit_latency_micros(), + ); + }); + slog::debug!( + self.log, + "out_slot_ctx: in@{:#x} out@{:#x} {out_slot_ctx:?}", + in_slot_addr.0, + out_slot_addr.0 + ); + } + // xHCI 1.2 sect 6.2.3.3: pay attention to max packet size + if input_ctx.add_context_bit(1).unwrap() { + let in_ep0_addr = input_context_ptr + .offset::(1) + .offset::(1); + let out_ep0_addr = out_slot_addr.offset::(1); + + let in_ep0_ctx = + memctx.read::(in_ep0_addr)?; + + let mut out_ep0_ctx = + Self::endpoint_context(out_slot_addr, 1, memctx)?; + out_ep0_ctx.mutate(|ctx| { + ctx.set_max_packet_size(in_ep0_ctx.max_packet_size()) + }); + + slog::debug!( + self.log, + "out_slot_ctx: {:#x} {out_slot_ctx:?}", + out_slot_addr.0 + ); + slog::debug!( + self.log, + "out_ep0_ctx: in@{:#x} out@{:#x} {out_ep0_ctx:?}", + in_ep0_addr.0, + out_ep0_addr.0 + ); + } + // (again, per xHCI 1.2 sect 6.2.3.3: contexts 2 through 31 + // are not evaluated by this command.) + TrbCompletionCode::Success + } + x => { + slog::error!( + self.log, + "Evaluate Context for slot in state {x:?}" + ); + TrbCompletionCode::ContextStateError + } + }) + } + + // xHCI 1.2 sect 4.6.8 + pub fn reset_endpoint( + &self, + slot_id: SlotId, + endpoint_id: u8, + transfer_state_preserve: bool, + memctx: &MemCtx, + ) -> Option { + let slot_addr = self.dev_context_addr(slot_id, memctx)?; + + let mut ep_ctx = + Self::endpoint_context(slot_addr, endpoint_id, memctx)?; + Some(match ep_ctx.endpoint_state() { + EndpointState::Halted => TrbCompletionCode::ContextStateError, + _ => { + if transfer_state_preserve { + // TODO: + // retry last transaction the next time the doorbell is rung, + // if no other commands have been issued to the endpoint + } else { + // TODO: + // reset data toggle for usb2 device / sequence number for usb3 device + // reset any usb2 split transaction state on this endpoint + // invalidate cached Transfer TRBs + } + ep_ctx.mutate(|ctx| { + ctx.set_endpoint_state(EndpointState::Stopped) + }); + + TrbCompletionCode::Success + } + }) + } + + // xHCI 1.2 sect 4.6.9 + pub fn stop_endpoint( + &mut self, + slot_id: SlotId, + endpoint_id: u8, + _suspend: bool, + memctx: &MemCtx, + ) -> Option { + // TODO: spec says to also insert a Transfer Event to the Event Ring + // if we interrupt the execution of a Transfer Descriptor, but we at + // present cannot interrupt TD execution. + + // if enabled by previous enable slot command + Some(if self.slot(slot_id).is_ok() { + // retrieve dev ctx + let slot_addr = self.dev_context_addr(slot_id, memctx)?; + let output_slot_ctx = memctx.read::(slot_addr)?; + match output_slot_ctx.slot_state() { + SlotState::Default + | SlotState::Addressed + | SlotState::Configured => { + let mut ep_ctx = + Self::endpoint_context(slot_addr, endpoint_id, memctx)?; + match ep_ctx.endpoint_state() { + EndpointState::Running => { + // TODO: + // stop USB activity for pipe + // stop transfer ring activity for pipe + + // write dequeue pointer value to output endpoint tr dequeue pointer field + // write ccs value to output endpoint dequeue cycle state field + + // unwrap: self.slot(slot_id).is_ok(), above + let slot = self.slot_mut(slot_id).unwrap(); + if let Some(evt_ring) = + slot.endpoints.get_mut(&endpoint_id) + { + ep_ctx.mutate(|ctx| { + ctx.set_tr_dequeue_pointer( + evt_ring.current_dequeue_pointer(), + ); + ctx.set_dequeue_cycle_state( + evt_ring.consumer_cycle_state(), + ); + }); + } + + // TODO: remove endpoint from pipe schedule + + // set ep state to stopped + ep_ctx.mutate(|ctx| { + ctx.set_endpoint_state(EndpointState::Stopped) + }); + + // TODO: wait for any partially completed split transactions + TrbCompletionCode::Success + } + x => { + slog::error!( + self.log, + "Stop Endpoint for endpoint in state {x:?}" + ); + TrbCompletionCode::ContextStateError + } + } + } + x => { + slog::error!( + self.log, + "Stop Endpoint for slot in state {x:?}" + ); + TrbCompletionCode::ContextStateError + } + } + } else { + TrbCompletionCode::SlotNotEnabledError + }) + } + + // xHCI 1.2 sect 4.6.10 + pub fn set_tr_dequeue_pointer( + &mut self, + new_tr_dequeue_ptr: GuestAddr, + slot_id: SlotId, + endpoint_id: u8, + dequeue_cycle_state: bool, + memctx: &MemCtx, + ) -> Option { + // if enabled by previous enable slot command + Some(if self.slot(slot_id).is_ok() { + // retrieve dev ctx + let slot_addr = self.dev_context_addr(slot_id, memctx)?; + let output_slot_ctx = memctx.read::(slot_addr)?; + match output_slot_ctx.slot_state() { + SlotState::Default + | SlotState::Addressed + | SlotState::Configured => { + let mut ep_ctx = + Self::endpoint_context(slot_addr, endpoint_id, memctx)?; + // TODO(USB3): cmd trb decode currently assumes MaxPStreams and StreamID are 0 + match ep_ctx.endpoint_state() { + EndpointState::Stopped | EndpointState::Error => { + // copy new_tr_dequeue_ptr to target Endpoint Context + // copy dequeue_cycle_state to target Endpoint Context + ep_ctx.mutate(|ctx| { + ctx.set_tr_dequeue_pointer(new_tr_dequeue_ptr); + ctx.set_dequeue_cycle_state(dequeue_cycle_state) + }); + + // unwrap: if self.slot(slot_id).is_ok(), above + let slot = self.slot_mut(slot_id).unwrap(); + if let Some(endpoint) = + slot.endpoints.get_mut(&endpoint_id) + { + if let Err(e) = endpoint + .set_dequeue_pointer_and_cycle( + new_tr_dequeue_ptr, + dequeue_cycle_state, + ) + { + slog::error!(self.log, "Error setting Transfer Ring's dequeue pointer and cycle bit for {slot_id:?}, endpoint {endpoint_id}: {e}"); + } + } else { + slog::error!(self.log, "can't set Transfer Ring's dequeue pointer and cycle bit for {slot_id:?}'s nonexistent endpoint {endpoint_id}"); + } + + TrbCompletionCode::Success + } + _ => TrbCompletionCode::ContextStateError, + } + } + _ => TrbCompletionCode::ContextStateError, + } + } else { + TrbCompletionCode::SlotNotEnabledError + }) + } + + // xHCI 1.2 sect 4.6.11 + pub fn reset_device( + &self, + slot_id: SlotId, + memctx: &MemCtx, + ) -> Option { + let slot_addr = self.dev_context_addr(slot_id, memctx)?; + let mut output_slot_ctx = + MemCtxValue::::new(slot_addr, memctx)?; + Some(match output_slot_ctx.slot_state() { + SlotState::Addressed | SlotState::Configured => { + // TODO: abort any usb transactions to the device + + // set slot state to default + // set context entries to 1 + // set usb dev address to 0 + output_slot_ctx.mutate(|ctx| { + ctx.set_slot_state(SlotState::Default); + ctx.set_context_entries(1); + ctx.set_usb_device_address(0); + }); + + // for each endpoint context (except the default control endpoint) + let last_endpoint = output_slot_ctx.context_entries(); + for endpoint_id in 1..=last_endpoint { + let mut ep_ctx = + Self::endpoint_context(slot_addr, endpoint_id, memctx)?; + // set ep state to disabled + ep_ctx.mutate(|ctx| { + ctx.set_endpoint_state(EndpointState::Disabled) + }); + } + + TrbCompletionCode::Success + } + _ => TrbCompletionCode::ContextStateError, + }) + } + + pub fn transfer_ring( + &mut self, + slot_id: SlotId, + endpoint_id: u8, + ) -> Option<&mut TransferRing> { + let log = self.log.clone(); + + match self.slot_mut(slot_id) { + Ok(slot) => { + let Some(endpoint) = slot.endpoints.get_mut(&endpoint_id) + else { + slog::error!(log, "rang Doorbell for {slot_id:?}'s endpoint {endpoint_id}, which was absent"); + return None; + }; + Some(endpoint) + } + Err(e) => { + slog::error!( + log, + "rang Doorbell for {slot_id:?}, which was absent: {e}" + ); + None + } + } + } + + pub fn export(&self) -> migrate::DeviceSlotTableV1 { + let Self { dcbaap, slots, port_devs, log: _ } = self; + migrate::DeviceSlotTableV1 { + dcbaap: dcbaap.as_ref().map(|x| x.0), + slots: slots + .iter() + .map(|opt| opt.as_ref().map(|slot| slot.export())) + .collect(), + port_devs: port_devs + .iter() + .map(|opt| opt.as_ref().map(|usbdev| usbdev.export())) + .collect(), + } + } + + pub fn import( + &mut self, + value: &migrate::DeviceSlotTableV1, + ) -> Result<(), crate::migrate::MigrateStateError> { + let migrate::DeviceSlotTableV1 { dcbaap, slots, port_devs } = value; + self.dcbaap = dcbaap.map(GuestAddr); + if port_devs.len() != self.port_devs.len() { + return Err(crate::migrate::MigrateStateError::ImportFailed( + format!( + "payload port devs array size wrong: {} != {}", + port_devs.len(), + self.port_devs.len() + ), + )); + } + for (dst, src) in self.port_devs.iter_mut().zip(port_devs) { + if src.is_none() { + *dst = None; + } else { + let src_dev = src.as_ref().unwrap(); + if let Some(dst_dev) = dst { + dst_dev.import(src_dev)?; + } else { + let mut dst_dev = NullUsbDevice::default(); + dst_dev.import(src_dev)?; + *dst = Some(dst_dev); + } + } + } + for (dst, src) in self.slots.iter_mut().zip(slots) { + if src.is_none() { + *dst = None; + } else { + let src_slot = src.as_ref().unwrap(); + if let Some(dst_slot) = dst { + dst_slot.import(src_slot)?; + } else { + let mut dst_slot = DeviceSlot::new(); + dst_slot.import(src_slot)?; + *dst = Some(dst_slot); + } + } + } + Ok(()) + } +} + +pub mod migrate { + use crate::hw::usb::xhci::port::PortId; + use crate::hw::usb::xhci::rings::consumer::migrate::ConsumerRingV1; + use crate::hw::usb::{ + usbdev::migrate::UsbDeviceV1, + xhci::rings::consumer::transfer::TransferRing, + }; + use serde::{Deserialize, Serialize}; + use std::collections::HashMap; + + #[derive(Serialize, Deserialize)] + pub struct DeviceSlotV1 { + pub endpoints: HashMap, + pub port_address: Option, + } + + impl TryFrom for super::DeviceSlot { + type Error = crate::migrate::MigrateStateError; + + fn try_from(value: DeviceSlotV1) -> Result { + let DeviceSlotV1 { endpoints, port_address } = value; + Ok(Self { + endpoints: endpoints + .into_iter() + .map(|(ep_id, ring)| { + Ok((ep_id, TransferRing::try_from(&ring)?)) + }) + .collect::, Self::Error>>()? + .into_iter() + .collect(), + port_address: port_address + .map(|x| { + PortId::try_from(x).map_err( + crate::migrate::MigrateStateError::ImportFailed, + ) + }) + .transpose()?, + }) + } + } + + #[derive(Serialize, Deserialize)] + pub struct DeviceSlotTableV1 { + pub dcbaap: Option, + pub slots: Vec>, + pub port_devs: Vec>, + } +} diff --git a/lib/propolis/src/hw/usb/xhci/interrupter.rs b/lib/propolis/src/hw/usb/xhci/interrupter.rs new file mode 100644 index 000000000..9cd5a2101 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/interrupter.rs @@ -0,0 +1,599 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Condvar, Mutex, Weak}; + +use crate::common::WriteOp; +use crate::hw::pci; +use crate::hw::usb::xhci::bits; +use crate::hw::usb::xhci::registers::InterrupterRegisters; +use crate::hw::usb::xhci::rings::producer::event::{ + Error as TrbRingProducerError, EventInfo, EventRing, +}; +use crate::hw::usb::xhci::{RegRWOpValue, NUM_INTRS}; +use crate::vmm::{time, MemCtx, VmmHdl}; + +#[usdt::provider(provider = "propolis")] +mod probes { + fn xhci_interrupter_pending(intr_num: u16) {} + fn xhci_interrupter_fired(intr_num: u16) {} +} + +pub struct XhciInterrupter { + number: u16, + evt_ring_seg_tbl_size: bits::EventRingSegmentTableSize, + evt_ring_seg_base_addr: bits::EventRingSegmentTableBaseAddress, + evt_ring: Option, + interrupts: Arc<(Mutex, Condvar)>, + imod_loop: Option>, + vmm_hdl: Arc, +} + +struct InterruptRegulation { + usbcmd_inte: bool, + any_ip_raised: Weak, + + number: u16, + management: bits::InterrupterManagement, + moderation: bits::InterrupterModeration, + + // ERDP contains Event Handler Busy + evt_ring_deq_ptr: bits::EventRingDequeuePointer, + + // IMOD pending has special meaning for INTxPin support, + // but we still need to + intr_pending_enable: bool, + /// IPE, xHCI 1.2 sect 4.17.2 + imod_allow_at: time::VmGuestInstant, + + /// Used for firing PCI interrupts + pci_intr: XhciPciIntr, + + terminate: bool, + + log: slog::Logger, +} + +impl XhciInterrupter { + pub fn new( + number: u16, + pci_intr: XhciPciIntr, + vmm_hdl: Arc, + any_ip_raised: Weak, + log: slog::Logger, + ) -> Self { + let interrupts = Arc::new(( + Mutex::new(InterruptRegulation { + usbcmd_inte: true, + any_ip_raised, + number, + management: bits::InterrupterManagement::default(), + moderation: bits::InterrupterModeration::default(), + evt_ring_deq_ptr: bits::EventRingDequeuePointer(0), + imod_allow_at: time::VmGuestInstant::now(&vmm_hdl).unwrap(), + intr_pending_enable: false, + pci_intr, + terminate: false, + log, + }), + Condvar::new(), + )); + let pair = Arc::downgrade(&interrupts); + let vmm_hdl_loop = Arc::clone(&vmm_hdl); + let imod_loop = Some(std::thread::spawn(move || { + InterruptRegulation::imod_wait_loop(pair, vmm_hdl_loop) + })); + Self { + number, + evt_ring_seg_tbl_size: bits::EventRingSegmentTableSize(0), + evt_ring_seg_base_addr: + bits::EventRingSegmentTableBaseAddress::default(), + evt_ring: None, + interrupts, + imod_loop, + vmm_hdl, + } + } + + pub(super) fn reg_read( + &self, + intr_regs: InterrupterRegisters, + ) -> RegRWOpValue { + use RegRWOpValue::*; + match intr_regs { + InterrupterRegisters::Management => { + U32(self.interrupts.0.lock().unwrap().management.0) + } + InterrupterRegisters::Moderation => { + let regulation = self.interrupts.0.lock().unwrap(); + let duration = + regulation.imod_allow_at.saturating_duration_since( + time::VmGuestInstant::now(&self.vmm_hdl).unwrap(), + ); + // NOTE: div_duration_f32 not available until 1.80, MSRV is 1.70 + let imodc = (duration.as_secs_f64() + / bits::IMOD_TICK.as_secs_f64()) + as u16; + U32(regulation.moderation.with_counter(imodc).0) + } + InterrupterRegisters::EventRingSegmentTableSize => { + U32(self.evt_ring_seg_tbl_size.0) + } + InterrupterRegisters::EventRingSegmentTableBaseAddress => { + U64(self.evt_ring_seg_base_addr.address().0) + } + InterrupterRegisters::EventRingDequeuePointer => { + let regulation = self.interrupts.0.lock().unwrap(); + U64(regulation.evt_ring_deq_ptr.0) + } + } + } + + pub(super) fn reg_write( + &mut self, + wo: &mut WriteOp, + intr_regs: InterrupterRegisters, + memctx: &MemCtx, + ) -> RegRWOpValue { + use RegRWOpValue::*; + + let written_value = match intr_regs { + InterrupterRegisters::Management => { + let iman = bits::InterrupterManagement(wo.read_u32()); + let mut regulation = self.interrupts.0.lock().unwrap(); + // RW1C + if iman.pending() { + // deassert pin interrupt on Interrupt Pending clear + // (only relevant for INTxPin mode) + regulation.pci_intr.deassert(self.number); + // simulate IMODC being loaded with IMODI when IP cleared to 0 + // (xHCI 1.2 sect 5.5.2.2) + regulation.imod_allow_at = + time::VmGuestInstant::now(&self.vmm_hdl) + .unwrap() + .checked_add( + regulation.moderation.interval_duration(), + ) + .unwrap(); + regulation.management.set_pending(false); + } + // RW + regulation.management.set_enable(iman.enable()); + self.interrupts.1.notify_one(); + // remainder of register is reserved + U32(iman.0) + } + InterrupterRegisters::Moderation => { + let mut regulation = self.interrupts.0.lock().unwrap(); + regulation.moderation = + bits::InterrupterModeration(wo.read_u32()); + // emulating setting the value of IMODC, which counts down to zero. + if let Some(inst) = time::VmGuestInstant::now(&self.vmm_hdl) + .unwrap() + .checked_add( + bits::IMOD_TICK + * regulation.moderation.counter() as u32, + ) + { + regulation.imod_allow_at = inst; + self.interrupts.1.notify_one(); + } + U32(regulation.moderation.0) + } + InterrupterRegisters::EventRingSegmentTableSize => { + self.evt_ring_seg_tbl_size = + bits::EventRingSegmentTableSize(wo.read_u32()); + U32(self.evt_ring_seg_tbl_size.0) + } + InterrupterRegisters::EventRingSegmentTableBaseAddress => { + self.evt_ring_seg_base_addr = + bits::EventRingSegmentTableBaseAddress(wo.read_u64()); + U64(self.evt_ring_seg_base_addr.0) + } + InterrupterRegisters::EventRingDequeuePointer => { + let erdp = bits::EventRingDequeuePointer(wo.read_u64()); + let mut regulation = self.interrupts.0.lock().unwrap(); + regulation.evt_ring_deq_ptr.set_dequeue_erst_segment_index( + erdp.dequeue_erst_segment_index(), + ); + regulation.evt_ring_deq_ptr.set_pointer(erdp.pointer()); + // RW1C + if erdp.handler_busy() { + regulation.evt_ring_deq_ptr.set_handler_busy(false); + } + self.interrupts.1.notify_one(); + U64(erdp.0) + } + }; + + let mut regulation = self.interrupts.0.lock().unwrap(); + let erstba = self.evt_ring_seg_base_addr.address(); + let erstsz = self.evt_ring_seg_tbl_size.size() as usize; + let erdp = regulation.evt_ring_deq_ptr.pointer(); + + if let Some(event_ring) = &mut self.evt_ring { + match intr_regs { + InterrupterRegisters::EventRingSegmentTableSize + | InterrupterRegisters::EventRingSegmentTableBaseAddress => { + if let Err(e) = + event_ring.update_segment_table(erstba, erstsz, &memctx) + { + slog::error!( + regulation.log, + "Event Ring Segment Table update failed: {e}" + ); + } + } + InterrupterRegisters::EventRingDequeuePointer => { + let empty_before = event_ring.is_empty(); + event_ring.update_dequeue_pointer(erdp); + if !empty_before && event_ring.is_empty() { + // IPE should be set to 0 "when the Event Ring transitions to empty" + regulation.intr_pending_enable = false; + self.interrupts.1.notify_one(); + } + } + _ => (), + } + } else { + match intr_regs { + InterrupterRegisters::EventRingSegmentTableBaseAddress => { + match EventRing::new(erstba, erstsz, erdp, &memctx) { + Ok(evt_ring) => self.evt_ring = Some(evt_ring), + Err(e) => { + slog::error!( + regulation.log, + "Event Ring Segment Table update failed: {e}" + ); + } + } + } + _ => (), + } + } + + written_value + } + + // returns Ok when an event was enqueued and an interrupt was fired + pub fn enqueue_event( + &mut self, + event_info: EventInfo, + memctx: &MemCtx, + block_event_interrupt: bool, + ) -> Result<(), TrbRingProducerError> { + if let Some(evt_ring) = self.evt_ring.as_mut() { + let mut regulation = self.interrupts.0.lock().unwrap(); + + if let Err(e) = evt_ring.enqueue(event_info.into(), &memctx) { + slog::error!( + regulation.log, + "failed to enqueue Event TRB: {e}" + ); + return Err(e); + } + // check imod/iman for when to fire pci intr + if !block_event_interrupt { + regulation.intr_pending_enable = true; + self.interrupts.1.notify_one(); + let intr_num = self.number; + probes::xhci_interrupter_pending!(move || (intr_num)); + } + + Ok(()) + } else { + Err(TrbRingProducerError::Interrupter) + } + } + + pub fn set_pci_intr_mode( + &mut self, + mode: pci::IntrMode, + pci_state: &pci::DeviceState, + ) { + self.interrupts.0.lock().unwrap().pci_intr.set_mode(mode, pci_state); + self.interrupts.1.notify_one(); + } + + pub fn set_usbcmd_inte(&self, usbcmd_inte: bool) { + self.interrupts.0.lock().unwrap().usbcmd_inte = usbcmd_inte; + self.interrupts.1.notify_one(); + } +} + +impl Drop for XhciInterrupter { + fn drop(&mut self) { + self.interrupts.0.lock().unwrap().terminate = true; + self.interrupts.1.notify_one(); + // Thread::join requires ownership + self.imod_loop.take().unwrap().join().ok(); + } +} + +// xHCI 1.2 sect 4.17.5: +// """ +// The IPE flag of an Interrupter is managed as follows: +// - IPE shall be cleared to 0: +// - When the Event Ring is initialized. +// - If the Event Ring transitions to empty. +// - When an Event TRB is inserted on the Event Ring and BEI = 0 then: +// - IPE shall be set to 1. +// Note: Only Normal, Isoch, and Event Data TRBs support a BEI flag. +// +// The Interrupt Pending (IP) flag of an Interrupter shall be managed as follows: +// - When IPE transitions to 1: +// - If Interrupt Moderation Counter (IMODC) = 0 and Event Handler Busy (EHB) = 0, +// then IP shall be set to 1. +// - When IMODC transitions to 0: +// - If EHB = 0 and IPE = 1, then IP shall be set to 1. +// +// - If MSI or MSI-X interrupts are enabled, IP shall be cleared to 0 automatically when +// the PCI Dword write generated by the Interrupt assertion is complete. +// - If PCI Pin Interrupts are enabled then, IP shall be cleared to 0 by software. +// """ + +impl InterruptRegulation { + // handling IMODI / IMODC / IP / IE. + // xHCI 1.2 figure 4-22 + fn imod_wait_loop( + pair: Weak<(Mutex, Condvar)>, + vmm_hdl: Arc, + ) { + while let Some(pair) = pair.upgrade() { + let (regulation, cvar) = &*pair; + + let guard = regulation.lock().unwrap(); + + // wait for simulated IMODC to tick down to 0 + let Ok(now) = time::VmGuestInstant::now(&vmm_hdl) else { break }; + let imod_allow_at = guard.imod_allow_at; + let timeout = guard.imod_allow_at.saturating_duration_since(now); + // the golden path here *is* for this to time out - we're only + // checking for writes to IMODC during our wait + let (guard, timeout_result) = cvar + .wait_timeout_while(guard, timeout, |ir| { + ir.imod_allow_at == imod_allow_at + }) + .unwrap(); + if !timeout_result.timed_out() { + // IMODC written, restart loop with new value of imod_allow_at + continue; + } + + let mut guard = cvar + .wait_while(guard, |ir| { + !(ir.terminate + || (ir.usbcmd_inte + && ir.management.enable() + && ir.intr_pending_enable + && !ir.evt_ring_deq_ptr.handler_busy())) + }) + .unwrap(); + + if guard.terminate || vmm_hdl.is_destroyed() { + break; + } + + let Some(ip_raised) = guard.any_ip_raised.upgrade() else { break }; + guard.management.set_pending(true); + ip_raised.store(true, Ordering::Release); + + guard.evt_ring_deq_ptr.set_handler_busy(true); + guard.pci_intr.fire_interrupt(guard.number); + // IP flag cleared by the completion of PCI write + // (xHCI 1.2 fig 4-22 description) + guard.management.set_pending(false); + // load counter with interval + guard.imod_allow_at = time::VmGuestInstant::now(&vmm_hdl) + .unwrap() + .checked_add(guard.moderation.interval_duration()) + .unwrap(); + } + } +} + +pub struct XhciPciIntr { + msix_hdl: Option, + pin: Option>, + pci_intr_mode: pci::IntrMode, + log: slog::Logger, +} + +impl XhciPciIntr { + pub fn set_mode( + &mut self, + mode: pci::IntrMode, + pci_state: &pci::DeviceState, + ) { + slog::debug!(self.log, "xHC set interrupt mode to {mode:?}"); + self.pci_intr_mode = mode; + self.msix_hdl = pci_state.msix_hdl(); + self.pin = pci_state.lintr_pin(); + } + + // xHCI 1.2 sect 4.17 + pub fn fire_interrupt(&self, interrupter_num: u16) { + match self.pci_intr_mode { + pci::IntrMode::Disabled => { + slog::error!( + self.log, + "xHC fired PCIe interrupt, but IntrMode was Disabled" + ); + } + pci::IntrMode::INTxPin => { + // NOTE: Only supports one interrupter, per xHCI 1.2 sect 4.17. + // If changing number of interrupters, either remove support for INTxPin here, + // or implement disabling all but the first interrupter everywhere else. + const _: () = const { assert!(NUM_INTRS <= 1) }; + if interrupter_num == 0 { + if let Some(pin) = &self.pin { + slog::debug!(self.log, "xHC interrupter asserting"); + pin.assert(); + probes::xhci_interrupter_fired!(|| interrupter_num); + } else { + slog::error!( + self.log, + "xHC in INTxPin mode with no pin" + ); + } + } else { + slog::error!( + self.log, + "xHC INTxPin tried to fire for non-zero Interrupter" + ); + } + } + pci::IntrMode::Msix => { + if let Some(msix_hdl) = self.msix_hdl.as_ref() { + msix_hdl.fire(interrupter_num); + probes::xhci_interrupter_fired!(|| interrupter_num); + slog::debug!( + self.log, + "xHC interrupter firing: {interrupter_num}" + ); + } else { + slog::error!( + self.log, + "xHC interrupter missing MSI-X handle" + ); + } + } + } + } + + pub fn deassert(&self, interrupter_num: u16) { + // only relevant for INTxPin mode + if let Some(pin) = &self.pin { + if interrupter_num == 0 { + pin.deassert(); + } else { + slog::error!( + self.log, + "tried to deassert INTxPin of non-zero interrupter number" + ); + } + } + } + + pub fn new(pci_state: &pci::DeviceState, log: slog::Logger) -> Self { + Self { + msix_hdl: pci_state.msix_hdl(), + pin: pci_state.lintr_pin(), + pci_intr_mode: pci_state.get_intr_mode(), + log, + } + } +} + +// migration + +impl XhciInterrupter { + pub fn export( + &self, + ) -> Result + { + let XhciInterrupter { + number, + evt_ring_seg_tbl_size, + evt_ring_seg_base_addr, + evt_ring, + interrupts, + imod_loop: _, + vmm_hdl: _, + } = self; + let guard = interrupts.0.lock().unwrap(); + let cvar = &interrupts.1; + + let payload = migrate::XhciInterrupterV1 { + number: *number, + evt_ring_seg_tbl_size: evt_ring_seg_tbl_size.0, + evt_ring_seg_base_addr: evt_ring_seg_base_addr.0, + evt_ring: evt_ring.as_ref().map(From::from), + interrupts: migrate::InterruptRegulationV1 { + usbcmd_inte: guard.usbcmd_inte, + number: guard.number, + management: guard.management.0, + moderation: guard.moderation.0, + evt_ring_deq_ptr: guard.evt_ring_deq_ptr.0, + intr_pending_enable: guard.intr_pending_enable, + imod_allow_at: guard.imod_allow_at, + terminate: guard.terminate, + }, + }; + cvar.notify_one(); + Ok(payload) + } + + pub fn import( + &mut self, + value: migrate::XhciInterrupterV1, + ) -> Result<(), crate::migrate::MigrateStateError> { + let migrate::XhciInterrupterV1 { + number, + evt_ring_seg_tbl_size, + evt_ring_seg_base_addr, + evt_ring, + interrupts: + migrate::InterruptRegulationV1 { + usbcmd_inte, + number: _, + management, + moderation, + evt_ring_deq_ptr, + intr_pending_enable, + imod_allow_at, + terminate, + }, + } = value; + + let mut guard = self.interrupts.0.lock().unwrap(); + let cvar = &self.interrupts.1; + + self.number = number; + self.evt_ring_seg_tbl_size = + bits::EventRingSegmentTableSize(evt_ring_seg_tbl_size); + self.evt_ring_seg_base_addr = + bits::EventRingSegmentTableBaseAddress(evt_ring_seg_base_addr); + self.evt_ring = evt_ring.as_ref().map(From::from); + guard.usbcmd_inte = usbcmd_inte; + guard.number = number; + guard.management = bits::InterrupterManagement(management); + guard.moderation = bits::InterrupterModeration(moderation); + guard.evt_ring_deq_ptr = + bits::EventRingDequeuePointer(evt_ring_deq_ptr); + guard.intr_pending_enable = intr_pending_enable; + guard.imod_allow_at = imod_allow_at; + guard.terminate = terminate; + + cvar.notify_one(); + Ok(()) + } +} + +pub mod migrate { + use crate::{hw::usb::xhci::rings::producer::event::migrate::*, vmm::time}; + use serde::{Deserialize, Serialize}; + + #[derive(Deserialize, Serialize)] + pub struct XhciInterrupterV1 { + pub number: u16, + pub evt_ring_seg_tbl_size: u32, + pub evt_ring_seg_base_addr: u64, + pub evt_ring: Option, + pub interrupts: InterruptRegulationV1, + } + + #[derive(Deserialize, Serialize)] + pub struct InterruptRegulationV1 { + pub usbcmd_inte: bool, + pub number: u16, + pub management: u32, + pub moderation: u32, + pub evt_ring_deq_ptr: u64, + pub intr_pending_enable: bool, + pub imod_allow_at: time::VmGuestInstant, + pub terminate: bool, + } +} diff --git a/lib/propolis/src/hw/usb/xhci/mod.rs b/lib/propolis/src/hw/usb/xhci/mod.rs new file mode 100644 index 000000000..eb905431c --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/mod.rs @@ -0,0 +1,250 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +/*! +## Emulated eXtensible Host Controller Interface (xHCI) device. + +The version of the standard[^xhci-std] referenced throughout the comments in +this module is xHCI 1.2, but we do not implement the features required of a +1.1 or 1.2 compliant host controller - that is, we are only implementing a +subset of what xHCI version 1.0 requires of an xHC, as described by version 1.2 +of the *specification*. + +[^xhci-std]: + +At present, the only USB device supported is a USB 2.0 [NullUsbDevice] with +no actual functionality, which exists as a proof-of-concept and as a means to +show that USB [DeviceDescriptor]s are successfully communicated to the guest +in phd-tests. + +[NullUsbDevice]: super::usbdev::demo_state_tracker::NullUsbDevice +[DeviceDescriptor]: super::usbdev::descriptor::DeviceDescriptor + +```text + +---------+ + | PciXhci | + +---------+ + | has-a + +-----------------------------+ + | XhciState | + |-----------------------------| + | PCI MMIO registers | + | XhciInterrupter | + | DeviceSlotTable | + | Usb2Ports + Usb3Ports | + | CommandRing | + | newly attached USB devices | + +-----------------------------+ + | has-a | ++-------------------+ | has-a +| XhciInterrupter | +-----------------+ +|-------------------| | DeviceSlotTable | +| EventRing | |-----------------| +| MSI-X/INTxPin | | DeviceSlot(s) |___+------------------+ ++-------------------+ | DCBAAP | | DeviceSlot | + | Active USB devs | |------------------| + +-----------------+ | TransferRing(s) | + +------------------+ +``` + +### Conventions + +Wherever possible, the framework represents [TRB] data through a further level +of abstraction, such as enums constructed from the raw TRB bitfields before +being passed to other parts of the system that use them, such that the behavior +of identifying [TrbType] and accessing their fields properly according to the +spec lives in a conversion function rather than strewn across implementation of +other xHC functionality. + +[TRB]: bits::ring_data::Trb +[TrbType]: bits::ring_data::TrbType + +The nomenclature used is generally trading the "Descriptor" suffix for "Info", +e.g. the high-level enum-variant version of an [EventDescriptor] is [EventInfo] +(which is passed to the [EventRing] to be converted into Event TRBs and written +into guest memory). + +[EventRing]: rings::producer::event::EventRing +[EventInfo]: rings::producer::event::EventInfo +[EventDescriptor]: rings::producer::event::EventDescriptor + +For 1-based indices defined by the spec (slot ID, port ID), we use [SlotId] and +[PortId] and their respective `.as_index()` methods to index into our internal +arrays of slots and ports, such that we aspire to categorically avoid off-by-one +errors of omission (of `- 1`). (Note that indexing into the DCBAA is *not* done +with this method, as position 0 in it is reserved by the spec for the Scratchpad +described in xHCI 1.2 section 4.20) + +[SlotId]: device_slots::SlotId +[PortId]: port::PortId + +### Implementation + +#### [DeviceSlotTable] + +When a USB device is attached to the xHC, it is enqueued in a list within +[XhciState] along with its [PortId]. The next time the xHC runs: + +- it will update the corresponding [PORTSC] register and inform the guest with + a TRB on the [EventRing], and if enabled, a hardware interrupt. +- it moves the USB device to the [DeviceSlotTable] in preparation for being + configured and assigned a slot. When the guest xHCD rings Doorbell 0 to run + an [EnableSlot] Command, the [DeviceSlotTable] assigns the first unused + slot ID to it. + +Hot-plugging devices live (i.e. not just attaching all devices defined by the +instance spec at boot time as is done now) is not yet implemented. + +[DeviceSlotTable]: device_slots::DeviceSlotTable +[XhciState]: controller::XhciState +[PortId]: port::PortId +[PORTSC]: bits::PortStatusControl +[EnableSlot]: rings::consumer::command::CommandInfo::EnableSlot + +Device-slot-related Command TRBs are handled by the [DeviceSlotTable]. +The command interface methods are written as translations of the behaviors +defined in xHCI 1.2 section 4.6 to Rust, with liberties taken around redundant +[TrbCompletionCode] writes; i.e. when the outlined behavior from the spec +describes the xHC placing a [Success] into a new TRB on the +[EventRing] immediately at the beginning of the command's execution, and then +overwriting it with a failure code in the event of a failure, +our implementation postpones the creation and enqueueing of the event until +after the outcome of the command's execution (and thus the Event TRB's values) +are all known. + +[TrbCompletionCode]: bits::ring_data::TrbCompletionCode +[Success]: bits::ring_data::TrbCompletionCode::Success + +#### Ports + +Root hub port state machines (xHCI 1.2 section 4.19.1) and port registers are +managed by [Usb2Port], which has separate methods for handling register writes +by the guest and by the xHC itself. + +[Usb2Port]: port::Usb2Port + +#### TRB Rings + +##### Consumer + +The [CommandRing] and each slot endpoint's [TransferRing] are implemented as +[ConsumerRing]<[CommandInfo]> and [ConsumerRing]<[TransferInfo]>. +Dequeued work items are converted from raw [CommandDescriptor]s and +[TransferDescriptor]s, respectively. + +Starting at the dequeue pointer provided by the guest, the [ConsumerRing] will +consume non-Link TRBs (and follow Link TRBs, as in xHCI 1.2 figure 4-15) into +complete work items. In the case of the [CommandRing], [CommandDescriptor]s are +each only made up of one [TRB], but for the [TransferRing] multi-TRB work items +are possible, where all but the last item have the [chain_bit] set. + +[ConsumerRing]: rings::consumer::ConsumerRing +[CommandRing]: rings::consumer::command::CommandRing +[CommandInfo]: rings::consumer::command::CommandInfo +[CommandDescriptor]: rings::consumer::command::CommandDescriptor +[TransferRing]: rings::consumer::transfer::TransferRing +[TransferInfo]: rings::consumer::transfer::TransferInfo +[TransferDescriptor]: rings::consumer::transfer::TransferDescriptor +[chain_bit]: bits::ring_data::TrbControlFieldNormal::chain_bit + +##### Producer + +The only type of producer ring is the [EventRing]. Events destined for it are +fed through the [XhciInterrupter], which handles enablement and rate-limiting +of PCI-level machine interrupts being generated as a result of the events. + +Similarly (and inversely) to the consumer rings, the [EventRing] converts the +[EventInfo]s enqueued in it into [EventDescriptor]s to be written into guest +memory regions defined by the [EventRingSegment] Table. + +[XhciInterrupter]: interrupter::XhciInterrupter +[EventRingSegment]: bits::ring_data::EventRingSegment + +#### Doorbells + +The guest writing to a [DoorbellRegister] makes the host controller process a +consumer TRB ring (the [CommandRing] for doorbell 0, or the corresponding +slot's [TransferRing] for nonzero doorbells). The ring consumption is performed +by the doorbell register write handler, in [process_command_ring] and +[process_transfer_ring]. + +[DoorbellRegister]: bits::DoorbellRegister +[process_command_ring]: rings::consumer::doorbell::process_command_ring +[process_transfer_ring]: rings::consumer::doorbell::process_transfer_ring + +#### Timer registers + +The value of registers defined as incrementing/decrementing per time interval, +such as [MFINDEX] and the [XhciInterrupter]'s [IMODC], are simulated with +[VmGuestInstant]s and [Duration]s rather than by repeated incrementation. + +[IMODC]: bits::InterrupterModeration::counter +[MFINDEX]: bits::MicroframeIndex +[VmGuestInstant]: crate::vmm::time::VmGuestInstant +[Duration]: std::time::Duration + +### DTrace + +To see a trace of all MMIO register reads/writes and TRB enqueue/dequeues: + +```sh +pfexec ./scripts/xhci-trace.d -p $(pgrep propolis-server) +``` + +The name of each register as used by DTrace is `&'static`ally defined in +[registers::Registers::reg_name]. + +*/ + +// pub for rustdoc's sake +pub mod bits; +pub mod controller; +pub mod device_slots; +pub mod interrupter; +pub mod port; +pub mod registers; +pub mod rings; + +pub use controller::PciXhci; + +/// The number of USB2 ports the controller supports. +const NUM_USB2_PORTS: u8 = 4; + +/// The number of USB3 ports the controller supports. +const NUM_USB3_PORTS: u8 = 4; + +/// Value returned for HCSPARAMS1 max ports field. +const MAX_PORTS: u8 = NUM_USB2_PORTS + NUM_USB3_PORTS; + +/// Max number of device slots the controller supports. +// (up to 255) +const MAX_DEVICE_SLOTS: u8 = 64; + +/// Max number of interrupters the controller supports (up to 1024). +const NUM_INTRS: u16 = 1; + +/// An indirection used in [PciXhci::reg_read] and [PciXhci::reg_write], +/// for reporting values to [usdt] probes. +#[derive(Copy, Clone, Debug)] +enum RegRWOpValue { + NoOp, + U8(u8), + U16(u16), + U32(u32), + U64(u64), + Fill(u8), +} + +impl RegRWOpValue { + fn as_u64(&self) -> u64 { + match self { + RegRWOpValue::NoOp => 0, + RegRWOpValue::U8(x) => *x as u64, + RegRWOpValue::U16(x) => *x as u64, + RegRWOpValue::U32(x) => *x as u64, + RegRWOpValue::U64(x) => *x, + RegRWOpValue::Fill(x) => *x as u64, + } + } +} diff --git a/lib/propolis/src/hw/usb/xhci/port.rs b/lib/propolis/src/hw/usb/xhci/port.rs new file mode 100644 index 000000000..4d7525571 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/port.rs @@ -0,0 +1,560 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use migrate::PortTypeV1; + +use crate::hw::usb::xhci::{ + bits::{self, ring_data::TrbCompletionCode, PortStatusControl}, + registers::PortRegisters, + rings::producer::event::EventInfo, + RegRWOpValue, MAX_PORTS, +}; + +#[must_use] +pub enum PortWrite { + NoAction, + BusReset, +} + +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +#[repr(transparent)] +pub struct PortId(u8); + +impl TryFrom for PortId { + type Error = String; + fn try_from(value: u8) -> Result { + if value > 0 && value <= MAX_PORTS { + Ok(Self(value - 1)) + } else { + Err(format!("PortId {value} out of range 1..={MAX_PORTS}")) + } + } +} + +impl PortId { + pub fn as_raw_id(&self) -> u8 { + self.0 + 1 + } + pub fn as_index(&self) -> usize { + self.0 as usize + } +} + +#[derive(Copy, Clone, Default)] +pub struct Usb2Port { + portsc: bits::PortStatusControl, + portpmsc: bits::PortPowerManagementStatusControlUsb2, + portli: bits::PortLinkInfoUsb2, + porthlpmc: bits::PortHardwareLpmControlUsb2, +} + +#[derive(Copy, Clone)] +pub struct Usb3Port { + portsc: bits::PortStatusControl, + portpmsc: bits::PortPowerManagementStatusControlUsb3, + portli: bits::PortLinkInfoUsb3, + porthlpmc: bits::PortHardwareLpmControlUsb3, +} + +impl Default for Usb3Port { + fn default() -> Self { + Self { + // xHCI 1.2 sect 4.19.1.2, figure 4-27: + // the initial state is Disconnected (RxDetect, PP=1) + portsc: PortStatusControl::default() + .with_port_link_state(bits::PortLinkState::RxDetect) + .with_port_power(true), + portpmsc: Default::default(), + portli: Default::default(), + porthlpmc: Default::default(), + } + } +} + +#[allow(private_bounds)] +pub(super) trait XhciUsbPort: XhciUsbPortPrivate + Send + Sync { + fn reg_read(&self, regs: PortRegisters) -> RegRWOpValue { + RegRWOpValue::U32(match regs { + PortRegisters::PortStatusControl => self.portsc_ref().0, + PortRegisters::PortPowerManagementStatusControl => { + self.portpmsc_raw() + } + PortRegisters::PortLinkInfo => self.portli_raw(), + PortRegisters::PortHardwareLpmControl => self.porthlpmc_raw(), + }) + } + + fn reg_write( + &mut self, + value: u32, + regs: PortRegisters, + log: &slog::Logger, + ) -> PortWrite { + match regs { + PortRegisters::PortStatusControl => { + // return: only one of these that can result in a bus reset + return self.portsc_write(bits::PortStatusControl(value), log); + } + PortRegisters::PortPowerManagementStatusControl => { + self.portpmsc_write(value, log); + } + PortRegisters::PortLinkInfo => self.portli_write(value, log), + PortRegisters::PortHardwareLpmControl => { + self.porthlpmc_write(value, log) + } + } + PortWrite::NoAction + } + + /// xHCI 1.2 table 5-27 + fn portsc_write( + &mut self, + wo: bits::PortStatusControl, + log: &slog::Logger, + ) -> PortWrite { + let mut retval = PortWrite::NoAction; + + let is_usb3 = self.is_usb3(); + let portsc = self.portsc_mut(); + // software may disable by writing a 1 to PED + // (*not* enable - only the xHC can do that) + if wo.port_enabled_disabled() && portsc.port_enabled_disabled() { + if wo.port_reset() { + slog::error!(log, "PED=1 and PR=1 written simultaneously - undefined behavior"); + } + portsc.set_port_enabled_disabled(false); + } + + // xHCI 1.2 sect 4.19.5 + if wo.port_reset() { + if !portsc.port_reset() { + // set the PR bit + portsc.set_port_reset(true); + // clear the PED bit to disabled + portsc.set_port_enabled_disabled(false); + // tell controller to initiate USB bus reset sequence + retval = PortWrite::BusReset; + } + } + + if wo.port_link_state_write_strobe() { + // (see xHCI 1.2 sect 4.19, 4.15.2, 4.23.5) + let to_state = wo.port_link_state(); + let from_state = portsc.port_link_state(); + if to_state != from_state { + if port_link_state_write_valid(from_state, to_state) { + // note: a jump straight from U2 to U3 *technically* + // should transition through U0, which we haven't modeled + portsc.set_port_link_state(wo.port_link_state()); + portsc.set_port_link_state_change(true); + } else { + slog::error!(log, + "attempted invalid USB{} port transition from {:?} to {:?}", + if is_usb3 { '3' } else { '2' }, + from_state, to_state + ); + } + } + + // PP shouldn't have to be implemented because PPC in HCCPARAMS1 unset, + // but we must allow it to be set so software can change port state + // (xHCI 1.2 sect 5.4.8) + portsc.set_port_power(wo.port_power()); + } + + if is_usb3 { + use bits::PortLinkState::*; + // xHCI 1.2 figure 4-27 + // xHCI 1.2 sect 4.19.1.2.1 + if wo.port_enabled_disabled() { + // transition from any state except powered-off to Disabled + } + // xHCI 1.2 sect 4.19.1.2.2 + if !wo.port_power() { + // write PP=0 => transition to Powered-off + portsc.set_port_link_state(Disabled); + } else { + // write PP=1 => transition to Disconnected + portsc.set_port_link_state(RxDetect); + } + } else { + use bits::PortLinkState::*; + // xHCI 1.2 figure 4-25 + if wo.port_power() { + portsc.set_port_power(true); + // write PP=1 transitions from Powered-off to Disconnected + if portsc.port_link_state() == Disabled { + portsc.set_port_link_state(RxDetect); + } + } else { + portsc.set_port_power(false); + // write PP=0 transitions any state to Powered-off + portsc.set_port_link_state(Disabled); + } + + // xHCI 1.2 sect 4.19.1.1.6: + // write PED=1 => transition from Enabled to Disabled + if wo.port_enabled_disabled() + && matches!( + portsc.port_link_state(), + U0 | U2 | U3Suspended | Resume + ) + { + portsc.set_port_link_state(Polling); + } + } + + // PIC not implemented because PIND in HCCPARAMS1 unset + + // following are RW1CS fields (software clears by writing 1) + + if wo.connect_status_change() { + portsc.set_connect_status_change(false); + } + + if wo.port_enabled_disabled_change() { + portsc.set_port_enabled_disabled_change(false); + } + + if is_usb3 && wo.warm_port_reset_change() { + portsc.set_warm_port_reset_change(false); + } + + if wo.overcurrent_change() { + portsc.set_overcurrent_change(false); + } + + if wo.port_reset_change() { + portsc.set_port_reset_change(false); + } + + if wo.port_link_state_change() { + portsc.set_port_link_state_change(false); + } + + if wo.port_config_error_change() { + portsc.set_port_config_error_change(false); + } + + // following are RWS (not RW1CS) + portsc.set_wake_on_connect_enable(wo.wake_on_connect_enable()); + portsc.set_wake_on_disconnect_enable(wo.wake_on_disconnect_enable()); + portsc.set_wake_on_overcurrent_enable(wo.wake_on_overcurrent_enable()); + + if is_usb3 && wo.warm_port_reset() { + // TODO: initiate USB3 warm port reset sequence + // (implement whenever any USB devices exist) + portsc.set_port_reset(true); + } + + retval + } + + // entirely different registers between USB2/3 + fn portpmsc_write(&mut self, value: u32, log: &slog::Logger); + fn portli_write(&mut self, value: u32, log: &slog::Logger); + fn porthlpmc_write(&mut self, value: u32, log: &slog::Logger); + + /// Update the given port's PORTSC register from the xHC itself. + /// Returns Some(EventInfo) with a Port Status Change Event to be + /// enqueued in the Event Ring if the PSCEG signal changes from 0 to 1. + fn xhc_update_portsc( + &mut self, + update: &dyn Fn(&mut bits::PortStatusControl), + port_id: PortId, + ) -> Option { + let is_usb3 = self.is_usb3(); + let portsc_before = *self.portsc_ref(); + + let portsc = self.portsc_mut(); + update(portsc); + + // any change to CCS or CAS => force CSC to 1 + if portsc_before.current_connect_status() + != portsc.current_connect_status() + || portsc_before.cold_attach_status() != portsc.cold_attach_status() + { + portsc.set_connect_status_change(true); + } + + if is_usb3 { + todo!("usb3 portsc controller-side change") + // TODO: if Hot Reset transitioned to Warm Reset, set WRC to 1 + } else { + // for concise readability + use bits::PortLinkState::*; + + // xHCI 1.2 sect 4.19.1.1 + if portsc_before.current_connect_status() + && !portsc.current_connect_status() + && matches!( + portsc.port_link_state(), + Polling | U0 | U2 | U3Suspended | Resume + ) + { + portsc.set_port_link_state(RxDetect); + } else if !portsc_before.current_connect_status() + && portsc.current_connect_status() + && portsc.port_link_state() == RxDetect + { + portsc.set_port_link_state(Polling); + } + + // xHCI 1.2 sect 4.19.1.1.4: + // PR=0 => advance to Enabled, set PED and PRC to 1 + if portsc_before.port_reset() && !portsc.port_reset() { + portsc.set_port_link_state(U0); + portsc.set_port_enabled_disabled(true); + portsc.set_port_reset_change(true); + } + } + + let psceg_before = portsc_before.port_status_change_event_generation(); + let psceg_after = portsc.port_status_change_event_generation(); + if psceg_after && !psceg_before { + Some(EventInfo::PortStatusChange { + port_id, + completion_code: TrbCompletionCode::Success, + }) + } else { + None + } + } + + fn import( + &mut self, + value: &migrate::XhcUsbPortV1, + ) -> Result<(), crate::migrate::MigrateStateError> { + let migrate::XhcUsbPortV1 { + port_type, + portsc, + portpmsc, + portli, + porthlpmc, + } = value; + match port_type { + PortTypeV1::Usb2 if !self.is_usb3() => (), + PortTypeV1::Usb3 if self.is_usb3() => (), + _ => { + return Err(crate::migrate::MigrateStateError::ImportFailed( + format!("wrong port type for this port: {port_type:?}"), + )); + } + } + self.portsc_mut().0 = *portsc; + *self.portpmsc_mut() = *portpmsc; + *self.portli_mut() = *portli; + *self.porthlpmc_mut() = *porthlpmc; + Ok(()) + } + + fn export(&self) -> migrate::XhcUsbPortV1 { + migrate::XhcUsbPortV1 { + port_type: if self.is_usb3() { + PortTypeV1::Usb3 + } else { + PortTypeV1::Usb2 + }, + portsc: self.portsc_ref().0, + portpmsc: self.portpmsc_raw(), + portli: self.portli_raw(), + porthlpmc: self.porthlpmc_raw(), + } + } +} + +trait XhciUsbPortPrivate { + fn is_usb3(&self) -> bool; + + fn portsc_ref(&self) -> &bits::PortStatusControl; + fn portsc_mut(&mut self) -> &mut bits::PortStatusControl; + fn portpmsc_raw(&self) -> u32; + fn portli_raw(&self) -> u32; + fn porthlpmc_raw(&self) -> u32; + fn portpmsc_mut(&mut self) -> &mut u32; + fn portli_mut(&mut self) -> &mut u32; + fn porthlpmc_mut(&mut self) -> &mut u32; +} + +impl XhciUsbPortPrivate for Usb2Port { + fn is_usb3(&self) -> bool { + false + } + + fn portsc_ref(&self) -> &bits::PortStatusControl { + &self.portsc + } + fn portsc_mut(&mut self) -> &mut bits::PortStatusControl { + &mut self.portsc + } + fn portpmsc_raw(&self) -> u32 { + self.portpmsc.0 + } + fn portli_raw(&self) -> u32 { + self.portli.0 + } + fn porthlpmc_raw(&self) -> u32 { + self.porthlpmc.0 + } + fn portpmsc_mut(&mut self) -> &mut u32 { + &mut self.portpmsc.0 + } + fn portli_mut(&mut self) -> &mut u32 { + &mut self.portli.0 + } + fn porthlpmc_mut(&mut self) -> &mut u32 { + &mut self.porthlpmc.0 + } +} + +impl XhciUsbPort for Usb2Port { + fn portpmsc_write(&mut self, value: u32, log: &slog::Logger) { + let portpmsc = bits::PortPowerManagementStatusControlUsb2(value); + slog::debug!(log, "{portpmsc:?}"); + self.portpmsc.set_remote_wake_enable(portpmsc.remote_wake_enable()); + self.portpmsc.set_best_effort_service_latency( + portpmsc.best_effort_service_latency(), + ); + self.portpmsc.set_l1_device_slot(portpmsc.l1_device_slot()); + self.portpmsc.set_hardware_lpm_enable(portpmsc.hardware_lpm_enable()); + self.portpmsc.set_port_test_control(portpmsc.port_test_control()); + // xHCI 1.2 sect 4.19.1.1.1: write to PORTPMSC Test Mode > 0 + // transitions from Powered-off state to Test Mode state + if portpmsc.port_test_control() + != bits::PortTestControl::TestModeNotEnabled + { + if self.portsc.port_link_state() == bits::PortLinkState::Disabled { + self.portsc.set_port_link_state(bits::PortLinkState::TestMode); + } + } + } + + fn portli_write(&mut self, _value: u32, _log: &slog::Logger) { + // (in USB2 PORTLI is reserved) + } + + fn porthlpmc_write(&mut self, value: u32, log: &slog::Logger) { + let porthlpmc = bits::PortHardwareLpmControlUsb2(value); + slog::debug!(log, "{porthlpmc:?}"); + self.porthlpmc.set_host_initiated_resume_duration_mode( + porthlpmc.host_initiated_resume_duration_mode(), + ); + self.porthlpmc.set_l1_timeout(porthlpmc.l1_timeout()); + self.porthlpmc.set_best_effort_service_latency_deep( + porthlpmc.best_effort_service_latency_deep(), + ); + } +} + +impl XhciUsbPortPrivate for Usb3Port { + fn is_usb3(&self) -> bool { + true + } + + fn portsc_ref(&self) -> &bits::PortStatusControl { + &self.portsc + } + fn portsc_mut(&mut self) -> &mut bits::PortStatusControl { + &mut self.portsc + } + fn portpmsc_raw(&self) -> u32 { + self.portpmsc.0 + } + fn portli_raw(&self) -> u32 { + self.portli.0 + } + fn porthlpmc_raw(&self) -> u32 { + self.porthlpmc.0 + } + fn portpmsc_mut(&mut self) -> &mut u32 { + &mut self.portpmsc.0 + } + fn portli_mut(&mut self) -> &mut u32 { + &mut self.portli.0 + } + fn porthlpmc_mut(&mut self) -> &mut u32 { + &mut self.porthlpmc.0 + } +} + +impl XhciUsbPort for Usb3Port { + fn portpmsc_write(&mut self, value: u32, _log: &slog::Logger) { + // all RWS and RW fields + self.portpmsc = bits::PortPowerManagementStatusControlUsb3(value); + } + + fn portli_write(&mut self, value: u32, _log: &slog::Logger) { + let portli = bits::PortLinkInfoUsb3(value); + // only field writable by software + self.portli.set_link_error_count(portli.link_error_count()); + } + + fn porthlpmc_write(&mut self, _value: u32, _log: &slog::Logger) { + // (in USB3 PORTHLPMC is reserved) + } +} + +fn port_link_state_write_valid( + from_state: bits::PortLinkState, + to_state: bits::PortLinkState, +) -> bool { + use bits::PortLinkState::*; + // see xHCI 1.2 table 5-27 and section 4.19.1.1 + match (from_state, to_state) { + // xHCI 1.2 sect 4.19.1.2.1, fig 4-27 (usb3 root hub port diagram) + // (outside of the usb2 port enabled substate, but implemented identically) + (Disabled, RxDetect) => true, + + // software may write 0 (U0), 2 (U2), 3 (U3), 5 (RxDetect), 10 (ComplianceMode), 15 (Resume). + // software writing 1, 4, 6-9, 11-14 (the below variants) shall be ignored. + ( + _, + U1 | Disabled | Inactive | Polling | Recovery | HotReset | TestMode, + ) => false, + // writes to PLC don't change anything outside of the usb2 port enabled substate + ( + U1 | Disabled | Inactive | Polling | Recovery | HotReset | TestMode, + _, + ) => false, + // we haven't implemented USB3 ports yet + (_, RxDetect | ComplianceMode) | (RxDetect | ComplianceMode, _) => { + false + } + // xHCI 1.2 fig 4-26 (usb2 port enabled subspace diagram) + (U0, U2 | U3Suspended) => true, + (U0, Resume) => false, + (U2, U0 | U3Suspended) => true, + (U2, Resume) => false, + (U3Suspended, U0 | U2) => false, + (U3Suspended, Resume) => true, + (Resume, U0) => true, + (Resume, U2 | U3Suspended) => false, + // ignore unchanged + (U0, U0) | (U2, U2) | (U3Suspended, U3Suspended) | (Resume, Resume) => { + false + } + // reserved + (Reserved12 | Reserved13 | Reserved14, _) + | (_, Reserved12 | Reserved13 | Reserved14) => false, + } +} + +pub mod migrate { + use serde::{Deserialize, Serialize}; + + #[derive(Serialize, Deserialize, Debug)] + pub enum PortTypeV1 { + Usb2, + Usb3, + } + + #[derive(Serialize, Deserialize)] + pub struct XhcUsbPortV1 { + pub port_type: PortTypeV1, + pub portsc: u32, + pub portpmsc: u32, + pub portli: u32, + pub porthlpmc: u32, + } +} diff --git a/lib/propolis/src/hw/usb/xhci/registers.rs b/lib/propolis/src/hw/usb/xhci/registers.rs new file mode 100644 index 000000000..837ceef14 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/registers.rs @@ -0,0 +1,369 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +//! XHCI Registers + +#![allow(dead_code)] + +use crate::util::regmap::RegMap; + +use super::bits; +use super::port::PortId; +use super::{MAX_DEVICE_SLOTS, MAX_PORTS, NUM_INTRS}; + +use lazy_static::lazy_static; + +/// USB-specific PCI configuration registers. +/// +/// See xHCI 1.2 Section 5.2 PCI Configuration Registers (USB) +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum UsbPciCfgReg { + /// Serial Bus Release Number Register (SBRN) + /// + /// Indicates which version of the USB spec the controller implements. + /// + /// See xHCI 1.2 Section 5.2.3 + SerialBusReleaseNumber, + + /// Frame Length Adjustment Register (FLADJ) + /// + /// See xHCI 1.2 Section 5.2.4 + FrameLengthAdjustment, + + /// Default Best Effort Service Latency \[Deep\] (DBESL / DBESLD) + /// + /// See xHCI 1.2 Section 5.2.5 & 5.2.6 + DefaultBestEffortServiceLatencies, +} + +lazy_static! { + pub static ref USB_PCI_CFG_REGS: RegMap = { + use UsbPciCfgReg::*; + + let layout = [ + (SerialBusReleaseNumber, 1), + (FrameLengthAdjustment, 1), + (DefaultBestEffortServiceLatencies, 1), + ]; + + RegMap::create_packed(bits::USB_PCI_CFG_REG_SZ.into(), &layout, None) + }; +} + +/// Registers in MMIO space pointed to by BAR0/1 +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum Registers { + Reserved, + Cap(CapabilityRegisters), + Op(OperationalRegisters), + Runtime(RuntimeRegisters), + Doorbell(u8), + ExtCap(ExtendedCapabilityRegisters), +} + +/// eXtensible Host Controller Capability Registers +/// +/// See xHCI 1.2 Section 5.3 Host Controller Capability Registers +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum CapabilityRegisters { + CapabilityLength, + HciVersion, + HcStructuralParameters1, + HcStructuralParameters2, + HcStructuralParameters3, + HcCapabilityParameters1, + HcCapabilityParameters2, + DoorbellOffset, + RuntimeRegisterSpaceOffset, +} + +/// eXtensible Host Controller Operational Port Registers +/// +/// See xHCI 1.2 Sections 5.4.8-5.4.11 +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum PortRegisters { + PortStatusControl, + PortPowerManagementStatusControl, + PortLinkInfo, + PortHardwareLpmControl, +} + +/// eXtensible Host Controller Operational Registers +/// +/// See xHCI 1.2 Section 5.4 Host Controller Operational Registers +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum OperationalRegisters { + UsbCommand, + UsbStatus, + PageSize, + DeviceNotificationControl, + CommandRingControlRegister1, + CommandRingControlRegister2, + DeviceContextBaseAddressArrayPointerRegister, + Configure, + Port(PortId, PortRegisters), +} + +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum InterrupterRegisters { + Management, + Moderation, + EventRingSegmentTableSize, + EventRingSegmentTableBaseAddress, + EventRingDequeuePointer, +} + +/// eXtensible Host Controller Runtime Registers +/// +/// See xHCI 1.2 Section 5.5 Host Controller Runtime Registers +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum RuntimeRegisters { + MicroframeIndex, + Interrupter(u16, InterrupterRegisters), +} + +/// eXtensible Host Controller Capability Registers +/// +/// See xHCI 1.2 Section 7 xHCI Extended Capabilities +#[derive(Copy, Clone, Debug, Eq, PartialEq)] +pub enum ExtendedCapabilityRegisters { + SupportedProtocol1(u8), + SupportedProtocol2(u8), + SupportedProtocol3(u8), + SupportedProtocol4(u8), + // used for end of list + Reserved, +} + +pub struct XhcRegMap { + pub map: RegMap, + pub cap_len: usize, + pub op_len: usize, + pub run_len: usize, + pub db_len: usize, +} + +impl XhcRegMap { + pub(super) const fn operational_offset(&self) -> usize { + self.cap_len + } + pub(super) const fn runtime_offset(&self) -> usize { + self.operational_offset() + self.op_len + } + pub(super) const fn doorbell_offset(&self) -> usize { + self.runtime_offset() + self.run_len + } + pub(super) const fn extcap_offset(&self) -> usize { + self.doorbell_offset() + self.db_len + } +} + +lazy_static! { + pub static ref XHC_REGS: XhcRegMap = { + use CapabilityRegisters::*; + use OperationalRegisters::*; + use RuntimeRegisters::*; + use Registers::*; + + // xHCI 1.2 Table 5-9 + // (may be expanded if implementing extended capabilities) + let cap_layout = [ + (Cap(CapabilityLength), 1), + (Reserved, 1), + (Cap(HciVersion), 2), + (Cap(HcStructuralParameters1), 4), + (Cap(HcStructuralParameters2), 4), + (Cap(HcStructuralParameters3), 4), + (Cap(HcCapabilityParameters1), 4), + (Cap(DoorbellOffset), 4), + (Cap(RuntimeRegisterSpaceOffset), 4), + (Cap(HcCapabilityParameters2), 4), + ].into_iter(); + + let op_layout = [ + (Op(UsbCommand), 4), + (Op(UsbStatus), 4), + (Op(PageSize), 4), + (Reserved, 8), + (Op(DeviceNotificationControl), 4), + (Op(CommandRingControlRegister1), 4), + (Op(CommandRingControlRegister2), 4), + (Reserved, 16), + (Op(DeviceContextBaseAddressArrayPointerRegister), 8), + (Op(Configure), 4), + (Reserved, 964), + ].into_iter(); + + // Add the port registers + let op_layout = op_layout.chain((1..=MAX_PORTS).flat_map(|i| { + use PortRegisters::*; + let port_id = PortId::try_from(i).unwrap(); + [ + (Op(OperationalRegisters::Port(port_id, PortStatusControl)), 4), + (Op(OperationalRegisters::Port(port_id, PortPowerManagementStatusControl)), 4), + (Op(OperationalRegisters::Port(port_id, PortLinkInfo)), 4), + (Op(OperationalRegisters::Port(port_id, PortHardwareLpmControl)), 4), + ] + })); + + let run_layout = [ + (Runtime(MicroframeIndex), 4), + (Reserved, 28), + ].into_iter(); + let run_layout = run_layout.chain((0..NUM_INTRS).flat_map(|i| { + use InterrupterRegisters::*; + [ + (Runtime(Interrupter(i, Management)), 4), + (Runtime(Interrupter(i, Moderation)), 4), + (Runtime(Interrupter(i, EventRingSegmentTableSize)), 4), + (Reserved, 4), + (Runtime(Interrupter(i, EventRingSegmentTableBaseAddress)), 8), + (Runtime(Interrupter(i, EventRingDequeuePointer)), 8), + ] + })); + + // +1: 0th doorbell is Command Ring's. + let db_layout = (0..MAX_DEVICE_SLOTS + 1).map(|i| (Doorbell(i), 4)); + + let extcap_layout = [ + (ExtCap(ExtendedCapabilityRegisters::SupportedProtocol1(0)), 4), + (ExtCap(ExtendedCapabilityRegisters::SupportedProtocol2(0)), 4), + (ExtCap(ExtendedCapabilityRegisters::SupportedProtocol3(0)), 4), + (ExtCap(ExtendedCapabilityRegisters::SupportedProtocol4(0)), 4), + (ExtCap(ExtendedCapabilityRegisters::SupportedProtocol1(1)), 4), + (ExtCap(ExtendedCapabilityRegisters::SupportedProtocol2(1)), 4), + (ExtCap(ExtendedCapabilityRegisters::SupportedProtocol3(1)), 4), + (ExtCap(ExtendedCapabilityRegisters::SupportedProtocol4(1)), 4), + (ExtCap(ExtendedCapabilityRegisters::Reserved), 4), + ].into_iter(); + + // Stash the lengths for later use. + let cap_len = cap_layout.clone().map(|(_, sz)| sz).sum(); + let op_len = op_layout.clone().map(|(_, sz)| sz).sum(); + let run_len = run_layout.clone().map(|(_, sz)| sz).sum(); + let db_len = db_layout.clone().map(|(_, sz)| sz).sum(); + let extcap_len: usize = extcap_layout.clone().map(|(_, sz)| sz).sum(); + + let layout = cap_layout + .chain(op_layout) + .chain(run_layout) + .chain(db_layout) + .chain(extcap_layout); + + assert_eq!(cap_len, bits::XHC_CAP_BASE_REG_SZ); + + let xhc_reg_map = XhcRegMap { + map: RegMap::create_packed_iter( + cap_len + op_len + run_len + db_len + extcap_len, + layout, + Some(Reserved), + ), + cap_len, + op_len, + run_len, + db_len, + }; + + // xHCI 1.2 Table 5-2: + // Capability registers must be page-aligned, and they're first. + // Operational-registers must be 4-byte-aligned. They follow cap regs. + // `cap_len` is a multiple of 4 (32 at time of writing). + assert_eq!(xhc_reg_map.operational_offset() % 4, 0); + // Runtime registers must be 32-byte-aligned. + // Both `cap_len` and `op_len` are (at present, cap_len is 1024 + 16*8), + // so we can safely put Runtime registers immediately after them. + // (Note: if VTIO is implemented, virtual fn's must be *page*-aligned) + assert_eq!(xhc_reg_map.runtime_offset() % 32, 0); + // Finally, the Doorbell array merely must be 4-byte-aligned. + // All the runtime registers immediately preceding are 4 bytes wide. + assert_eq!(xhc_reg_map.doorbell_offset() % 4, 0); + + xhc_reg_map + }; +} + +impl Registers { + /// Returns the abbreviation (or name, where unabbreviated) of the register + /// from the xHCI 1.2 specification + pub const fn reg_name(&self) -> &'static str { + use Registers::*; + match self { + Reserved => "RsvdZ.", + Cap(capability_registers) => { + use CapabilityRegisters::*; + match capability_registers { + CapabilityLength => "CAPLENGTH", + HciVersion => "HCIVERSION", + HcStructuralParameters1 => "HCSPARAMS1", + HcStructuralParameters2 => "HCSPARAMS2", + HcStructuralParameters3 => "HCSPARAMS3", + HcCapabilityParameters1 => "HCCPARAMS1", + HcCapabilityParameters2 => "HCCPARAMS2", + DoorbellOffset => "DBOFF", + RuntimeRegisterSpaceOffset => "RTSOFF", + } + } + Op(operational_registers) => { + use OperationalRegisters::*; + match operational_registers { + UsbCommand => "USBCMD", + UsbStatus => "USBSTS", + PageSize => "PAGESIZE", + DeviceNotificationControl => "DNCTRL", + CommandRingControlRegister1 => "CRCR", + CommandRingControlRegister2 => { + "(upper DWORD of 64-bit CRCR)" + } + DeviceContextBaseAddressArrayPointerRegister => "DCBAAP", + Configure => "CONFIG", + Port(_, port_registers) => { + use PortRegisters::*; + match port_registers { + PortStatusControl => "PORTSC", + PortPowerManagementStatusControl => "PORTPMSC", + PortLinkInfo => "PORTLI", + PortHardwareLpmControl => "PORTHLPMC", + } + } + } + } + Runtime(runtime_registers) => { + use RuntimeRegisters::*; + match runtime_registers { + MicroframeIndex => "MFINDEX", + Interrupter(_, interrupter_registers) => { + use InterrupterRegisters::*; + match interrupter_registers { + Management => "IMAN", + Moderation => "IMOD", + EventRingSegmentTableSize => "ERSTSZ", + EventRingSegmentTableBaseAddress => "ERSTBA", + EventRingDequeuePointer => "ERDP", + } + } + } + } + Doorbell(0) => "Doorbell 0 (Command Ring)", + Doorbell(_) => "Doorbell (Transfer Ring)", + ExtCap(extended_capability_registers) => { + use ExtendedCapabilityRegisters::*; + match extended_capability_registers { + SupportedProtocol1(_) => { + "xHCI Supported Protocol Capability (1st DWORD)" + } + SupportedProtocol2(_) => { + "xHCI Supported Protocol Capability (2nd DWORD)" + } + SupportedProtocol3(_) => { + "xHCI Supported Protocol Capability (3rd DWORD)" + } + SupportedProtocol4(_) => { + "xHCI Supported Protocol Capability (4th DWORD)" + } + Reserved => "xHCI Extended Capability: Reserved", + } + } + } + } +} diff --git a/lib/propolis/src/hw/usb/xhci/rings/consumer/command.rs b/lib/propolis/src/hw/usb/xhci/rings/consumer/command.rs new file mode 100644 index 000000000..a4f51077d --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/rings/consumer/command.rs @@ -0,0 +1,467 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use crate::common::GuestAddr; +use crate::hw::usb::xhci::bits::ring_data::{Trb, TrbCompletionCode, TrbType}; +use crate::hw::usb::xhci::device_slots::{DeviceSlotTable, SlotId}; +use crate::hw::usb::xhci::rings::producer::event::EventInfo; +use crate::hw::usb::xhci::NUM_USB2_PORTS; +use crate::vmm::MemCtx; + +use super::{ConsumerRing, Error, Result, WorkItem}; + +pub type CommandRing = ConsumerRing; + +#[derive(Debug)] +pub struct CommandDescriptor(pub Trb); +impl WorkItem for CommandDescriptor { + fn try_from_trb_iter(trbs: impl IntoIterator) -> Result { + let mut trbs = trbs.into_iter(); + if let Some(trb) = trbs.next() { + if trbs.next().is_some() { + Err(Error::CommandDescriptorSize) + } else { + // xHCI 1.2 sect 6.4.3 + match trb.control.trb_type() { + TrbType::NoOpCmd + | TrbType::EnableSlotCmd + | TrbType::DisableSlotCmd + | TrbType::AddressDeviceCmd + | TrbType::ConfigureEndpointCmd + | TrbType::EvaluateContextCmd + | TrbType::ResetEndpointCmd + | TrbType::StopEndpointCmd + | TrbType::SetTRDequeuePointerCmd + | TrbType::ResetDeviceCmd + | TrbType::ForceEventCmd + | TrbType::NegotiateBandwidthCmd + | TrbType::SetLatencyToleranceValueCmd + | TrbType::GetPortBandwidthCmd + | TrbType::ForceHeaderCmd + | TrbType::GetExtendedPropertyCmd + | TrbType::SetExtendedPropertyCmd => Ok(Self(trb)), + _ => Err(Error::InvalidCommandDescriptor(trb)), + } + } + } else { + Err(Error::CommandDescriptorSize) + } + } +} +impl IntoIterator for CommandDescriptor { + type Item = Trb; + type IntoIter = std::iter::Once; + + fn into_iter(self) -> Self::IntoIter { + std::iter::once(self.0) + } +} + +impl TryFrom for CommandInfo { + type Error = Error; + + // xHCI 1.2 section 6.4.3 + fn try_from(cmd_desc: CommandDescriptor) -> Result { + Ok(match cmd_desc.0.control.trb_type() { + TrbType::NoOpCmd => CommandInfo::NoOp, + TrbType::EnableSlotCmd => CommandInfo::EnableSlot { + slot_type: unsafe { cmd_desc.0.control.slot_cmd.slot_type() }, + }, + TrbType::DisableSlotCmd => CommandInfo::DisableSlot { + slot_id: unsafe { cmd_desc.0.control.slot_cmd.slot_id() }, + }, + TrbType::AddressDeviceCmd => CommandInfo::AddressDevice { + input_context_ptr: GuestAddr(cmd_desc.0.parameter & !0b1111), + slot_id: unsafe { cmd_desc.0.control.slot_cmd.slot_id() }, + block_set_address_request: unsafe { + cmd_desc.0.control.slot_cmd.bit9() + }, + }, + TrbType::ConfigureEndpointCmd => CommandInfo::ConfigureEndpoint { + input_context_ptr: GuestAddr(cmd_desc.0.parameter & !0b1111), + slot_id: unsafe { cmd_desc.0.control.slot_cmd.slot_id() }, + deconfigure: unsafe { cmd_desc.0.control.slot_cmd.bit9() }, + }, + TrbType::EvaluateContextCmd => CommandInfo::EvaluateContext { + input_context_ptr: GuestAddr(cmd_desc.0.parameter & !0b1111), + slot_id: unsafe { cmd_desc.0.control.slot_cmd.slot_id() }, + }, + TrbType::ResetEndpointCmd => CommandInfo::ResetEndpoint { + slot_id: unsafe { cmd_desc.0.control.slot_cmd.slot_id() }, + endpoint_id: unsafe { + cmd_desc.0.control.endpoint_cmd.endpoint_id() + }, + transfer_state_preserve: unsafe { + cmd_desc.0.control.endpoint_cmd.transfer_state_preserve() + }, + }, + TrbType::StopEndpointCmd => CommandInfo::StopEndpoint { + slot_id: unsafe { cmd_desc.0.control.slot_cmd.slot_id() }, + endpoint_id: unsafe { + cmd_desc.0.control.endpoint_cmd.endpoint_id() + }, + suspend: unsafe { cmd_desc.0.control.endpoint_cmd.suspend() }, + }, + TrbType::SetTRDequeuePointerCmd => unsafe { + CommandInfo::SetTRDequeuePointer { + new_tr_dequeue_ptr: GuestAddr( + cmd_desc.0.parameter & !0b1111, + ), + dequeue_cycle_state: (cmd_desc.0.parameter & 1) != 0, + // (streams not implemented) + // stream_context_type: ((cmd_desc.0.parameter >> 1) & 0b111) as u8, + // stream_id: cmd_desc.0.status.command.stream_id(), + slot_id: cmd_desc.0.control.endpoint_cmd.slot_id(), + endpoint_id: cmd_desc.0.control.endpoint_cmd.endpoint_id(), + } + }, + TrbType::ResetDeviceCmd => CommandInfo::ResetDevice { + slot_id: unsafe { cmd_desc.0.control.slot_cmd.slot_id() }, + }, + // optional normative, ignored by us + TrbType::ForceEventCmd => CommandInfo::ForceEvent, + // optional normative, ignored by us + TrbType::NegotiateBandwidthCmd => CommandInfo::NegotiateBandwidth, + // optional normative, ignored by us + TrbType::SetLatencyToleranceValueCmd => { + CommandInfo::SetLatencyToleranceValue + } + // optional + TrbType::GetPortBandwidthCmd => CommandInfo::GetPortBandwidth { + port_bandwidth_ctx_ptr: GuestAddr( + cmd_desc.0.parameter & !0b1111, + ), + hub_slot_id: unsafe { + cmd_desc.0.control.get_port_bw_cmd.hub_slot_id() + }, + dev_speed: unsafe { + cmd_desc.0.control.get_port_bw_cmd.dev_speed() + }, + }, + TrbType::ForceHeaderCmd => CommandInfo::ForceHeader { + packet_type: (cmd_desc.0.parameter & 0b1_1111) as u8, + header_info: (cmd_desc.0.parameter >> 5) as u128 + | ((unsafe { cmd_desc.0.status.command_ext.0 } as u128) + << 59), + root_hub_port_number: unsafe { + // hack, same bits + cmd_desc.0.control.get_port_bw_cmd.hub_slot_id().into() + }, + }, + // optional + TrbType::GetExtendedPropertyCmd => unsafe { + CommandInfo::GetExtendedProperty { + extended_property_ctx_ptr: GuestAddr( + cmd_desc.0.parameter & !0b1111, + ), + extended_capability_id: cmd_desc + .0 + .status + .command_ext + .extended_capability_id(), + command_subtype: cmd_desc.0.control.ext_props_cmd.subtype(), + endpoint_id: cmd_desc.0.control.ext_props_cmd.endpoint_id(), + slot_id: cmd_desc.0.control.ext_props_cmd.slot_id(), + } + }, + // optional + TrbType::SetExtendedPropertyCmd => unsafe { + CommandInfo::SetExtendedProperty { + extended_capability_id: cmd_desc + .0 + .status + .command_ext + .extended_capability_id(), + capability_parameter: cmd_desc + .0 + .status + .command_ext + .capability_parameter(), + command_subtype: cmd_desc.0.control.ext_props_cmd.subtype(), + endpoint_id: cmd_desc.0.control.ext_props_cmd.endpoint_id(), + slot_id: cmd_desc.0.control.ext_props_cmd.slot_id(), + } + }, + _ => return Err(Error::InvalidCommandDescriptor(cmd_desc.0)), + }) + } +} + +#[derive(Debug)] +pub enum CommandInfo { + /// xHCI 1.2 sect 3.3.1, 4.6.2 + NoOp, + /// xHCI 1.2 sect 3.3.1, 4.6.2 + EnableSlot { + slot_type: u8, + }, + /// xHCI 1.2 sect 3.3.3, 4.6.4 + DisableSlot { + slot_id: SlotId, + }, + /// xHCI 1.2 sect 3.3.4, 4.6.5 + AddressDevice { + input_context_ptr: GuestAddr, + slot_id: SlotId, + block_set_address_request: bool, + }, + /// xHCI 1.2 sect 3.3.5, 4.6.6 + ConfigureEndpoint { + input_context_ptr: GuestAddr, + slot_id: SlotId, + deconfigure: bool, + }, + EvaluateContext { + input_context_ptr: GuestAddr, + slot_id: SlotId, + }, + ResetEndpoint { + slot_id: SlotId, + endpoint_id: u8, + transfer_state_preserve: bool, + }, + StopEndpoint { + slot_id: SlotId, + endpoint_id: u8, + suspend: bool, + }, + SetTRDequeuePointer { + new_tr_dequeue_ptr: GuestAddr, + dequeue_cycle_state: bool, + slot_id: SlotId, + endpoint_id: u8, + }, + ResetDevice { + slot_id: SlotId, + }, + ForceEvent, + NegotiateBandwidth, + SetLatencyToleranceValue, + #[allow(unused)] + GetPortBandwidth { + port_bandwidth_ctx_ptr: GuestAddr, + hub_slot_id: SlotId, + dev_speed: u8, + }, + /// xHCI 1.2 section 4.6.16 + #[allow(unused)] + ForceHeader { + packet_type: u8, + header_info: u128, + root_hub_port_number: u8, + }, + #[allow(unused)] + GetExtendedProperty { + extended_property_ctx_ptr: GuestAddr, + extended_capability_id: u16, + command_subtype: u8, + endpoint_id: u8, + slot_id: SlotId, + }, + #[allow(unused)] + SetExtendedProperty { + extended_capability_id: u16, + capability_parameter: u8, + command_subtype: u8, + endpoint_id: u8, + slot_id: SlotId, + }, +} + +// TODO: return an iterator of EventInfo's for commands that may produce +// multiple Event TRB's, such as the Stop Endpoint Command +impl CommandInfo { + pub fn run( + self, + cmd_trb_addr: GuestAddr, + dev_slots: &mut DeviceSlotTable, + memctx: &MemCtx, + ) -> EventInfo { + match self { + // xHCI 1.2 sect 3.3.1, 4.6.2 + CommandInfo::NoOp => EventInfo::CommandCompletion { + completion_code: TrbCompletionCode::Success, + slot_id: SlotId::from(0), // 0 for no-op (table 6-42) + cmd_trb_addr, + }, + // xHCI 1.2 sect 3.3.2, 4.6.3 + CommandInfo::EnableSlot { slot_type } => { + match dev_slots.enable_slot(slot_type) { + Some(slot_id) => EventInfo::CommandCompletion { + completion_code: TrbCompletionCode::Success, + slot_id, + cmd_trb_addr, + }, + None => EventInfo::CommandCompletion { + completion_code: + TrbCompletionCode::NoSlotsAvailableError, + slot_id: SlotId::from(0), + cmd_trb_addr, + }, + } + } + // xHCI 1.2 sect 3.3.3, 4.6.4 + CommandInfo::DisableSlot { slot_id } => { + EventInfo::CommandCompletion { + completion_code: dev_slots.disable_slot(slot_id, memctx), + slot_id, + cmd_trb_addr, + } + } + // xHCI 1.2 sect 3.3.4, 4.6.5 + CommandInfo::AddressDevice { + input_context_ptr, + slot_id, + block_set_address_request, + } => { + // xHCI 1.2 pg. 113 + let completion_code = dev_slots + .address_device( + slot_id, + input_context_ptr, + block_set_address_request, + memctx, + ) + // we'll call invalid pointers a context state error + .unwrap_or(TrbCompletionCode::ContextStateError); + + EventInfo::CommandCompletion { + completion_code, + slot_id, + cmd_trb_addr, + } + } + // xHCI 1.2 sect 3.3.5, 4.6.6 + CommandInfo::ConfigureEndpoint { + input_context_ptr, + slot_id, + deconfigure, + } => { + let completion_code = dev_slots + .configure_endpoint( + input_context_ptr, + slot_id, + deconfigure, + memctx, + ) + .unwrap_or(TrbCompletionCode::ResourceError); + EventInfo::CommandCompletion { + completion_code, + slot_id, + cmd_trb_addr, + } + } + CommandInfo::EvaluateContext { input_context_ptr, slot_id } => { + let completion_code = dev_slots + .evaluate_context(slot_id, input_context_ptr, memctx) + // TODO: handle properly. for now just: + .unwrap_or(TrbCompletionCode::ContextStateError); + EventInfo::CommandCompletion { + completion_code, + slot_id, + cmd_trb_addr, + } + } + CommandInfo::ResetEndpoint { + slot_id, + endpoint_id, + transfer_state_preserve, + } => { + let completion_code = dev_slots + .reset_endpoint( + slot_id, + endpoint_id, + transfer_state_preserve, + memctx, + ) + .unwrap_or(TrbCompletionCode::ContextStateError); + EventInfo::CommandCompletion { + completion_code, + slot_id, + cmd_trb_addr, + } + } + CommandInfo::StopEndpoint { slot_id, endpoint_id, suspend } => { + let completion_code = dev_slots + .stop_endpoint(slot_id, endpoint_id, suspend, memctx) + .unwrap_or(TrbCompletionCode::ContextStateError); + EventInfo::CommandCompletion { + completion_code, + slot_id, + cmd_trb_addr, + } + } + CommandInfo::SetTRDequeuePointer { + new_tr_dequeue_ptr, + dequeue_cycle_state, + slot_id, + endpoint_id, + } => { + let completion_code = dev_slots + .set_tr_dequeue_pointer( + new_tr_dequeue_ptr, + slot_id, + endpoint_id, + dequeue_cycle_state, + memctx, + ) + .unwrap_or(TrbCompletionCode::ContextStateError); + EventInfo::CommandCompletion { + completion_code, + slot_id, + cmd_trb_addr, + } + } + CommandInfo::ResetDevice { slot_id } => { + let completion_code = dev_slots + .reset_device(slot_id, memctx) + .unwrap_or(TrbCompletionCode::ContextStateError); + EventInfo::CommandCompletion { + completion_code, + slot_id, + cmd_trb_addr, + } + } + // xHCI 1.2 section 4.6.16 + CommandInfo::ForceHeader { + packet_type: _, + header_info: _, + root_hub_port_number, + } => { + let completion_code = match root_hub_port_number { + 0..NUM_USB2_PORTS => { + // TODO: transmit Force Header packet + TrbCompletionCode::UndefinedError + } + _ => TrbCompletionCode::TrbError, + }; + EventInfo::CommandCompletion { + completion_code, + slot_id: SlotId::from(0), + cmd_trb_addr, + } + } + // optional, unimplemented + CommandInfo::ForceEvent + | CommandInfo::NegotiateBandwidth + | CommandInfo::SetLatencyToleranceValue => { + EventInfo::CommandCompletion { + completion_code: TrbCompletionCode::TrbError, + slot_id: SlotId::from(0), + cmd_trb_addr, + } + } + // optional, unimplemented + CommandInfo::GetPortBandwidth { hub_slot_id: slot_id, .. } + | CommandInfo::GetExtendedProperty { slot_id, .. } + | CommandInfo::SetExtendedProperty { slot_id, .. } => { + EventInfo::CommandCompletion { + completion_code: TrbCompletionCode::TrbError, + slot_id, + cmd_trb_addr, + } + } + } + } +} diff --git a/lib/propolis/src/hw/usb/xhci/rings/consumer/doorbell.rs b/lib/propolis/src/hw/usb/xhci/rings/consumer/doorbell.rs new file mode 100644 index 000000000..9719bf1d9 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/rings/consumer/doorbell.rs @@ -0,0 +1,181 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use crate::common::GuestAddr; +use crate::hw::usb::xhci::controller::XhciState; +use crate::hw::usb::xhci::device_slots::SlotId; +use crate::hw::usb::xhci::rings::consumer; +use crate::hw::usb::xhci::rings::producer::event::EventInfo; +use crate::vmm::MemCtx; + +use super::command::CommandInfo; +use super::transfer::{TransferEventParams, TransferInfo}; +use super::TrbCompletionCode; + +pub fn command_ring_stop( + state: &mut XhciState, + completion_code: TrbCompletionCode, + memctx: &MemCtx, + log: &slog::Logger, +) { + state.crcr.set_command_ring_running(false); + + let cmd_trb_addr = state + .command_ring + .as_ref() + .map(|cmd_ring| cmd_ring.current_dequeue_pointer()) + .unwrap_or(GuestAddr(0)); + let event_info = EventInfo::CommandCompletion { + completion_code, + slot_id: SlotId::from(0), + cmd_trb_addr, + }; + // xHCI 1.2 table 5-24 + if let Err(e) = + state.interrupters[0].enqueue_event(event_info, &memctx, false) + { + slog::error!(log, "couldn't inform xHCD of stopped Control Ring: {e}"); + } else { + slog::debug!(log, "stopped Command Ring with {completion_code:?}"); + } +} + +pub fn process_transfer_ring( + state: &mut XhciState, + slot_id: SlotId, + endpoint_id: u8, + memctx: &MemCtx, + log: &slog::Logger, +) { + loop { + let Some(xfer_ring) = + state.dev_slots.transfer_ring(slot_id, endpoint_id) + else { + break; + }; + + slog::debug!(log, "Transfer Ring at {:#x}", xfer_ring.start_addr.0); + + let trb_pointer = xfer_ring.current_dequeue_pointer(); + match xfer_ring + .dequeue_work_item(&memctx) + .and_then(TransferInfo::try_from) + { + Ok(xfer) => { + let XhciState { evt_data_xfer_len_accum, dev_slots, .. } = + &mut *state; + let dummy_usbdev_stub = match dev_slots.usbdev_for_slot(slot_id) + { + Ok(dev) => dev, + Err(e) => { + slog::error!(log, "No USB device in slot: {e}"); + break; + } + }; + for TransferEventParams { + evt_info, + interrupter, + block_event_interrupt, + } in xfer.run( + trb_pointer, + slot_id, + endpoint_id, + evt_data_xfer_len_accum, + dummy_usbdev_stub, + &memctx, + log, + ) { + slog::debug!(log, "Transfer Event: {evt_info:?}"); + + if let Some(intr) = + state.interrupters.get_mut(interrupter as usize) + { + if let Err(e) = intr.enqueue_event( + evt_info, + &memctx, + block_event_interrupt, + ) { + slog::error!( + log, + "enqueueing Event Data Transfer Event failed: {e}" + ) + } + } else { + slog::error!(log, "no such interrupter {interrupter}"); + } + } + } + Err(consumer::Error::EmptyTransferDescriptor) => { + slog::debug!(log, "Transfer Ring empty"); + break; + } + Err(e) => { + slog::error!(log, "dequeueing TD from endpoint failed: {e}"); + break; + } + } + } +} + +pub fn process_command_ring( + state: &mut XhciState, + memctx: &MemCtx, + log: &slog::Logger, +) { + loop { + if !state.crcr.command_ring_running() { + break; + } + + let cmd_opt = if let Some(ref mut cmd_ring) = state.command_ring { + slog::debug!( + log, + "executing Command Ring from {:#x}", + cmd_ring.start_addr.0, + ); + let cmd_trb_addr = cmd_ring.current_dequeue_pointer(); + match cmd_ring.dequeue_work_item(&memctx).map(|x| (x, cmd_trb_addr)) + { + Ok(work_item) => Some(work_item), + Err(consumer::Error::CommandDescriptorSize) => { + // HACK - matching cycle bits in uninitialized memory trips this, + // should do away with this error entirely + None + } + Err(e) => { + slog::error!( + log, + "Failed to dequeue item from Command Ring: {e}" + ); + None + } + } + } else { + slog::error!(log, "Command Ring not initialized via CRCR yet"); + None + }; + if let Some((cmd_desc, cmd_trb_addr)) = cmd_opt { + match CommandInfo::try_from(cmd_desc) { + Ok(cmd) => { + slog::debug!(log, "Command TRB running: {cmd:?}"); + let event_info = + cmd.run(cmd_trb_addr, &mut state.dev_slots, memctx); + slog::debug!(log, "Command result: {event_info:?}"); + if let Err(e) = state.interrupters[0] + .enqueue_event(event_info, &memctx, false) + { + slog::error!( + log, + "couldn't signal Command TRB completion: {e}" + ); + } + } + Err(e) => slog::error!(log, "Command Ring processing: {e}"), + } + } else { + // command ring absent or empty + break; + } + } +} diff --git a/lib/propolis/src/hw/usb/xhci/rings/consumer/mod.rs b/lib/propolis/src/hw/usb/xhci/rings/consumer/mod.rs new file mode 100644 index 000000000..88ea08e19 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/rings/consumer/mod.rs @@ -0,0 +1,533 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use std::marker::PhantomData; + +use crate::common::GuestAddr; +use crate::hw::usb::xhci::bits::ring_data::*; +use crate::vmm::MemCtx; + +pub mod command; +pub mod transfer; + +pub mod doorbell; + +#[usdt::provider(provider = "propolis")] +mod probes { + fn xhci_consumer_ring_dequeue_trb(offset: usize, data: u64, trb_type: u8) {} + fn xhci_consumer_ring_set_dequeue_ptr(ptr: usize, cycle_state: bool) {} +} + +#[derive(Debug, thiserror::Error)] +pub enum Error { + #[error("I/O error in xHC Ring: {0:?}")] + IoError(#[from] std::io::Error), + #[error("Tried to construct Command Descriptor from multiple TRBs")] + CommandDescriptorSize, + // XXX: the spec says this, but FreeBSD trips this error; + // either my understanding or their xHCD may be slightly wrong + // #[error("Guest defined a consumer TRB ring larger than 64K bytes")] + // RingTooLarge, + #[error("Failed reading TRB from guest memory at {0:?}")] + FailedReadingTRB(GuestAddr), + #[error("Incomplete TD: no more TRBs in cycle to complete chain: {0:?}")] + IncompleteWorkItem(Vec), + #[error("Incomplete TD: TRBs with chain bit set formed a full ring circuit: {0:?}")] + IncompleteWorkItemChainCyclic(Vec), + #[error("TRB Ring Dequeue Pointer was not aligned to size_of: {0:?}")] + InvalidDequeuePointer(GuestAddr), + #[error("Invalid TRB type for a Command Descriptor: {0:?}")] + InvalidCommandDescriptor(Trb), + #[error("Tried to parse empty Transfer Descriptor")] + EmptyTransferDescriptor, + #[error("Invalid TRB type for a Transfer Descriptor: {0:?}")] + InvalidTransferDescriptor(Trb), + #[error("Unexpected TRB type {0:?}, expected {1:?}")] + WrongTrbType(TrbType, TrbType), + #[error("Encountered a complete circuit of matching cycle bits in TRB consumer ring")] + CompleteCircuitOfMatchingCycleBits, + #[error("Apparent corrupt Link TRB pointer (lower 4 bits nonzero): *{0:?} = {1:#x}")] + LinkTRBAlignment(GuestAddr, u64), +} +pub type Result = core::result::Result; + +pub struct ConsumerRing { + // where the ring *starts*, but note that it may be disjoint via Link TRBs + start_addr: GuestAddr, + dequeue_ptr: GuestAddr, + consumer_cycle_state: bool, + _ghost: PhantomData, +} + +pub trait WorkItem: Sized + IntoIterator { + fn try_from_trb_iter(trbs: impl IntoIterator) -> Result; +} + +fn check_aligned_addr(addr: GuestAddr) -> Result<()> { + if addr.0 as usize % size_of::() != 0 { + Err(Error::InvalidDequeuePointer(addr)) + } else { + Ok(()) + } +} + +/// See xHCI 1.2 section 4.14 "Managing Transfer Rings" +impl ConsumerRing { + pub fn new(addr: GuestAddr, cycle_state: bool) -> Result { + check_aligned_addr(addr)?; + + Ok(Self { + start_addr: addr, + dequeue_ptr: addr, + consumer_cycle_state: cycle_state, + _ghost: PhantomData, + }) + } + + fn current_trb(&mut self, memctx: &MemCtx) -> Result { + memctx + .read(self.dequeue_ptr) + .map(|x| *x) + .ok_or(Error::FailedReadingTRB(self.dequeue_ptr)) + } + + fn queue_advance(&mut self, memctx: &MemCtx) -> Result<()> { + let trb = self.current_trb(memctx)?; + + // xHCI 1.2 figure 4-7 + self.dequeue_ptr = if trb.control.trb_type() == TrbType::Link { + if unsafe { trb.control.link.toggle_cycle() } { + // xHCI 1.2 figure 4-8 + self.consumer_cycle_state = !self.consumer_cycle_state; + } + + // xHCI 1.2 sect 4.11.5.1: "The Ring Segment Pointer field in a Link TRB + // is not required to point to the beginning of a physical memory page." + // (They *are* required to be at least 16-byte aligned, i.e. sizeof::()) + // xHCI 1.2 figure 6-38: lower 4 bits are RsvdZ, so we can ignore them; + // but they may be a good indicator of error (pointing at garbage memory) + if trb.parameter & 0b1111 != 0 { + return Err(Error::LinkTRBAlignment( + self.dequeue_ptr, + trb.parameter, + )); + } + GuestAddr(trb.parameter & !0b1111) + } else { + self.dequeue_ptr.offset::(1) + }; + + // xHCI 1.2 sect 4.9: "TRB Rings may be larger than a Page, + // however they shall not cross a 64K byte boundary." + if self.dequeue_ptr.0.abs_diff(self.start_addr.0) > 65536 { + // XXX: FreeBSD seems to have problems with this + // return Err(Error::RingTooLarge); + } + + Ok(()) + } + + /// xHCI 1.2 sects 4.6.10, 6.4.3.9 + pub fn set_dequeue_pointer_and_cycle( + &mut self, + deq_ptr: GuestAddr, + cycle_state: bool, + ) -> Result<()> { + check_aligned_addr(deq_ptr)?; + + probes::xhci_consumer_ring_set_dequeue_ptr!(|| ( + deq_ptr.0 as usize, + cycle_state + )); + + // xHCI 1.2 sect 4.9.2: When a Transfer Ring is enabled or reset, + // the xHC initializes its copies of the Enqueue and Dequeue Pointers + // with the value of the Endpoint/Stream Context TR Dequeue Pointer field. + self.start_addr = deq_ptr; + self.dequeue_ptr = deq_ptr; + self.consumer_cycle_state = cycle_state; + + Ok(()) + } + + /// Return the guest address corresponding to the current dequeue pointer. + pub fn current_dequeue_pointer(&self) -> GuestAddr { + self.dequeue_ptr + } + + pub fn consumer_cycle_state(&self) -> bool { + self.consumer_cycle_state + } + + /// Find the first transfer-related TRB, if one exists. + /// (See xHCI 1.2 sect 4.9.2) + fn dequeue_trb(&mut self, memctx: &MemCtx) -> Result> { + let start_deq_ptr = self.dequeue_ptr; + loop { + let trb = self.current_trb(memctx)?; + + // cycle bit transition - found enqueue pointer + if trb.control.cycle() != self.consumer_cycle_state { + return Ok(None); + } + + self.queue_advance(memctx)?; + + if trb.control.trb_type() != TrbType::Link { + probes::xhci_consumer_ring_dequeue_trb!(|| ( + self.dequeue_ptr.0 as usize, + trb.parameter, + trb.control.trb_type() as u8 + )); + return Ok(Some(trb)); + } + // failsafe - in case of full circuit of matching cycle bits + // without a toggle_cycle occurring, avoid infinite loop + if self.dequeue_ptr == start_deq_ptr { + return Err(Error::CompleteCircuitOfMatchingCycleBits); + } + } + } + + pub fn dequeue_work_item(&mut self, memctx: &MemCtx) -> Result { + let start_deq_ptr = self.dequeue_ptr; + let mut trbs: Vec = + self.dequeue_trb(memctx)?.into_iter().collect(); + while trbs + .last() + .and_then(|end_trb| end_trb.control.chain_bit()) + .unwrap_or(false) + { + // failsafe - if full circuit of chain bits causes an incomplete work item + if self.dequeue_ptr == start_deq_ptr { + return Err(Error::IncompleteWorkItemChainCyclic(trbs)); + } + if let Some(trb) = self.dequeue_trb(memctx)? { + trbs.push(trb); + } else { + // we need more TRBs for this work item that aren't here yet! + return Err(Error::IncompleteWorkItem(trbs)); + } + } + T::try_from_trb_iter(trbs) + } + + pub fn export(&self) -> migrate::ConsumerRingV1 { + let Self { start_addr, dequeue_ptr, consumer_cycle_state, _ghost } = + self; + migrate::ConsumerRingV1 { + start_addr: start_addr.0, + dequeue_ptr: dequeue_ptr.0, + consumer_cycle_state: *consumer_cycle_state, + } + } + + pub fn import(&mut self, value: &migrate::ConsumerRingV1) { + let migrate::ConsumerRingV1 { + start_addr, + dequeue_ptr, + consumer_cycle_state, + } = value; + self.start_addr = GuestAddr(*start_addr); + self.dequeue_ptr = GuestAddr(*dequeue_ptr); + self.consumer_cycle_state = *consumer_cycle_state; + } +} + +impl TryFrom<&migrate::ConsumerRingV1> for ConsumerRing { + type Error = crate::migrate::MigrateStateError; + fn try_from( + value: &migrate::ConsumerRingV1, + ) -> std::result::Result { + let migrate::ConsumerRingV1 { + start_addr, + dequeue_ptr, + consumer_cycle_state, + } = value; + let mut ring = Self::new(GuestAddr(*start_addr), *consumer_cycle_state) + .map_err(|e| { + crate::migrate::MigrateStateError::ImportFailed(format!( + "Consumer ring address error: {e}" + )) + })?; + ring.dequeue_ptr = GuestAddr(*dequeue_ptr); + Ok(ring) + } +} + +pub mod migrate { + use serde::{Deserialize, Serialize}; + + #[derive(Deserialize, Serialize)] + pub struct ConsumerRingV1 { + pub start_addr: u64, + pub dequeue_ptr: u64, + pub consumer_cycle_state: bool, + } +} + +#[cfg(test)] +mod test { + use super::*; + use crate::{ + hw::usb::usbdev::requests::{ + Request, RequestDirection, RequestRecipient, RequestType, + SetupData, StandardRequest, + }, + vmm::PhysMap, + }; + use transfer::TransferRing; + + #[test] + fn test_get_device_descriptor_transfer_ring() { + let mut phys_map = PhysMap::new_test(16 * 1024); + phys_map.add_test_mem("guest-ram".to_string(), 0, 16 * 1024).unwrap(); + let memctx = phys_map.memctx(); + + // mimicking pg. 85 of xHCI 1.2, but with Links thrown in + let ring_segments: &[&[_]] = &[ + &[ + // setup stage + Trb { + parameter: SetupData(0) + .with_direction(RequestDirection::DeviceToHost) + .with_request_type(RequestType::Standard) + .with_recipient(RequestRecipient::Device) + .with_request(Request::Standard( + StandardRequest::GetDescriptor, + )) + .with_value(0x100) + .with_index(0) + .with_length(8) + .0, + status: TrbStatusField { + transfer: TrbStatusFieldTransfer::default() + .with_trb_transfer_length(8) + .with_interrupter_target(0), + }, + control: TrbControlField { + setup_stage: TrbControlFieldSetupStage::default() + .with_cycle(true) + .with_immediate_data(true) + .with_trb_type(TrbType::SetupStage) + .with_transfer_type(TrbTransferType::InDataStage), + }, + }, + // data stage + Trb { + parameter: 0x123456789abcdef0u64, + status: TrbStatusField { + transfer: TrbStatusFieldTransfer::default() + .with_trb_transfer_length(8), + }, + control: TrbControlField { + data_stage: TrbControlFieldDataStage::default() + .with_cycle(true) + .with_trb_type(TrbType::DataStage) + .with_direction(TrbDirection::In), + }, + }, + // link to next ring segment + Trb { + parameter: 2048, + status: TrbStatusField::default(), + control: TrbControlField { + link: TrbControlFieldLink::default() + .with_cycle(true) + .with_trb_type(TrbType::Link), + }, + }, + ], + &[ + // status stage + Trb { + parameter: 0, + status: TrbStatusField::default(), + control: TrbControlField { + status_stage: TrbControlFieldStatusStage::default() + .with_cycle(true) + .with_interrupt_on_completion(true) + .with_trb_type(TrbType::StatusStage) + .with_direction(TrbDirection::Out), + }, + }, + // link back to first ring segment (with toggle cycle) + Trb { + parameter: 1024, + status: TrbStatusField::default(), + control: TrbControlField { + link: TrbControlFieldLink::default() + .with_toggle_cycle(true) + .with_trb_type(TrbType::Link), + }, + }, + ], + ]; + + for (i, seg) in ring_segments.iter().enumerate() { + memctx.write_many(GuestAddr((i as u64 + 1) * 1024), seg); + } + + let mut ring = TransferRing::new(GuestAddr(1024), true).unwrap(); + + let setup_td = ring.dequeue_work_item(&memctx).unwrap(); + + assert_eq!( + ring.current_dequeue_pointer(), + GuestAddr(1024).offset::(1) + ); + + let data_td = ring.dequeue_work_item(&memctx).unwrap(); + + assert_eq!( + ring.current_dequeue_pointer(), + GuestAddr(1024).offset::(2) + ); + + // test setting the dequeue pointer + ring.set_dequeue_pointer_and_cycle( + GuestAddr(1024).offset::(1), + true, + ) + .unwrap(); + + assert_eq!( + ring.current_dequeue_pointer(), + GuestAddr(1024).offset::(1) + ); + + let data_td_copy = ring.dequeue_work_item(&memctx).unwrap(); + + assert_eq!(data_td.trb0_type(), data_td_copy.trb0_type()); + + assert_eq!( + ring.current_dequeue_pointer(), + GuestAddr(1024).offset::(2) + ); + + let status_td = ring.dequeue_work_item(&memctx).unwrap(); + + // skips link trbs + assert_eq!( + ring.current_dequeue_pointer(), + GuestAddr(2048).offset::(1) + ); + + assert!(ring.dequeue_work_item(&memctx).is_ok()); + + assert_eq!(setup_td.trbs.len(), 1); + assert_eq!(data_td.trbs.len(), 1); + assert_eq!(status_td.trbs.len(), 1); + + assert_eq!(setup_td.trb0_type().unwrap(), TrbType::SetupStage); + assert_eq!(data_td.trb0_type().unwrap(), TrbType::DataStage); + assert_eq!(status_td.trb0_type().unwrap(), TrbType::StatusStage); + + assert_eq!(data_td.transfer_size(), 8); + } + + // test chained TD + #[test] + fn test_get_chained_td() { + let mut phys_map = PhysMap::new_test(16 * 1024); + phys_map.add_test_mem("guest-ram".to_string(), 0, 16 * 1024).unwrap(); + let memctx = phys_map.memctx(); + + let status = + TrbStatusField { transfer: TrbStatusFieldTransfer::default() }; + let ring_segments: &[&[_]] = &[ + &[ + Trb { + parameter: 0, + status, + control: TrbControlField { + data_stage: TrbControlFieldDataStage::default() + .with_cycle(true) + .with_trb_type(TrbType::DataStage) + .with_direction(TrbDirection::Out) + .with_chain_bit(true), + }, + }, + Trb { + parameter: 0, + status, + control: TrbControlField { + normal: TrbControlFieldNormal::default() + .with_trb_type(TrbType::Normal) + .with_cycle(true) + .with_chain_bit(true), + }, + }, + // link to next ring segment + Trb { + parameter: 2048, + status, + control: TrbControlField { + link: TrbControlFieldLink::default() + .with_cycle(true) + .with_trb_type(TrbType::Link), + }, + }, + ], + &[ + Trb { + parameter: 0, + status, + control: TrbControlField { + normal: TrbControlFieldNormal::default() + // NOTICE: cycle bit change! we'll test incomplete TD first + .with_cycle(false), + }, + }, + // link back to first ring segment (with toggle cycle) + Trb { + parameter: 1024, + status: TrbStatusField::default(), + control: TrbControlField { + link: TrbControlFieldLink::default() + .with_toggle_cycle(true) + .with_trb_type(TrbType::Link), + }, + }, + ], + ]; + + for (i, seg) in ring_segments.iter().enumerate() { + memctx.write_many(GuestAddr((i as u64 + 1) * 1024), seg); + } + + let mut ring = TransferRing::new(GuestAddr(1024), true).unwrap(); + + let Error::IncompleteWorkItem(incomplete_td) = + ring.dequeue_work_item(&memctx).unwrap_err() + else { + panic!("wrong error returned for incomplete TD!"); + }; + + assert_eq!(incomplete_td.len(), 2); + assert_eq!(incomplete_td[0].control.trb_type(), TrbType::DataStage); + assert_eq!(incomplete_td[1].control.trb_type(), TrbType::Normal); + + // complete the TD (cycle match, chain unset) + memctx.write( + GuestAddr(2048), + &Trb { + parameter: 0, + status, + control: TrbControlField { + normal: TrbControlFieldNormal::default() + .with_trb_type(TrbType::Normal) + .with_cycle(true), + }, + }, + ); + ring.set_dequeue_pointer_and_cycle(GuestAddr(1024), true).unwrap(); + + let complete_td = ring.dequeue_work_item(&memctx).unwrap().trbs; + assert_eq!(complete_td.len(), 3); + assert_eq!(complete_td[0].control.trb_type(), TrbType::DataStage); + assert_eq!(complete_td[1].control.trb_type(), TrbType::Normal); + assert_eq!(complete_td[2].control.trb_type(), TrbType::Normal); + } +} diff --git a/lib/propolis/src/hw/usb/xhci/rings/consumer/transfer.rs b/lib/propolis/src/hw/usb/xhci/rings/consumer/transfer.rs new file mode 100644 index 000000000..f5c5a66ea --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/rings/consumer/transfer.rs @@ -0,0 +1,611 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use crate::common::{GuestAddr, GuestRegion}; +use crate::hw::usb::usbdev::demo_state_tracker::NullUsbDevice; +use crate::hw::usb::usbdev::requests::{ + Request, RequestDirection, SetupData, StandardRequest, +}; +use crate::hw::usb::xhci::bits::ring_data::{ + Trb, TrbDirection, TrbTransferType, TrbType, +}; +use crate::hw::usb::xhci::device_slots::SlotId; +use crate::hw::usb::xhci::rings::consumer::TrbCompletionCode; +use crate::hw::usb::xhci::rings::producer::event::EventInfo; +use crate::vmm::MemCtx; + +use super::{ConsumerRing, Error, Result, WorkItem}; + +#[usdt::provider(provider = "propolis")] +mod probes { + fn xhci_td_consume(trb_type: u8, size: usize, zero_len: bool) {} +} + +pub type TransferRing = ConsumerRing; + +#[derive(Debug)] +pub struct TransferDescriptor { + pub(super) trbs: Vec, +} +impl WorkItem for TransferDescriptor { + fn try_from_trb_iter(trbs: impl IntoIterator) -> Result { + let td = Self { trbs: trbs.into_iter().collect() }; + probes::xhci_td_consume!(|| ( + td.trb0_type().map(|t| t as u8).unwrap_or_default(), + td.transfer_size(), + td.is_zero_length() + )); + Ok(td) + } +} +impl IntoIterator for TransferDescriptor { + type Item = Trb; + type IntoIter = std::vec::IntoIter; + + fn into_iter(self) -> Self::IntoIter { + self.trbs.into_iter() + } +} + +pub enum Never {} +impl TryFrom> for TransferDescriptor { + type Error = Never; + fn try_from(trbs: Vec) -> core::result::Result { + Ok(Self { trbs }) + } +} + +#[allow(dead_code)] +impl TransferDescriptor { + /// xHCI 1.2 sect 4.14: The TD Transfer Size is defined by the sum of the + /// TRB Transfer Length fields in all TRBs that comprise the TD. + pub fn transfer_size(&self) -> usize { + self.trbs + .iter() + .map(|trb| unsafe { trb.status.transfer.trb_transfer_length() } + as usize) + .sum() + } + + pub fn trb0_type(&self) -> Option { + self.trbs.first().map(|trb| trb.control.trb_type()) + } + + /// xHCI 1.2 sect 4.9.1: To generate a zero-length USB transaction, + /// software shall define a TD with a single Transfer TRB with its + /// transfer length set to 0. (it may include others, such as Link TRBs or + /// Event Data TRBs, but only one 'Transfer TRB') + /// (see also xHCI 1.2 table 6-21; as 4.9.1 is ambiguously worded. + /// we're looking at *Normal* Transfer TRBs) + pub fn is_zero_length(&self) -> bool { + let mut trb_transfer_length = None; + for trb in &self.trbs { + if trb.control.trb_type() == TrbType::Normal { + let x = unsafe { trb.status.transfer.trb_transfer_length() }; + if x != 0 { + return false; + } + // more than one Normal encountered + if trb_transfer_length.replace(x).is_some() { + return false; + } + } + } + return true; + } +} + +#[derive(Copy, Clone, Debug)] +pub enum PointerOrImmediate { + Pointer(GuestRegion), + #[allow(dead_code)] + // have not yet implemented anything with out payloads that can use this + Immediate([u8; 8], usize), +} + +impl PointerOrImmediate { + /// size in bytes. + pub fn len(&self) -> usize { + match self { + PointerOrImmediate::Pointer(GuestRegion(_, len)) + | PointerOrImmediate::Immediate(_, len) => *len, + } + } +} + +impl From<&Trb> for PointerOrImmediate { + fn from(trb: &Trb) -> Self { + let data_length = + unsafe { trb.status.transfer.trb_transfer_length() as usize }; + // without loss of generality: IDT same bit for all TRB types + if unsafe { trb.control.normal.immediate_data() } { + PointerOrImmediate::Immediate( + trb.parameter.to_ne_bytes(), + data_length, + ) + } else { + PointerOrImmediate::Pointer(GuestRegion( + GuestAddr(trb.parameter), + data_length, + )) + } + } +} + +#[derive(Debug)] +pub struct TDEventData { + pub event_data: u64, + pub interrupt_target_on_completion: Option, + pub block_event_interrupt: bool, + // TODO: Evaluate Next TRB ? +} + +impl TryFrom<&Trb> for TDEventData { + type Error = Error; + + fn try_from(trb: &Trb) -> Result { + let trb_type = unsafe { trb.control.status_stage.trb_type() }; + if trb_type != TrbType::EventData { + Err(Error::WrongTrbType(trb_type, TrbType::EventData)) + } else { + let interrupt_target_on_completion = unsafe { + if trb.control.status_stage.interrupt_on_completion() { + Some(trb.status.transfer.interrupter_target()) + } else { + None + } + }; + let block_event_interrupt = + unsafe { trb.control.normal.block_event_interrupt() }; + Ok(Self { + event_data: trb.parameter, + interrupt_target_on_completion, + block_event_interrupt, + }) + } + } +} + +#[derive(Debug)] +pub struct TDNormal { + pub data_buffer: PointerOrImmediate, + pub interrupt_target_on_completion: Option, +} + +impl TryFrom<&Trb> for TDNormal { + type Error = Error; + + fn try_from(trb: &Trb) -> Result { + let trb_type = unsafe { trb.control.normal.trb_type() }; + if trb_type != TrbType::Normal { + Err(Error::WrongTrbType(trb_type, TrbType::Normal)) + } else { + let interrupt_target_on_completion = unsafe { + if trb.control.normal.interrupt_on_completion() { + Some(trb.status.transfer.interrupter_target()) + } else { + None + } + }; + Ok(Self { + data_buffer: PointerOrImmediate::from(trb), + interrupt_target_on_completion, + }) + } + } +} + +#[derive(Debug)] +pub enum TransferInfo { + Normal(TDNormal), + SetupStage { + data: SetupData, + interrupt_target_on_completion: Option, + transfer_type: TrbTransferType, + }, + DataStage { + data_buffer: PointerOrImmediate, + interrupt_target_on_completion: Option, + direction: TrbDirection, + payload: Vec, + event_data: Option, + }, + StatusStage { + interrupt_target_on_completion: Option, + direction: TrbDirection, + event_data: Option, + }, + // unimplemented + Isoch {}, + EventData(TDEventData), + NoOp, +} + +impl TryFrom for TransferInfo { + type Error = Error; + + fn try_from(td: TransferDescriptor) -> Result { + let first = td.trbs.first().ok_or(Error::EmptyTransferDescriptor)?; + let interrupt_target_on_completion = unsafe { + // without loss of generality (IOC at same bit position in all TRB types) + if first.control.normal.interrupt_on_completion() { + Some(first.status.transfer.interrupter_target()) + } else { + None + } + }; + Ok(match first.control.trb_type() { + TrbType::Normal => TransferInfo::Normal(TDNormal::try_from(first)?), + TrbType::SetupStage => TransferInfo::SetupStage { + data: SetupData(first.parameter), + interrupt_target_on_completion, + transfer_type: unsafe { + first.control.setup_stage.transfer_type() + }, + }, + TrbType::DataStage => { + let event_data; + let payload; + if unsafe { first.control.data_stage.immediate_data() } { + // xHCI 1.2 table 6-29: "If the IDT flag is set in one + // Data Stage TRB of a TD, then it shall be the only + // Transfer TRB of the TD. An Event Data TRB may also + // be included in the TD." + payload = Vec::new(); + event_data = td + .trbs + .get(1) + .map(|trb| TDEventData::try_from(trb)) + .transpose()? + } else { + // xHCI 1.2 table 6-29 (and sect 3.2.9): "a Data Stage TD is + // defined as a Data Stage TRB followed by zero or more + // Normal TRBs." + // + // yes, this seemingly contradicts the IDT description above, + // as Event Data TRBs are not Normal TRBs, right? + // (more on this later) + // + // however, xHCI 1.2 sect 4.11.2.2 gives the "rule" that + // "A Data Stage TD shall consist of a Data Stage TRB + // chained to zero or more Normal TRBs, or Event Data TRBs." + // + // (to further complicate our terminology: + // xHCI table 6-91 defines "Normal" as a TRB Type with ID 1, + // and "Event Data" as a TRB Type with ID 7. + // xHCI 1.2 sect 1.6 defines "Event Data TRB" as "A Normal + // Transfer TRB with its Event Data (ED) flag equal to 1.") + payload = td.trbs[1..] + .into_iter() + .filter(|trb| { + trb.control.trb_type() != TrbType::EventData + }) + .map(|trb| TDNormal::try_from(trb)) + .collect::>>()?; + event_data = td.trbs[1..] + .into_iter() + .find(|trb| { + trb.control.trb_type() == TrbType::EventData + }) + .map(|trb| TDEventData::try_from(trb)) + .transpose()?; + }; + TransferInfo::DataStage { + data_buffer: PointerOrImmediate::from(first), + interrupt_target_on_completion, + direction: unsafe { first.control.data_stage.direction() }, + payload, + event_data, + } + } + TrbType::StatusStage => TransferInfo::StatusStage { + interrupt_target_on_completion, + direction: unsafe { first.control.status_stage.direction() }, + // xHCI 1.2 table 6-31 (and sect 3.2.9): "a Status Stage TD is + // defined as a Status Stage TRB followed by zero or one + // Event Data TRB." + event_data: td + .trbs + .get(1) + .map(TDEventData::try_from) + .transpose()?, + }, + TrbType::Isoch => TransferInfo::Isoch {}, + TrbType::EventData => { + TransferInfo::EventData(TDEventData::try_from(first)?) + } + TrbType::NoOp => TransferInfo::NoOp, + _ => return Err(Error::InvalidTransferDescriptor(*first)), + }) + } +} + +pub struct TransferEventParams { + pub evt_info: EventInfo, + pub interrupter: u16, + pub block_event_interrupt: bool, +} + +impl TransferInfo { + #[allow(clippy::too_many_arguments)] + pub fn run( + self, + trb_pointer: GuestAddr, + slot_id: SlotId, + endpoint_id: u8, + evt_data_xfer_len_accum: &mut u32, + dummy_usbdev_stub: &mut NullUsbDevice, + memctx: &MemCtx, + log: &slog::Logger, + ) -> Vec { + let mut event_params = self.run_inner( + trb_pointer, + slot_id, + endpoint_id, + evt_data_xfer_len_accum, + dummy_usbdev_stub, + memctx, + log, + ); + for TransferEventParams { evt_info, .. } in &mut event_params { + let EventInfo::Transfer { trb_transfer_length, event_data, .. } = + evt_info + else { + continue; + }; + + if *event_data { + // xHCI 1.2 sect 4.11.5.2: when Transfer TRB completed, + // the number of bytes transferred are added to the EDTLA, + // wrapping at 24-bit max (16,777,215) + *evt_data_xfer_len_accum &= 0xffffff; + // xHCI 1.2 table 6-38: if Event Data flag is 1, this field + // is set to the value of EDTLA + *trb_transfer_length = *evt_data_xfer_len_accum; + } + } + event_params + } + + #[allow(clippy::too_many_arguments)] + fn run_inner( + self, + trb_pointer: GuestAddr, + slot_id: SlotId, + endpoint_id: u8, + evt_data_xfer_len_accum: &mut u32, + dummy_usbdev_stub: &mut NullUsbDevice, + memctx: &MemCtx, + log: &slog::Logger, + ) -> Vec { + if let TransferInfo::EventData(TDEventData { + event_data, + interrupt_target_on_completion, + block_event_interrupt, + }) = self + { + return interrupt_target_on_completion + .map(|interrupter| TransferEventParams { + evt_info: EventInfo::Transfer { + trb_pointer: GuestAddr(event_data), + completion_code: TrbCompletionCode::Success, + trb_transfer_length: *evt_data_xfer_len_accum, + slot_id, + endpoint_id, + event_data: true, + }, + interrupter, + block_event_interrupt, + }) + .into_iter() + .collect(); + } + + // xHCI 1.2 sect 4.11.5.2: + // EDTLA set to 0 prior to executing the first Transfer TRB of a TD + *evt_data_xfer_len_accum = 0; + + match self { + TransferInfo::Normal(TDNormal { + data_buffer, + interrupt_target_on_completion, + }) => { + slog::error!(log, "Normal TD unimplemented (parameter {data_buffer:?}, interrupt target {interrupt_target_on_completion:?})"); + Vec::new() + } + TransferInfo::SetupStage { + data, + interrupt_target_on_completion, + transfer_type, + } => { + if transfer_type == TrbTransferType::Reserved { + slog::error!( + log, + "usb: Setup Stage TRT reserved ({data:x?})" + ); + } + // xHCI 1.2 sect 4.6.5 + let completion_code = if matches!( + data.request(), + Request::Standard(StandardRequest::SetAddress) + ) { + slog::error!(log, "attempted to issue a SET_ADDRESS request through a Transfer Ring"); + TrbCompletionCode::UsbTransactionError + } else { + match dummy_usbdev_stub.setup_stage(data) { + Ok(()) => TrbCompletionCode::Success, + Err(e) => { + slog::error!(log, "USB Setup Stage: {e}"); + TrbCompletionCode::UsbTransactionError + } + } + }; + interrupt_target_on_completion + .map(|interrupter| TransferEventParams { + evt_info: EventInfo::Transfer { + trb_pointer, + completion_code, + trb_transfer_length: 0, + slot_id, + endpoint_id, + event_data: false, + }, + interrupter, + block_event_interrupt: false, + }) + .into_iter() + .collect() + } + TransferInfo::DataStage { + data_buffer, + interrupt_target_on_completion, + direction, + payload, + event_data, + } => { + let req_dir = match direction { + TrbDirection::Out => RequestDirection::HostToDevice, + TrbDirection::In => RequestDirection::DeviceToHost, + }; + + if !payload.is_empty() { + slog::warn!( + log, + "ignoring {} Normal TDs in Data Stage", + payload.len(), + ) + } + + let (trb_transfer_length, completion_code) = + match dummy_usbdev_stub.data_stage( + data_buffer, + req_dir, + &memctx, + ) { + Ok(x) => (x as u32, TrbCompletionCode::Success), + Err(e) => { + slog::error!(log, "USB Data Stage: {e}"); + (0, TrbCompletionCode::UsbTransactionError) + } + }; + // xHCI 1.2 sect 4.11.5.2: when Transfer TRB completed, + // the number of bytes transferred are added to the EDTLA + // (we wrap to 24-bits before using the value elsewhere) + *evt_data_xfer_len_accum += trb_transfer_length; + + interrupt_target_on_completion + .map(|interrupter| TransferEventParams { + evt_info: EventInfo::Transfer { + trb_pointer, + completion_code, + trb_transfer_length, + slot_id, + endpoint_id, + event_data: false, + }, + interrupter, + block_event_interrupt: false, + }) + .into_iter() + .chain(event_data.and_then( + |TDEventData { + event_data, + interrupt_target_on_completion, + block_event_interrupt, + }| { + interrupt_target_on_completion.map(|interrupter| { + TransferEventParams { + evt_info: EventInfo::Transfer { + trb_pointer: GuestAddr(event_data), + // xHCI 1.2 sect 4.11.5.2: Event Data + // inherits the completion code of the + // previous TRB + completion_code, + trb_transfer_length: 0, + slot_id, + endpoint_id, + event_data: true, + }, + interrupter, + block_event_interrupt, + } + }) + }, + )) + .collect() + } + TransferInfo::StatusStage { + interrupt_target_on_completion, + direction, + event_data, + } => { + let req_dir = match direction { + TrbDirection::Out => RequestDirection::HostToDevice, + TrbDirection::In => RequestDirection::DeviceToHost, + }; + + let completion_code = + match dummy_usbdev_stub.status_stage(req_dir) { + Ok(()) => TrbCompletionCode::Success, + Err(e) => { + slog::error!(log, "USB Status Stage: {e}"); + TrbCompletionCode::UsbTransactionError + } + }; + + interrupt_target_on_completion + .map(|interrupter| TransferEventParams { + evt_info: EventInfo::Transfer { + trb_pointer, + completion_code, + trb_transfer_length: 0, + slot_id, + endpoint_id, + event_data: false, + }, + interrupter, + block_event_interrupt: false, + }) + .into_iter() + .chain(event_data.and_then( + |TDEventData { + event_data, + interrupt_target_on_completion, + block_event_interrupt, + }| { + interrupt_target_on_completion.map(|interrupter| { + TransferEventParams { + evt_info: EventInfo::Transfer { + trb_pointer: GuestAddr(event_data), + completion_code, + trb_transfer_length: 0, + slot_id, + endpoint_id, + event_data: true, + }, + interrupter, + block_event_interrupt, + } + }) + }, + )) + .collect() + } + + TransferInfo::Isoch {} => { + // unimplemented on purpose + slog::warn!(log, "Isochronous TD unimplemented"); + Vec::new() + } + TransferInfo::EventData(tdevent_data) => { + slog::warn!( + log, + "Event Data TD unimplemented ({tdevent_data:?})" + ); + Vec::new() + } + TransferInfo::NoOp => Vec::new(), + } + } +} diff --git a/lib/propolis/src/hw/usb/xhci/rings/mod.rs b/lib/propolis/src/hw/usb/xhci/rings/mod.rs new file mode 100644 index 000000000..250d0e07b --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/rings/mod.rs @@ -0,0 +1,6 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +pub mod consumer; +pub mod producer; diff --git a/lib/propolis/src/hw/usb/xhci/rings/producer/event.rs b/lib/propolis/src/hw/usb/xhci/rings/producer/event.rs new file mode 100644 index 000000000..a1fb351f6 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/rings/producer/event.rs @@ -0,0 +1,636 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use crate::common::{GuestAddr, GuestData}; +use crate::hw::usb::xhci::bits::ring_data::*; +use crate::hw::usb::xhci::device_slots::SlotId; +use crate::hw::usb::xhci::port::PortId; +use crate::vmm::MemCtx; + +#[usdt::provider(provider = "propolis")] +mod probes { + fn xhci_producer_ring_enqueue_trb(offset: usize, data: u64, trb_type: u8) {} + fn xhci_producer_ring_set_dequeue_ptr(ptr: usize) {} +} + +#[derive(Debug, thiserror::Error)] +pub enum Error { + #[error("Event Ring full when trying to enqueue {0:?}")] + EventRingFull(Trb), + #[error("Tried to enqueue {0:?} in Event Ring with empty Segment Table")] + EventRingSegmentTableSizeZero(Trb), + #[error("Event Ring Segment Table of size {1} cannot be read from address {0:?}")] + EventRingSegmentTableLocationInvalid(GuestAddr, usize), + #[error("Event Ring Segment Table Entry has invalid size: {0:?}")] + InvalidEventRingSegmentSize(EventRingSegment), + #[error("Interrupter error")] + Interrupter, +} +pub type Result = core::result::Result; + +#[derive(Debug)] +pub struct EventRing { + /// EREP. + enqueue_pointer: Option, + + /// xHCI 1.2 sect 4.9.4: software writes the ERDP register to inform + /// the xHC it has completed processing TRBs up to and including the + /// TRB pointed to by ERDP. + dequeue_pointer: Option, + + /// ESRTE's. + segment_table: Vec, + /// "ESRT Count". + segment_table_index: usize, + + /// "TRB Count". + segment_remaining_trbs: usize, + + /// PCS. + producer_cycle_state: bool, +} + +impl EventRing { + pub fn new( + erstba: GuestAddr, + erstsz: usize, + erdp: GuestAddr, + memctx: &MemCtx, + ) -> Result { + let mut x = Self { + enqueue_pointer: None, + dequeue_pointer: Some(erdp), + segment_table: Vec::new(), + segment_table_index: 0, + segment_remaining_trbs: 0, + producer_cycle_state: true, + }; + x.update_segment_table(erstba, erstsz, memctx)?; + Ok(x) + } + + /// Cache entire segment table. To be called when location (ERSTBA) or + /// size (ERSTSZ) registers are written, or when host controller is resumed. + /// (Per xHCI 1.2 sect 4.9.4.1: ERST entries themselves are not allowed + /// to be modified by software when HCHalted = 0) + pub fn update_segment_table( + &mut self, + erstba: GuestAddr, + erstsz: usize, + memctx: &MemCtx, + ) -> Result<()> { + let many = memctx.read_many(erstba, erstsz).ok_or( + Error::EventRingSegmentTableLocationInvalid(erstba, erstsz), + )?; + self.segment_table = many + .map(|mut erste: GuestData| { + // lower bits are reserved + erste.base_address.0 &= !63; + if erste.segment_trb_count < 16 + || erste.segment_trb_count > 4096 + { + Err(Error::InvalidEventRingSegmentSize(*erste)) + } else { + Ok(*erste) + } + }) + .collect::>>()?; + + if self.segment_table.is_empty() { + self.enqueue_pointer = None; + self.segment_remaining_trbs = 0; + } else if self.enqueue_pointer.is_none() { + self.enqueue_pointer = Some(self.segment_table[0].base_address); + self.segment_remaining_trbs = + self.segment_table[0].segment_trb_count; + } // XXX: do we do this even if enqueue_ptr *was* previously set? + + Ok(()) + } + + /// Must be called when interrupter's ERDP register is written + pub fn update_dequeue_pointer(&mut self, erdp: GuestAddr) { + probes::xhci_producer_ring_set_dequeue_ptr!(|| (erdp.0 as usize)); + self.dequeue_pointer = Some(erdp); + } + + /// Straight translation of xHCI 1.2 figure 4-12. + fn is_full(&self) -> bool { + let deq_ptr = self.dequeue_pointer.unwrap(); + if self.segment_remaining_trbs == 1 { + // check next segment + self.next_segment().base_address == deq_ptr + } else if let Some(enq_ptr) = &self.enqueue_pointer { + // check current segment + enq_ptr.offset::(1) == deq_ptr + } else { + // segment table not initialized yet + true + } + } + + pub fn is_empty(&self) -> bool { + // XXX: is this actually the way we want to handle Nones? + let Some(enq_ptr) = &self.enqueue_pointer else { return false }; + let Some(deq_ptr) = &self.dequeue_pointer else { return false }; + enq_ptr == deq_ptr + } + + /// Straight translation of xHCI 1.2 figure 4-12. + fn next_segment(&self) -> &EventRingSegment { + &self.segment_table + [(self.segment_table_index + 1) % self.segment_table.len()] + } + + /// Straight translation of xHCI 1.2 figure 4-12. + fn enqueue_trb_unchecked(&mut self, mut trb: Trb, memctx: &MemCtx) { + trb.control.set_cycle(self.producer_cycle_state); + + let enq_ptr = self.enqueue_pointer.as_mut().unwrap(); + memctx.write(*enq_ptr, &trb); + + probes::xhci_producer_ring_enqueue_trb!(|| ( + enq_ptr.0 as usize, + trb.parameter, + trb.control.trb_type() as u8 + )); + + enq_ptr.0 += size_of::() as u64; + self.segment_remaining_trbs -= 1; + + if self.segment_remaining_trbs == 0 { + self.segment_table_index += 1; + if self.segment_table_index >= self.segment_table.len() { + self.producer_cycle_state = !self.producer_cycle_state; + self.segment_table_index = 0; + } + let erst_entry = &self.segment_table[self.segment_table_index]; + *enq_ptr = erst_entry.base_address; + self.segment_remaining_trbs = erst_entry.segment_trb_count; + } + } + + /// Straight translation of xHCI 1.2 figure 4-12. + fn enqueue_trb(&mut self, trb: Trb, memctx: &MemCtx) -> Result<()> { + if self.enqueue_pointer.is_none() || self.segment_remaining_trbs == 0 { + // ERST is empty, no enqueue pointer set + Err(Error::EventRingSegmentTableSizeZero(trb)) + } else if self.dequeue_pointer.is_none() { + // waiting for ERDP write, don't write multiple EventRingFullErrors + Err(Error::EventRingFull(trb)) + } else if self.is_full() { + let event_ring_full_error = Trb { + parameter: 0, + status: TrbStatusField { + event: TrbStatusFieldEvent::default().with_completion_code( + TrbCompletionCode::EventRingFullError, + ), + }, + control: TrbControlField { + normal: TrbControlFieldNormal::default() + .with_trb_type(TrbType::HostControllerEvent), + }, + }; + self.enqueue_trb_unchecked(event_ring_full_error, memctx); + // must wait until another ERDP write + self.dequeue_pointer = None; + Err(Error::EventRingFull(trb)) + } else { + self.enqueue_trb_unchecked(trb, memctx); + Ok(()) + } + } + + pub fn enqueue( + &mut self, + value: EventDescriptor, + memctx: &MemCtx, + ) -> Result<()> { + self.enqueue_trb(value.0, memctx) + } +} + +/// xHCI 1.2 sect 4.11.3: Event Descriptors comprised of only one TRB +#[derive(Debug)] +pub struct EventDescriptor(pub Trb); + +#[derive(Debug)] +#[allow(unused)] +pub enum EventInfo { + Transfer { + trb_pointer: GuestAddr, + completion_code: TrbCompletionCode, + trb_transfer_length: u32, + slot_id: SlotId, + endpoint_id: u8, + event_data: bool, + }, + CommandCompletion { + completion_code: TrbCompletionCode, + slot_id: SlotId, + cmd_trb_addr: GuestAddr, + }, + PortStatusChange { + port_id: PortId, + completion_code: TrbCompletionCode, + }, + // optional + BandwidthRequest, + // optional, for 'virtualization' (not the kind we're doing) + Doorbell, + HostController { + completion_code: TrbCompletionCode, + }, + /// Several fields correspond to that of the received USB Device + /// Notification Transaction Packet (DNTP) (xHCI 1.2 table 6-53) + DeviceNotification { + /// Notification Type field of the USB DNTP + notification_type: u8, + /// the value of bytes 5 through 0x0B of the USB DNTP + /// (leave most-significant byte empty) + // TODO: just union/bitstruct this so we can use a [u8; 7]... + notification_data: u64, + completion_code: TrbCompletionCode, + slot_id: SlotId, + }, + /// Generated when USBCMD EWE (Enable Wrap Event) set and MFINDEX wraps, + /// used for periodic transfers with Interrupt and Isoch endpoints. + /// See xHCI 1.2 sect 4.14.2, 6.4.2.8. + MfIndexWrap, +} + +impl Into for EventInfo { + fn into(self) -> EventDescriptor { + match self { + EventInfo::Transfer { + trb_pointer, + completion_code, + trb_transfer_length, + slot_id, + endpoint_id, + event_data, + } => EventDescriptor(Trb { + parameter: trb_pointer.0, + status: TrbStatusField { + event: TrbStatusFieldEvent::default() + .with_completion_code(completion_code) + .with_completion_parameter(trb_transfer_length), + }, + control: TrbControlField { + transfer_event: TrbControlFieldTransferEvent::default() + .with_trb_type(TrbType::TransferEvent) + .with_slot_id(slot_id) + .with_endpoint_id(endpoint_id) + .with_event_data(event_data), + }, + }), + // xHCI 1.2 sect 6.4.2.2 + Self::CommandCompletion { + completion_code: code, + slot_id, + cmd_trb_addr, + } => EventDescriptor(Trb { + parameter: cmd_trb_addr.0, + status: TrbStatusField { + event: TrbStatusFieldEvent::default() + .with_completion_code(code), + }, + control: TrbControlField { + event: TrbControlFieldEvent::default() + .with_trb_type(TrbType::CommandCompletionEvent) + .with_slot_id(slot_id), + }, + }), + EventInfo::PortStatusChange { port_id, completion_code } => { + EventDescriptor(Trb { + parameter: (port_id.as_raw_id() as u64) << 24, + status: TrbStatusField { + event: TrbStatusFieldEvent::default() + .with_completion_code(completion_code), + }, + control: TrbControlField { + event: TrbControlFieldEvent::default() + .with_trb_type(TrbType::PortStatusChangeEvent), + }, + }) + } + EventInfo::BandwidthRequest => { + unimplemented!("xhci: Bandwidth Request Event TRB") + } + EventInfo::Doorbell => { + unimplemented!("xhci: Doorbell Event TRB") + } + EventInfo::HostController { completion_code } => { + EventDescriptor(Trb { + parameter: 0, + status: TrbStatusField { + event: TrbStatusFieldEvent::default() + .with_completion_code(completion_code), + }, + control: TrbControlField { + event: TrbControlFieldEvent::default() + .with_trb_type(TrbType::HostControllerEvent), + }, + }) + } + EventInfo::DeviceNotification { + notification_type, + notification_data, + completion_code, + slot_id, + } => EventDescriptor(Trb { + parameter: ((notification_type as u64) << 4) + | notification_data << 8, + status: TrbStatusField { + event: TrbStatusFieldEvent::default() + .with_completion_code(completion_code), + }, + control: TrbControlField { + event: TrbControlFieldEvent::default() + .with_trb_type(TrbType::DeviceNotificationEvent) + .with_slot_id(slot_id), + }, + }), + EventInfo::MfIndexWrap => { + EventDescriptor(Trb { + parameter: 0, + status: TrbStatusField { + // always set to Success for MFINDEX Wrap Event. + // xHCI 1.2 sect 6.4.2.8 + event: TrbStatusFieldEvent::default() + .with_completion_code(TrbCompletionCode::Success), + }, + control: TrbControlField { + event: TrbControlFieldEvent::default() + .with_trb_type(TrbType::MfIndexWrapEvent), + }, + }) + } + } + } +} + +pub mod migrate { + use crate::common::GuestAddr; + + use super::{EventRing, EventRingSegment}; + use serde::{Deserialize, Serialize}; + + #[derive(Deserialize, Serialize)] + pub struct EventRingV1 { + pub enqueue_pointer: Option, + pub dequeue_pointer: Option, + pub segment_table: Vec, + pub segment_table_index: usize, + pub segment_remaining_trbs: usize, + pub producer_cycle_state: bool, + } + + impl From<&EventRing> for EventRingV1 { + fn from(evt_ring: &EventRing) -> Self { + let EventRing { + enqueue_pointer, + dequeue_pointer, + segment_table, + segment_table_index, + segment_remaining_trbs, + producer_cycle_state, + } = evt_ring; + Self { + enqueue_pointer: enqueue_pointer.map(|x| x.0), + dequeue_pointer: dequeue_pointer.map(|x| x.0), + segment_table: segment_table.iter().map(From::from).collect(), + segment_table_index: *segment_table_index, + segment_remaining_trbs: *segment_remaining_trbs, + producer_cycle_state: *producer_cycle_state, + } + } + } + + impl From<&EventRingV1> for EventRing { + fn from(value: &EventRingV1) -> Self { + let EventRingV1 { + enqueue_pointer, + dequeue_pointer, + segment_table, + segment_table_index, + segment_remaining_trbs, + producer_cycle_state, + } = value; + Self { + enqueue_pointer: enqueue_pointer.map(GuestAddr), + dequeue_pointer: dequeue_pointer.map(GuestAddr), + segment_table: segment_table.iter().map(From::from).collect(), + segment_table_index: *segment_table_index, + segment_remaining_trbs: *segment_remaining_trbs, + producer_cycle_state: *producer_cycle_state, + } + } + } + + #[derive(Deserialize, Serialize)] + pub struct EventRingSegmentV1 { + pub base_address: u64, + pub segment_trb_count: usize, + } + + impl From<&EventRingSegment> for EventRingSegmentV1 { + fn from(ers: &EventRingSegment) -> Self { + let EventRingSegment { base_address, segment_trb_count } = ers; + Self { + base_address: base_address.0, + segment_trb_count: *segment_trb_count, + } + } + } + + impl From<&EventRingSegmentV1> for EventRingSegment { + fn from(value: &EventRingSegmentV1) -> Self { + let EventRingSegmentV1 { base_address, segment_trb_count } = value; + Self { + base_address: GuestAddr(*base_address), + segment_trb_count: *segment_trb_count, + } + } + } +} + +#[cfg(test)] +mod test { + use super::*; + use crate::vmm::PhysMap; + + #[test] + fn test_event_ring_enqueue() { + let mut phys_map = PhysMap::new_test(16 * 1024); + phys_map.add_test_mem("guest-ram".to_string(), 0, 16 * 1024).unwrap(); + let memctx = phys_map.memctx(); + + let erstba = GuestAddr(0); + let erstsz = 2; + let erst_entries = [ + EventRingSegment { + base_address: GuestAddr(1024), + segment_trb_count: 16, + }, + EventRingSegment { + base_address: GuestAddr(2048), + segment_trb_count: 16, + }, + ]; + + memctx.write_many(erstba, &erst_entries); + + let erdp = erst_entries[0].base_address; + + let mut ring = EventRing::new(erstba, erstsz, erdp, &memctx).unwrap(); + + let mut ed_trb = Trb { + parameter: 0, + status: TrbStatusField { + event: TrbStatusFieldEvent::default() + .with_completion_code(TrbCompletionCode::Success), + }, + control: TrbControlField { + normal: TrbControlFieldNormal::default() + .with_trb_type(TrbType::EventData), + }, + }; + // enqueue 31 out of 32 (EventRing must leave room for one final + // event in case of a full ring: the EventRingFullError event!) + for i in 1..32 { + ring.enqueue(EventDescriptor(ed_trb), &memctx).unwrap(); + ed_trb.parameter = i; + } + ring.enqueue(EventDescriptor(ed_trb), &memctx).unwrap_err(); + + // further additions should do nothing until we write a new ERDP + ring.enqueue(EventDescriptor(ed_trb), &memctx).unwrap_err(); + + let mut ring_contents = Vec::new(); + for erste in &erst_entries { + ring_contents.extend( + memctx + .read_many::( + erste.base_address, + erste.segment_trb_count, + ) + .unwrap(), + ); + } + + assert_eq!(ring_contents.len(), 32); + // cycle bits should be set in all these + for i in 0..31 { + assert_eq!(ring_contents[i].parameter, i as u64); + assert_eq!(ring_contents[i].control.trb_type(), TrbType::EventData); + assert_eq!(ring_contents[i].control.cycle(), true); + assert_eq!( + unsafe { ring_contents[i].status.event.completion_code() }, + TrbCompletionCode::Success + ); + } + { + let hce = ring_contents[31]; + assert_eq!(hce.control.cycle(), true); + assert_eq!(hce.control.trb_type(), TrbType::HostControllerEvent); + assert_eq!( + unsafe { hce.status.event.completion_code() }, + TrbCompletionCode::EventRingFullError + ); + } + + // let's say we (the "software") processed the first 8 events. + ring.update_dequeue_pointer( + erst_entries[0].base_address.offset::(8), + ); + + // try to enqueue another 8 events! + for i in 32..39 { + ed_trb.parameter = i; + ring.enqueue(EventDescriptor(ed_trb), &memctx).unwrap(); + } + ring.enqueue(EventDescriptor(ed_trb), &memctx).unwrap_err(); + + // check that they've overwritten previous entries appropriately + ring_contents.clear(); + for erste in &erst_entries { + ring_contents.extend( + memctx + .read_many::( + erste.base_address, + erste.segment_trb_count, + ) + .unwrap(), + ); + } + + // cycle bits should be cleared on the new entries + for i in 0..7 { + assert_eq!(ring_contents[i].parameter, 32 + i as u64); + assert_eq!(ring_contents[i].control.trb_type(), TrbType::EventData); + assert_eq!(ring_contents[i].control.cycle(), false); + assert_eq!( + unsafe { ring_contents[i].status.event.completion_code() }, + TrbCompletionCode::Success + ); + } + { + let hce = ring_contents[7]; + assert_eq!(hce.control.cycle(), false); + assert_eq!(hce.control.trb_type(), TrbType::HostControllerEvent); + assert_eq!( + unsafe { hce.status.event.completion_code() }, + TrbCompletionCode::EventRingFullError + ); + + // haven't overwritten this one (only wrote one EventRingFullError) + let prev = ring_contents[8]; + assert_eq!(prev.parameter, 8); + assert_eq!(prev.control.cycle(), true); + assert_eq!(prev.control.trb_type(), TrbType::EventData); + } + + // let's say the software processed the rest of the events, + // and decided it was time to increase the ring size. + ring.update_dequeue_pointer( + erst_entries[0].base_address.offset::(8), + ); + // test event ring segment table resizes: write a new entry in the table + memctx.write( + erstba + size_of_val(&erst_entries), + &EventRingSegment { + base_address: GuestAddr(3072), + segment_trb_count: 8, + }, + ); + // and update the ring to use it... wait, they need to be at least 16! + assert!(matches!( + ring.update_segment_table(erstba, erstsz + 1, &memctx).unwrap_err(), + Error::InvalidEventRingSegmentSize(_) + )); + + // alright, let's try that again + memctx.write( + erstba + size_of_val(&erst_entries), + &EventRingSegment { + base_address: GuestAddr(3072), + segment_trb_count: 16, + }, + ); + // and *now* update the ring to use it + ring.update_segment_table(erstba, erstsz + 1, &memctx).unwrap(); + + // are we no longer full? + for i in 39..86 { + ed_trb.parameter = i; + ring.enqueue(EventDescriptor(ed_trb), &memctx).unwrap(); + } + // but *now* we're full, right? + assert!(matches!( + ring.enqueue(EventDescriptor(ed_trb), &memctx).unwrap_err(), + Error::EventRingFull(_) + )); + } +} diff --git a/lib/propolis/src/hw/usb/xhci/rings/producer/mod.rs b/lib/propolis/src/hw/usb/xhci/rings/producer/mod.rs new file mode 100644 index 000000000..49ac640e1 --- /dev/null +++ b/lib/propolis/src/hw/usb/xhci/rings/producer/mod.rs @@ -0,0 +1,6 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +// submod to match the xhci::rings::consumer::{command, transfer} structure +pub mod event; diff --git a/lib/propolis/src/util/regmap.rs b/lib/propolis/src/util/regmap.rs index eddd0b8e8..beca9b4cf 100644 --- a/lib/propolis/src/util/regmap.rs +++ b/lib/propolis/src/util/regmap.rs @@ -228,10 +228,17 @@ impl RegMap { size: usize, regdef: &[(ID, usize)], resv_reg: Option, + ) -> Self { + RegMap::create_packed_iter(size, regdef.iter().copied(), resv_reg) + } + pub fn create_packed_iter( + size: usize, + regdef: impl IntoIterator, + resv_reg: Option, ) -> Self { let mut map = RegMap::new(size); let mut off = 0; - for reg in regdef.iter() { + for reg in regdef { let (id, reg_size) = (reg.0, reg.1); let flags = match resv_reg.as_ref() { Some(resv) if *resv == id => { diff --git a/lib/propolis/src/vmm/hdl.rs b/lib/propolis/src/vmm/hdl.rs index dd3690fbf..3334d1367 100644 --- a/lib/propolis/src/vmm/hdl.rs +++ b/lib/propolis/src/vmm/hdl.rs @@ -455,6 +455,11 @@ impl VmmHdl { }) } + /// Has 'destroy()' been called on this VMM + pub fn is_destroyed(&self) -> bool { + self.destroyed.load(Ordering::Acquire) + } + /// Set whether instance should auto-destruct when closed pub fn set_autodestruct(&self, enable_autodestruct: bool) -> Result<()> { self.ioctl_usize( diff --git a/lib/propolis/src/vmm/mem.rs b/lib/propolis/src/vmm/mem.rs index 2c78af26e..76e3c0133 100644 --- a/lib/propolis/src/vmm/mem.rs +++ b/lib/propolis/src/vmm/mem.rs @@ -38,6 +38,7 @@ bitflags! { } } +#[derive(Debug)] pub(crate) struct MapSeg { id: i32, @@ -50,12 +51,14 @@ pub(crate) struct MapSeg { map_seg: Arc, } +#[derive(Debug)] pub(crate) enum MapKind { Dram(MapSeg), Rom(MapSeg), MmioReserve, } +#[derive(Debug)] pub(crate) struct MapEnt { name: String, kind: MapKind, @@ -1009,6 +1012,7 @@ impl MemCtx { MapKind::Rom(seg) => Some((Prot::READ, seg)), MapKind::MmioReserve => None, }?; + // XXX .unwrap_or_else(|| panic!("start {start:#x} end {end:#x} addr {addr:#x} rlen {rlen:#x} ent {ent:?}")); let guest_map = SubMapping::new_base(self, &seg.map_guest) .constrain_access(prot) @@ -1058,6 +1062,12 @@ impl MemCtx { } } +impl core::fmt::Debug for MemCtx { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + core::fmt::Debug::fmt(&self.map.lock().unwrap(), f) + } +} + /// A contiguous region of memory containing generic objects. pub struct MemMany<'a, T: Copy> { mapping: SubMapping<'a>, diff --git a/lib/propolis/src/vmm/time.rs b/lib/propolis/src/vmm/time.rs index 29d6b2fc1..a4bdb2c53 100644 --- a/lib/propolis/src/vmm/time.rs +++ b/lib/propolis/src/vmm/time.rs @@ -77,6 +77,49 @@ impl From for bhyve_api::vdi_time_info_v1 { } } +/// Boot-time-relative timestamps for use in device states (which are migrated +/// after time data, at which point adjustments have been made and communicated +/// to the VMM) +#[derive( + Copy, Clone, Eq, PartialEq, PartialOrd, Ord, Deserialize, Serialize, +)] +pub struct VmGuestInstant(i64); + +impl VmGuestInstant { + pub fn now(vmm_hdl: &VmmHdl) -> std::io::Result { + let td = export_time_data(vmm_hdl)?; + let normalized_ns = + td.hrtime.checked_sub(td.boot_hrtime).ok_or_else(|| { + std::io::Error::new( + std::io::ErrorKind::Other, + TimeAdjustError::GuestUptimeOverflow { + desc: "hrtime - boot_hrtime", + src_hrt: td.hrtime, + boot_hrtime: td.boot_hrtime, + }, + ) + })?; + Ok(Self(normalized_ns)) + } + + pub fn checked_add(&self, dur: Duration) -> Option { + self.0.checked_add(dur.as_nanos().try_into().ok()?).map(Self) + } + + pub fn checked_duration_since(&self, then: Self) -> Option { + let diff_ns = self.0.checked_sub(then.0)?; + if diff_ns > 0 { + Some(Duration::from_nanos(diff_ns as u64)) + } else { + None + } + } + + pub fn saturating_duration_since(&self, then: Self) -> Duration { + self.checked_duration_since(then).unwrap_or_default() + } +} + pub fn import_time_data( hdl: &VmmHdl, time_info: VmTimeData, diff --git a/openapi/propolis-server.json b/openapi/propolis-server.json index cc7631239..4f6804e63 100644 --- a/openapi/propolis-server.json +++ b/openapi/propolis-server.json @@ -761,6 +761,44 @@ ], "additionalProperties": false }, + { + "type": "object", + "properties": { + "component": { + "$ref": "#/components/schemas/XhciController" + }, + "type": { + "type": "string", + "enum": [ + "xhci" + ] + } + }, + "required": [ + "component", + "type" + ], + "additionalProperties": false + }, + { + "type": "object", + "properties": { + "component": { + "$ref": "#/components/schemas/UsbDevice" + }, + "type": { + "type": "string", + "enum": [ + "usb_placeholder" + ] + } + }, + "required": [ + "component", + "type" + ], + "additionalProperties": false + }, { "type": "object", "properties": { @@ -1882,6 +1920,31 @@ } ] }, + "UsbDevice": { + "description": "Describes a USB device, requires the presence of an XhciController.\n\n(Note that at present no USB devices have yet been implemented outside of a null device for testing purposes.)", + "type": "object", + "properties": { + "root_hub_port_num": { + "description": "The root hub port number to which this USB device shall be attached. For USB 2.0 devices, valid values are 1-4, inclusive. For USB 3.0 devices, valid values are 5-8, inclusive.", + "type": "integer", + "format": "uint8", + "minimum": 0 + }, + "xhc_device": { + "description": "The name of the xHC to which this USB device shall be attached.", + "allOf": [ + { + "$ref": "#/components/schemas/SpecKey" + } + ] + } + }, + "required": [ + "root_hub_port_num", + "xhc_device" + ], + "additionalProperties": false + }, "VersionedInstanceSpec": { "description": "A versioned instance spec.", "oneOf": [ @@ -1990,6 +2053,24 @@ "required": [ "active" ] + }, + "XhciController": { + "description": "Describes a PCI device implementing the eXtensible Host Controller Interface for the purpose of attaching USB devices.\n\n(Note that at present no functional USB devices have yet been implemented.)", + "type": "object", + "properties": { + "pci_path": { + "description": "The PCI path at which to attach the guest to this xHC.", + "allOf": [ + { + "$ref": "#/components/schemas/PciPath" + } + ] + } + }, + "required": [ + "pci_path" + ], + "additionalProperties": false } }, "responses": { diff --git a/phd-tests/framework/src/test_vm/config.rs b/phd-tests/framework/src/test_vm/config.rs index a6653e805..d538ffebb 100644 --- a/phd-tests/framework/src/test_vm/config.rs +++ b/phd-tests/framework/src/test_vm/config.rs @@ -2,6 +2,7 @@ // License, v. 2.0. If a copy of the MPL was not distributed with this // file, You can obtain one at https://mozilla.org/MPL/2.0/. +use std::collections::BTreeMap; use std::sync::Arc; use anyhow::Context; @@ -11,7 +12,7 @@ use propolis_client::{ Board, BootOrderEntry, BootSettings, Chipset, ComponentV0, Cpuid, CpuidEntry, CpuidVendor, GuestHypervisorInterface, InstanceSpecV0, MigrationFailureInjector, NvmeDisk, PciPath, SerialPort, - SerialPortNumber, SpecKey, VirtioDisk, + SerialPortNumber, SpecKey, UsbDevice, VirtioDisk, XhciController, }, support::nvme_serial_from_str, types::InstanceMetadata, @@ -57,6 +58,7 @@ pub struct VmConfig<'dr> { disks: Vec>, migration_failure: Option, guest_hv_interface: Option, + xhc_usb_devs: BTreeMap)>, } impl<'dr> VmConfig<'dr> { @@ -77,6 +79,7 @@ impl<'dr> VmConfig<'dr> { disks: Vec::new(), migration_failure: None, guest_hv_interface: None, + xhc_usb_devs: BTreeMap::new(), }; config.boot_disk( @@ -208,6 +211,25 @@ impl<'dr> VmConfig<'dr> { self } + pub fn xhci_controller( + &mut self, + name: SpecKey, + xhc: XhciController, + ) -> &mut Self { + let _old = self.xhc_usb_devs.insert(name, (xhc, Vec::new())); + assert!(_old.is_none()); + self + } + + pub fn usb_device(&mut self, usb_dev: UsbDevice) -> &mut Self { + self.xhc_usb_devs + .get_mut(&usb_dev.xhc_device) + .expect("must add xhc first") + .1 + .push(usb_dev); + self + } + pub async fn vm_spec( &self, framework: &Framework, @@ -222,6 +244,7 @@ impl<'dr> VmConfig<'dr> { disks, migration_failure, guest_hv_interface, + xhc_usb_devs, } = self; let bootrom_path = framework @@ -380,6 +403,18 @@ impl<'dr> VmConfig<'dr> { assert!(_old.is_none()); } + for (xhc_key, (xhc, usb_devs)) in xhc_usb_devs { + spec.components + .insert(xhc_key.to_owned(), ComponentV0::Xhci(xhc.to_owned())); + for usb_dev in usb_devs { + let _old = spec.components.insert( + format!("{xhc_key}-{}", usb_dev.root_hub_port_num).into(), + ComponentV0::UsbPlaceholder(usb_dev.to_owned()), + ); + assert!(_old.is_none()); + } + } + // Generate random identifiers for this instance's timeseries metadata. let sled_id = Uuid::new_v4(); let metadata = InstanceMetadata { diff --git a/phd-tests/tests/src/lib.rs b/phd-tests/tests/src/lib.rs index da2437a87..ebfee1a15 100644 --- a/phd-tests/tests/src/lib.rs +++ b/phd-tests/tests/src/lib.rs @@ -15,3 +15,4 @@ mod migrate; mod server_state_machine; mod smoke; mod stats; +mod xhci; diff --git a/phd-tests/tests/src/xhci.rs b/phd-tests/tests/src/xhci.rs new file mode 100644 index 000000000..3cb80d3c6 --- /dev/null +++ b/phd-tests/tests/src/xhci.rs @@ -0,0 +1,43 @@ +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this +// file, You can obtain one at https://mozilla.org/MPL/2.0/. + +use phd_testcase::*; +use propolis_client::instance_spec::{ + PciPath, SpecKey, UsbDevice, XhciController, +}; + +#[phd_testcase] +async fn usb_device_enumerates(ctx: &Framework) { + let mut config = ctx.vm_config_builder("xhci_usb_device_enumerates_test"); + let xhc_name: SpecKey = "xhc0".into(); + const PCI_DEV: u8 = 3; + const USB_PORT: u8 = 2; + config + .xhci_controller( + xhc_name.to_owned(), + XhciController { pci_path: PciPath::new(0, PCI_DEV, 0)? }, + ) + .usb_device(UsbDevice { + xhc_device: xhc_name, + root_hub_port_num: USB_PORT, + }); + + let spec = config.vm_spec(ctx).await?; + + let mut vm = ctx.spawn_vm_with_spec(spec, None).await?; + if !vm.guest_os_kind().is_linux() { + phd_skip!("xhci/usb test uses sysfs to enumerate devices"); + } + + vm.launch().await?; + vm.wait_to_boot().await?; + + let output = vm + .run_shell_command( + &format!("cat /sys/devices/pci0000:00/0000:00:{PCI_DEV:02}.0/usb1/1-{USB_PORT}/idVendor"), + ) + .await?; + // it's a device provided by us (0x1de)! + assert_eq!(output, "01de"); +} diff --git a/scripts/xhci-trace.d b/scripts/xhci-trace.d new file mode 100755 index 000000000..bef096ddd --- /dev/null +++ b/scripts/xhci-trace.d @@ -0,0 +1,161 @@ +#!/usr/sbin/dtrace -s + +/* + * xhci-trace.d Print propolis emulated xHCI MMIO and TRB ring activity. + * + * USAGE: ./xhci-trace.d -p propolis-pid + */ + +#pragma D option quiet + +string trb_types[64]; + +dtrace:::BEGIN +{ + printf("Tracing propolis PID %d... Hit Ctrl-C to end.\n", $target); + + trb_types[0] = "Reserved"; + + trb_types[1] = "Normal Transfer"; + trb_types[2] = "Setup Stage Transfer"; + trb_types[3] = "Data Stage Transfer"; + trb_types[4] = "Status Stage Transfer"; + trb_types[5] = "Isoch Transfer"; + trb_types[6] = "Link Transfer"; + trb_types[7] = "Event Data Transfer"; + trb_types[8] = "No Op Transfer"; + + trb_types[9] = "Enable Slot Cmd"; + trb_types[10] = "Disable Slot Cmd"; + trb_types[11] = "Address Device Cmd"; + trb_types[12] = "Configure Endpoint Cmd"; + trb_types[13] = "Evaluate Context Cmd"; + trb_types[14] = "Reset Endpoint Cmd"; + trb_types[15] = "Stop Endpoint Cmd"; + trb_types[16] = "Set T R Dequeue Pointer Cmd"; + trb_types[17] = "Reset Device Cmd"; + trb_types[18] = "Force Event Cmd"; + trb_types[19] = "Negotiate Bandwidth Cmd"; + trb_types[20] = "Set Latency Tolerance Value Cmd"; + trb_types[21] = "Get Port Bandwidth Cmd"; + trb_types[22] = "Force Header Cmd"; + trb_types[23] = "No Op Cmd"; + trb_types[24] = "Get Extended Property Cmd"; + trb_types[25] = "Set Extended Property Cmd"; + + trb_types[26] = "Reserved"; + trb_types[27] = "Reserved"; + trb_types[28] = "Reserved"; + trb_types[29] = "Reserved"; + trb_types[30] = "Reserved"; + trb_types[31] = "Reserved"; + + trb_types[32] = "Transfer Event"; + trb_types[33] = "Command Completion Event"; + trb_types[34] = "Port Status Change Event"; + trb_types[35] = "Bandwidth Request Event"; + trb_types[36] = "Doorbell Event"; + trb_types[37] = "Host Controller Event"; + trb_types[38] = "Device Notification Event"; + trb_types[39] = "Mf Index Wrap Event"; + + trb_types[40] = "Reserved"; + trb_types[41] = "Reserved"; + trb_types[42] = "Reserved"; + trb_types[43] = "Reserved"; + trb_types[44] = "Reserved"; + trb_types[45] = "Reserved"; + trb_types[46] = "Reserved"; + trb_types[47] = "Reserved"; + + trb_types[48] = "Vendor"; + trb_types[49] = "Vendor"; + trb_types[50] = "Vendor"; + trb_types[51] = "Vendor"; + trb_types[52] = "Vendor"; + trb_types[53] = "Vendor"; + trb_types[54] = "Vendor"; + trb_types[55] = "Vendor"; + trb_types[56] = "Vendor"; + trb_types[57] = "Vendor"; + trb_types[58] = "Vendor"; + trb_types[59] = "Vendor"; + trb_types[60] = "Vendor"; + trb_types[61] = "Vendor"; + trb_types[62] = "Vendor"; + trb_types[63] = "Vendor"; +} + +struct io_info { + string op; + uint64_t ts; + uint64_t offset_bytes; + uint64_t size_bytes; +}; + +struct io_info io[uint64_t]; + +propolis$target:::xhci_consumer_ring_dequeue_trb /* (offset, parameter, trb_type) */ +{ + if (arg2 < 64) { + printf("[%Y] [0x%08x] Dequeued %s TRB (type %d) with parameter 0x%x\n", walltimestamp, arg0, trb_types[arg2], arg2, arg1); + } else { + printf("[%Y] [0x%08x] Dequeued invalid TRB type (%d)\n", walltimestamp, arg0, arg2); + } +} + +propolis$target:::xhci_consumer_ring_set_dequeue_ptr /* (pointer, cycle_state) */ +{ + printf("[%Y] [0x%08x] Guest xHCD set consumer ring dequeue pointer; cycle state %d\n", walltimestamp, arg0, arg1); +} + +propolis$target:::xhci_producer_ring_enqueue_trb /* (offset, data, trb_type) */ +{ + if (arg2 < 64) { + printf("[%Y] [0x%08x] Enqueued %s TRB (type %d) with parameter 0x%x\n", walltimestamp, arg0, trb_types[arg2], arg2, arg1); + } else { + printf("[%Y] [0x%08x] Enqueued invalid TRB type (%d)\n", walltimestamp, arg0, arg2); + } +} + +propolis$target:::xhci_producer_ring_set_dequeue_ptr /* (pointer) */ +{ + printf("[%Y] [0x%08x] Guest xHCD consumed Event TRBs\n", walltimestamp, arg0); +} + +propolis$target:::xhci_reg_read /* (name, value, index) */ +{ + if (arg2 >= 0) { + printf("[%Y] Read from %s[%d]: 0x%x\n", walltimestamp, copyinstr(arg0), arg2, arg1); + } else { + printf("[%Y] Read from %s: 0x%x\n", walltimestamp, copyinstr(arg0), arg1); + } +} + +propolis$target:::xhci_reg_write /* (name, value, index) */ +{ + if (arg2 >= 0) { + printf("[%Y] Write to %s[%d]: 0x%x\n", walltimestamp, copyinstr(arg0), arg2, arg1); + } else { + printf("[%Y] Write to %s: 0x%x\n", walltimestamp, copyinstr(arg0), arg1); + } +} + +propolis$target:::xhci_interrupter_pending /* (intr_num) */ +{ + printf("[%Y] Interrupter %d pending\n", walltimestamp, arg0); +} + +propolis$target:::xhci_interrupter_fired /* (intr_num) */ +{ + printf("[%Y] Interrupter %d fired\n", walltimestamp, arg0); +} + +propolis$target:::xhci_reset /* () */ +{ + printf("\n[%Y] xHC reset\n\n", walltimestamp); +} + +dtrace:::END +{ +}