Fault

A Python package for testing hardware (part of the magma ecosystem).

API Documentation

CHANGELOG

Installation

pip install fault

Documentation

Check out the fault tutorial

• Actions
• Tester
• Integrating External Verilog

Example

Here is a simple ALU defined in magma.

import magma as m
import mantle


class ConfigReg(m.Circuit):
    IO = ["D", m.In(m.Bits(2)), "Q", m.Out(m.Bits(2))] + \
        m.ClockInterface(has_ce=True)

    @classmethod
    def definition(io):
        reg = mantle.Register(2, has_ce=True, name="conf_reg")
        io.Q <= reg(io.D, CE=io.CE)


class SimpleALU(m.Circuit):
    IO = ["a", m.In(m.UInt(16)),
          "b", m.In(m.UInt(16)),
          "c", m.Out(m.UInt(16)),
          "config_data", m.In(m.Bits(2)),
          "config_en", m.In(m.Enable),
          ] + m.ClockInterface()

    @classmethod
    def definition(io):
        opcode = ConfigReg(name="config_reg")(io.config_data, CE=io.config_en)
        io.c <= mantle.mux(
            [io.a + io.b, io.a - io.b, io.a * io.b, io.a / io.b], opcode)

Here's an example test in fault that uses the configuration interface, expects a value on the internal register, and checks the result of performing the expected operation.

import operator
import fault

ops = [operator.add, operator.sub, operator.mul, operator.floordiv]
tester = fault.Tester(SimpleALU, SimpleALU.CLK)
tester.circuit.CLK = 0
tester.circuit.config_en = 1
for i in range(0, 4):
    tester.circuit.config_data = i
    tester.step(2)
    tester.circuit.a = 3
    tester.circuit.b = 2
    tester.eval()
    tester.circuit.c.expect(ops[i](3, 2))

We can run this with three different simulators

tester.compile_and_run("verilator", flags=["-Wno-fatal"], directory="build")
tester.compile_and_run("system-verilog", simulator="ncsim", directory="build")
tester.compile_and_run("system-verilog", simulator="vcs", directory="build")

Working with internal signals

Fault supports peeking, expecting, and printing internal signals. For the verilator target, you should use the keyword argument magma_opts with "verilator_debug" set to true. This will cause coreir to compile the verilog with the required debug comments. Example:

tester.compile_and_run("verilator", flags=["-Wno-fatal"], 
                       magma_opts={"verilator_debug": True}, directory="build")

If you're using mantle.Register from the coreir implementation, you can also poke the internal register value directly using the value field. Notice that conf_reg is defined in ConfigReg to be an instance of mantle.Register and the test bench pokes it by setting confg_reg.value equal to 1.

tester = fault.Tester(SimpleALU, SimpleALU.CLK)
tester.circuit.CLK = 0
# Set config_en to 0 so stepping the clock doesn't clobber the poked value
tester.circuit.config_en = 0
# Initialize
tester.step(2)
for i in reversed(range(4)):
    tester.circuit.config_reg.conf_reg.value = i
    tester.step(2)
    tester.circuit.config_reg.conf_reg.O.expect(i)
    # You can also print these internal signals using the getattr interface
    tester.print("O=%d\n", tester.circuit.config_reg.conf_reg.O)

FAQ

How can I write test bench logic that depends on the runtime state of the circuit?

A common pattern in testing is to only perform certain actions depending on the state of the circuit. For example, one may only want to expect an output value when a valid signal is high, ignoring it otherwise. Another pattern is to change the expected value over time by using a looping structure. Finally, one may want to expect a value that is a function of other runtime values. To support these pattern, fault provides support "peeking" values, performing expressions on "peeked" values, if statements, and while loops.

Peek Expressions

Suppose we had a circuit as follows:

class BinaryOpCircuit(m.Circuit):
    IO = ["I0", m.In(m.UInt[5]), "I1", m.In(m.UInt[5])]
    IO += ["O", m.Out(m.UInt[5])]

    @classmethod
    def definition(io):
        m.wire(io.O, io.I0 + io.I1 & (io.I1 - io.I0))

We can write a generic test that expects the output O in terms of the inputs I0 and I1 (rather than computing the expected value in Python).

tester = fault.Tester(BinaryOpCircuit)
for _ in range(5):
    tester.poke(tester._circuit.I0, hwtypes.BitVector.random(5))
    tester.poke(tester._circuit.I1, hwtypes.BitVector.random(5))
    tester.eval()
    expected = tester.circuit.I0 + tester.circuit.I1
    expected &= tester.circuit.I1 - tester.circuit.I0
    tester.circuit.O.expect(expected)

This is a useful pattern for writing reuseable test components (e.g. composign the output checking logic with various input stimuli generators).

Control Structures

The tester._while(<test>) action accepts a Peek value or expression as the test condition for a loop and returns a child tester that allows the user to add actions to the body of the loop. Here's a simple example that loops until a done signal is asserted, printing some debug information in the loop body:

# Wait for loop to complete
loop = tester._while(dut.n_done)
debug_print(loop, dut)
loop.step()
loop.step()

# check final state
tester.circuit.count.expect(expected_num_cycles - 1)

Notice that you can also add actions after the loop to check expected behavior after the loop has completed.

The tester._if(<test>) action behaves similarly by accepting a test peek value or expression and conditionally executes actions depending on the result of the expression. Here is a simple example:

if_tester = tester._if(tester.circuit.O == 0)
if_tester.circuit.I = 1
else_tester = if_tester._else()
else_tester.circuit.I = 0
tester.eval()

What Python values can I use to poke/expect ports?

Here are the supported Python values for poking the following port types:

• m.Bit - bool (True/False) or int (0/1) or hwtypes.Bit
• m.Bits[N] - hwtypes.BitVector[N], int (where the number of bits used to express it is equal to N)
• m.SInt[N] - hwtypes.SIntVector[N], int (where the number of bits used to express it is equal to N)
• m.UInt[N] - hwtypes.UIntVector[N], int (where the number of bits used to express it is equal to N)
• m.Array[N, T] - list (where the length of the list is equal to N and the elements recursively conform to the supported types of values for T). For example, suppose I have a port I of type m.Array[3, m.Bits[3]]. I can poke it as follows:

  val = [random.randint(0, (1 << 4) - 1) for _ in range(3)]
  tester.poke(circ.I, val)

  You can also poke it by element as follows:

  for i in range(3):
      val = random.randint(0, (1 << 4) - 1)
      tester.poke(circ.I[i], val)
      tester.eval()
      tester.expect(circ.O[i], val)

• m.Tuple(a=m.Bits[4], b=m.Bits[4]) - tuple (where the length of the tuple is equal to the number of fields), dict (where there is a one-to-one mapping between key/value pairs to the tuple fields). Example:

  tester.circuit.I = (4, 2)
  tester.eval()
  tester.circuit.O.expect((4, 2))
  tester.circuit.I = {"a": 4, "b": 2}
  tester.eval()
  tester.circuit.O.expect({"a": 4, "b": 2})

How do I generate waveforms with fault?

Fault supports generating .vcd dumps when using the verilator and system-verilog/ncsim target.

For the verilator target, use the flags keyword argument to pass the --trace flag. For example,

tester.compile_and_run("verilator", flags=["-Wno-fatal", "--trace"])

The --trace flag must be passed through to verilator so it generates code that supports waveform dumping. The test harness generated by fault will include the required logic for invoking tracer->dump(main_time) for every call to eval and step. main_time is incremented for every call to step. The output .vcd file will be saved in the file logs/{circuit_name} where circuit_name is the name of the ciruit passed to Tester. The logs directory will be placed in the same directory as the generated harness, which is controlled by the directory keyword argument (by default this is "build/").

For the system-verilog/ncsim target, tracing is enabled by default. For ncsim, the trace will be placed in a file called verilog.vcd in the same directory as the generated harness.