Note: Folk is in a pre-alpha state and isn't yet well-documented or well-exampled.

We're making Folk's source code free and available to the public, in case you're already excited about trying it, but we haven't formally announced it or made it ready for public use. We make no guarantee of support, of usability, or of continuing backward compatibility. Try at your own risk!

We're working on a more formal announcement, which will talk more about the goals of the project & provide canonical examples/demos to show what's possible. If you don't know what this is, then you might want to wait for that release.

---

Folk

Folk is a physical computing system: reactive database, programming environment, projection mapping. Instead of a phone/laptop/touchscreen/mouse/keyboard, your computational objects are physical objects in the real world, and you can program them inside the system itself. Folk is written in a mix of C and Tcl.

Installation

Hardware

You'll need to set up a dedicated PC to run Folk and connect to webcam+projector+printer+etc.

We tend to recommend a Beelink mini-PC (or maybe a Pi 5).

See https://folk.computer/pilot/

Manual Linux tabletop installation

On an Intel/AMD PC, set up Ubuntu Server 24.04 LTS (Noble Numbat).

(if you have an NVIDIA GPU, don't install the drivers in the Ubuntu OS installer. we'll install manually later. or if you do, install driver version 570 or newer)

(for a Pi 4/5, use Raspberry Pi Imager and get Raspberry Pi OS Lite 64-bit version [also see this issue if flashing from a Mac] -- Ubuntu doesn't have a good kernel for Pi 5)

1. Install Linux with username folk, hostname folk-SOMETHING? (check hosts.tcl in this repo to make sure you're not reusing one)

   If no folk user, then:

    sudo useradd -m folk; sudo passwd folk
   

   Add the folk user to groups:

    for group in adm dialout cdrom sudo audio video plugdev games users input tty render netdev lpadmin gpio i2c spi; do sudo usermod -a -G $group folk; done; groups folk
   

2. sudo apt update

3. Set up OpenSSH server if needed; connect to network. To ssh into folk@folk-WHATEVER.local by name, sudo apt install avahi-daemon and then on your laptop: ssh-copy-id folk@folk-WHATEVER.local

4. Install dependencies: sudo apt install rsync git libturbojpeg0-dev libpng-dev libdrm-dev pkg-config v4l-utils vulkan-tools libvulkan-dev libvulkan1 meson libgbm-dev glslc vulkan-validationlayers ghostscript console-data kbd psmisc zlib1g-dev libssl-dev automake libtool autoconf-archive

   (When prompted while installing console-data for Policy for handling keymaps type 3 (meaning 3. Keep kernel keymap) and press Enter)

   1. on a non-NVIDIA GPU: sudo apt install mesa-vulkan-drivers
   2. on an NVIDIA GPU: run sudo ubuntu-drivers install nvidia:580
   3. for debugging: elfutils (provides eu-stack), google-perftools, libgoogle-perftools-dev

5. Vulkan testing (optional):

   1. Try vulkaninfo and see if it works.

      1. On a Pi, if vulkaninfo reports "Failed to detect any valid GPUs in the current config", add dtoverlay=vc4-kms-v3d to the bottom of /boot/firmware/config.txt. (https://raspberrypi.stackexchange.com/questions/116507/open-dev-dri-card0-no-such-file-or-directory-on-rpi4)

   2. Try vkcube:

      git clone https://github.com/krh/vkcube
      cd vkcube
      mkdir build; cd build; meson .. && ninja
      ./vkcube -m khr -k 0:0:0
      

      If vkcube says Assertion ``vc->image_count > 0' failed, you might be able to still skip vkcube and continue the install process. See this bug

   3. See notes and Naveen's notes.

6. sudo sh -c 'echo SUBSYSTEM=="input", GROUP="input", MODE="0666" > /etc/udev/rules.d/99-input.rules && udevadm control --reload-rules && udevadm trigger'

7. Add the systemd service so it starts on boot and can be managed when you run it from laptop. On Ubuntu Server or Raspberry Pi OS (as root) (from here):

   # cat >/etc/systemd/system/folk.service
   [Unit]
   Description=Folk service
   After=network.target
   StartLimitIntervalSec=0
   
   [Service]
   Type=simple
   Restart=always
   RestartSec=1
   User=folk
   WorkingDirectory=/home/folk/folk
   ExecStart=make -C /home/folk/folk start
   
   [Install]
   WantedBy=multi-user.target
   

   Run these commands as root after editing the file above:

   # chmod 644 /etc/systemd/system/folk.service
   # systemctl start folk
   # systemctl enable folk
   

Use visudo to add folk ALL=(ALL) NOPASSWD: /usr/bin/systemctl to the bottom of /etc/sudoers on the tabletop. (This lets the make scripts from your laptop manage the Folk service by running systemctl without needing a password.)

To compile and run Folk:

$ make deps
$ make && ./folk

or (if remote machine):

$ make remote FOLK_REMOTE_NODE=<your-remote-hostname-here>

On your laptop Web browser, go to http://.local:4273 -- you should see the Folk web page with the live statement set.

Printer support

On the tabletop:

$ sudo apt update
$ sudo apt install cups cups-bsd
$ sudo usermod -a -G lpadmin folk

(cups-bsd provides the lpr command that we use to print)

ssh tunnel to get access to CUPS Web UI: run on your laptop ssh -L 6310:localhost:631 folk@folk-WHATEVER.local, leave it open

Go to http://localhost:6310 on your computer, go to Printers, hopefully it shows up there automatically, try printing test page. I could not get that implicitclass:// automatically-added printer in CUPS to work for my printer at all, so I did the below:

If job is paused due to cups-browsed issue or otherwise doesn't work, try https://askubuntu.com/questions/1128164/no-suitable-destination-host-found-by-cups-browsed : remove cups-browsed sudo apt-get purge --autoremove cups-browsed then add printer manually via IPP in Add Printer in Administration tab of CUPS Web UI (it might automatically show up under Discovered Network Printers there using dnssd)

Once printer is working, go to Administration dropdown on printer page and Set as Server Default.

Try printing from Folk!

You can also test printing again with lpr ~/folk-data/program/SOMETHING.pdf (you have to print the PDF and not the PS for it to work, probably)

Projector-camera setup and calibration

1. Make sure Folk is running. Go to your Folk server's Web page http://whatever.local:4273/setup . Select your camera and projector, using as high framerate and resolution as you're comfortable with.

2. You should see a live preview of the camera. Reposition your camera to cover your table closely (try not to waste too much of the camera viewport on pixels that the projector won't be able to hit)

3. Select your camera and projector at the bottom and then click Calibrate. Follow the calibration instructions.

After calibrating, on http://whatever.local:4273/ : click New Program, hit Save, drag it around. You should see the program move on your table as you drag it around on your laptop.

Connect a keyboard

Follow the instructions on this Folk wiki page to connect a new keyboard to your system.

Bluetooth keyboards

Install bluetoothctl. Follow the instructions in https://wiki.archlinux.org/title/bluetooth_keyboard to pair and trust and connect.

(FIXME: Write down the Bluetooth MAC address of your keyboard. We'll proceed as though it's "f4:73:35:93:7f:9d" (it's important that you turn it into lowercase).)

Development tools

Address Sanitizer

Address Sanitizer (ASan) support can be enabled by setting the ASAN_ENABLE environment variable:

make ASAN_ENABLE=1

Then when running:

ASAN_ENABLE=1 make start

Tracy profiling

To use Tracy, first do git submodule update --init here.

Pass CFLAGS=-DTRACY_ENABLE to make when compiling Folk (you might need to clean first if you already built Folk).

Ideally, run Folk as root with sudo ./folk or make sudo-remote.

Build the profiler client on your laptop/other PC:

$ cmake -B vendor/tracy/profiler/build -S vendor/tracy/profiler -DCMAKE_BUILD_TYPE=Release
$ make -C vendor/tracy/profiler/build

Run the profiler client on your laptop/other PC and connect to a running Folk:

$ make run-tracy

Potentially useful

Potentially useful for graphs: graphviz

Potentially useful: gdb, streamer, cec-utils, file, strace

Potentially useful: add folk-WHATEVER shortcut to your laptop ~/.ssh/config:

Host folk-WHATEVER
     HostName folk-WHATEVER.local
     User folk

Potentially useful: journalctl -f -u folk to see log of folk service

For audio: https://askubuntu.com/questions/1349221/which-packages-should-be-installed-to-have-sound-output-working-on-minimal-ubunt

Troubleshooting

HDMI No signal on Pi 4

Edit /boot/cmdline.txt raspberrypi/firmware#1647 (comment) (HDMI-A-1 or HDMI-A-2 depending on which port)

Ubuntu Server boots slowly

https://askubuntu.com/questions/1321443/very-long-startup-time-on-ubuntu-server-network-configuration (add optional: true to all netplan interfaces)

Why is my camera slow (why is tracking janky or laggy, why is camera time high)

Check that camera is plugged into a USB3 port

Turn off autoexposure and autofocus

for example, install v4l-utils and:

v4l2-ctl -c auto_exposure=1
v4l2-ctl -c focus_automatic_continuous=0
v4l2-ctl -c white_balance_automatic=0

Tcl troubleshooting

You can build Tcl with TCL_MEM_DEBUG. Download Tcl source code. (On Mac, do not go to the macosx/ subdir; go to the unix/ subdir.) Do ./configure --enable-symbols=all, do make, make install

License

Folk is available under the Apache 2.0 license. See the LICENSE file for more information.

Language reference

Folk is built around Tcl. We don't add any additional syntax or preprocessing to the basic Tcl language; all our 'language constructs' like When and Wish are really just plain Tcl functions that we've created. Therefore, it will eventually be useful for you to know basic Tcl syntax.

These are all implemented in main.tcl. For most things, you'll probably only need Wish, Claim, When, and maybe Hold!.

Wish and Claim

Wish $this is labelled "Hello, world!"

Claim $this is cool
Claim Omar is cool

When

When /actor/ is cool {
   Wish $this is labelled "$actor seems pretty cool"
   Wish $actor is outlined red
}

The inside block (body) of the When gets executed for each claim that is being made that it matches. It will get reactively rerun whenever a new matching claim is introduced.

Any wishes/claims you make in the body will get automatically revoked if the claim that the When was matching is revoked. (so if Omar stops being cool, the downstream label Omar seems pretty cool will go away automatically)

The /actor/ in the When binds the variable actor to whatever is at that position in the statement.

It's like variables in Datalog, or parentheses in regular expressions.

Non-capturing

/someone/, /something/, /anyone/, /anything/ are special cases if you want a wildcard that does not bind (you don't care about the value, like non-capturing groups (?:) in regex), so you don't get access to $someone or $something inside the When.

Negation

/nobody/, /nothing/ invert the polarity of the match, so it'll run only when no statements exist that it would match.

This When will stop labelling if someone does Claim Omar is cool:

When /nobody/ is cool {
   Wish $this is labelled "nobody is cool"
}

& joins

You can match multiple patterns at once:

Claim Omar is cool
Claim Omar is a person with 2 legs
When /x/ is cool & /x/ is a person with /n/ legs {
   Wish $this is labelled "$x is a cool person with $n legs"
}

Notice that x here will have to be the same in both arms of the match.

You can join as many patterns as you want, separated by &.

If you want to break your When onto multiple lines, remember to terminate each line with a \ so you can continue onto the next line:

When /x/ is cool & \
    /x/ is a person with /n/ legs {
  Wish $this is labelled "$x is a cool person with $n legs"
}

Collecting results

When the collected results for [list /actor/ is cool] are /results/ {
   Wish $this is labelled [join $results "\n"]
}

This gets you an array of all results for the pattern /actor/ is cool.

(We use the Tcl list function to construct a pattern as a first-class object. You can use & joins in that pattern as well.)

Hold!

Experimental: Hold! is used to register claims that will stick around until you do another Hold!. You can use this to create the equivalent of 'variables', stateful statements.

Hold! { Claim $this has a ball at x 100 y 100 }

When $this has a ball at x /x/ y /y/ {
    puts "ball at $x $y"
    After 10 milliseconds {
        Hold! { Claim $this has a ball at x $x y [expr {$y+1}] }
        if {$y > 115} { set ::done true }
    }
}

Hold! will overwrite all statements made by the previous Hold! (scoped to the current $this).

Notice that you should scope your claim: it's $this has a ball, not there is a ball, so different programs with different values of $this will not stomp over each other. Not scoping your claims will bite you once you print your program and have both virtual & printed instances of your program running.

If you want multiple state atoms, you can also provide a key -- you can be like

Hold! ball position {
  Claim $this has a ball at blahblah
}

and then future holds with that key, ball position, will overwrite this statement but not override different holds with different keys

You can overwrite another program's Hold! with the on parameter, like Hold! (on 852) { ... } (if the Hold! is from page 852) or Hold! (on builtin-programs/example.folk) { ... } (if the Hold! is from the example.folk builtin program)

Subscribe: and Notify:

Subscribe: subscribes to notifications. It executes once per Notify: and cannot support any Claims/Whens/Wishes in its body.

You can't make Claims, Whens, or Wishes inside a Subscribe: block. You can only Hold!.

Example:

Hold! { Claim $this has seen 0 boops }

Subscribe: there is a boop {
  ForEach! $this has seen /n/ boops {
    Hold! { Claim $this has seen [expr {$n + 1}] boops }
  }
}

Notify: there is a boop

Animation

Getting time

Get the global clock time with:

When the clock time is /t/ {
  Wish $this is labelled $t
}

Use it in an animation:

When the clock time is /t/ {
  Wish $this draws a circle with offset [list [expr {sin($t) * 50}] 0]
}

You usually won't need these

When when

Lets you create statements only on demand, when someone is looking for that statement.

When /thing/ is cool {
    Wish $this is labelled "$thing is cool"
}
When when /personVar/ is cool /lambda/ with environment /e/ {
    Claim Folk is cool
}

On unmatch

You should not use When, Claim, or Wish directly inside an On unmatch block; those only make sense inside a normal reactive context.

set pid [exec python3]
On unmatch {
    kill $pid
}

Non-capturing

You can disable capturing of lexical context around a When with the -noncapturing flag.

This is mostly to help runtime performance if a When is declared somewhere that has a lot of stuff in scope at declaration time.

set foo 3
When -noncapturing /p/ is cool {
   Claim $p is awesome
   # can't access $foo from in here
}

Assert! and Retract!

General note: Assert! and Retract! are used for weird non-reactive behavior.

You should generally not use Assert! and Retract! inside a When block. Use Claim, Wish, and When instead.

Tcl for JavaScripters

JS:

let names = ["64", "GameCube", "Wii", "Switch"];
names = names.map(name => `Nintendo ${name}`);
console.log(names);

function add(a, b) { return a + b; }
const numbers = [1, 2];
console.log(add(...numbers));

Tcl:

set names [list 64 GameCube Wii Switch]
set names [lmap name $names {expr {"Nintendo $name"}}]
puts $names

proc add {a b} { expr {$a + $b} }
set numbers [list 1 2]
puts [add {*}$numbers]

Style guide

Tcl code vs. builtin programs vs. printed programs

In general, avoid adding new .tcl files to the Git repo. Pure Tcl libraries are an antipattern; we should only need them for the hard core of the system.

Most new code (both libraries and applications) should be virtual programs (which ilve as .folk files in the builtin-programs/ subfolder) or printed programs.

Folk

• Use complete sentences when you word your claims and wishes.

  Bad: Claim $this firstName Omar

  Good: Claim $this has first name Omar

• Scope using $this where appropriate to prevent weird global interactions

  Bad: Claim the value is 3

  Good: Claim $this has value 3

• Style for joins across multiple lines -- use &\ and align on the first token after When:

  When the fox is out &\
       the label is "Hello" &\
       everything seems good {
    ...
  }
  

Tcl

fn

Use fn instead of proc to get a lexically captured command.

Error handling

Use try (and on error) in new code. Avoid using catch; it's older and easier to get wrong.

apply

Use apply instead of subst to construct lambdas/code blocks, except for one-liners (where you can use list)

Tcl datatypes

Create a namespace for your datatype that is an ensemble command with operations on that datatype.

(Examples: statement, c, region, point, image)

Call the constructor create, as in dict create and statement create.

Singletons

Capitalized namespace, like Statements.

Thanks

• Omar Rizwan: evaluator; display, AprilTag, camera subsystems
• Andrés Cuervo: keyboard library and code editor, sprites
• Naveen Michaud-Agrawal: connections, points-at,
• Jacob Haip: web endpoints, tag masking, blob detection
• Arcade Wise: fonts
• s-ol bekic: WebSocket library, keyboard locale support
• terminal subsystem
• Mason Jones: tag stabilization, Notify:/Subscribe: event system, -save statement persistence, new code editor with scrolling & font size & editing of existing programs

todo

• clean up shader reference errors (use trick from main?)
• fix camera-rpi corruption
• restore smj cam/display parameters
• fix remaining display/ primitives
• rebuild live image
• why is web endpoints so slow?
• optimize jpeg decoding
• only intern long strings?
• accidentally matches prefixes even when not all teh way up to end of statement
• camera stops working when calibration terminates
  • laser-cut or cnc or 3d print a plate with 2 sliders and a slot for a program
• camera slices cause hop/distortion when pulled off

next

• calibrate render loop blinks out regularly
• calibrate doesn't click in afterward, have to restart system (is it because of kill refiner?)
• rename resolved geometry to geometry
• On unmatch doesn't work if run at start of When block instead of end? -- it's probably because it gets pinned through descendant statements
• fix error reporting on table clean up title, clean up points-at
• "Added tag 1313" pileup (and removal pileup when flipped over)
• fix calibration screwing up system state
• persist transient errors

ideas

• aborted executions shouldn't be too high a percentage of total # of executions of the block? if they are, then we warn on the page that it isn't meeting timing
• build a settlement-based local fps counter like clock time labeler (how many frames are we dropping?)

other stuff

• stereo calibration
• run segmentation model
• editor cutoff bug
• editor 132 keyboard stops responding?
• recsale camera slice to have correct aspect ratio
• debug memory leaks
  • are we ever freeing AtomicallyVersion? -- no, but this isn't the main memory leak (not enough allocated)
  • not destructors
  • also not camera images
  • also not jim allocations
• bug where camera slices halt / slow down animation
  • animation blinky
• gadget-platinum outline blink
• remaining blink on clock time
  • the problem is that the root match has no atomic marker attached in the db (the When the clock time is /t/), which means that any direct child statements don't get made if they aren't made before the root match is revoked.
    • we need to also find a way to pin the root match
• slowdown where sysmon starts taking forever bc of endless chains of destructors/atomicallyversions
  • warn if sysmon is too slow?
• make RAM/metrics page to not clutter up stdout
• make errors page
• remove Hold and Atomically limits
• automatic default calibration so you can drag stuff around on laptop
• automatically allow optional fields on with
• try to maximize cpu usage
• dual camera calibration -> ML
• scan for invariant violation (statement w/o trie entry)
  • look at trie (make separate trie walker that gives us the interior pointer to examine)