Unified, vendored shell server

[untitaker]

Prerequisites

• I have read CONTRIBUTING.md

What problem does this feature solve or what does it enhance?

(this is more like an RFC right now and not planned work)

Our current experience of obtaining a shell is somewhat inconsistent across devices, and the quality varies:

• telnet without working prompt (orbic-network... and sometimes zte?)
• telnet without working stderr capture (zte Add support for ZTE MF920V #872)
• adb-only (uz801)
• or multiple options

This is mainly because the shell we give users is using whatever on-device tools are available, and is often just a very thin wrapper around them.

I think this hurts rayhunter's extensibility. People want to add random extensions like tailscale, messing with the wifi card or adding some type of notification bridge. There is no clean extension point to this, so one of two things happens:

• it's documented as something to run through the shell (in our FAQ) -- but how to get a shell is left as an exercise to the reader, and so the extension is only available to power users.
• the entire project is forked, these extensions are added and the fork eventually is abandoned (although vibecoding is a part of the reason this happens)

Ideally the people who write those features would be able to maintain them outside of rayhunter, write their own user-friendly installers, and those installers should not have to consider device differences at all just to get a shell on an existing rayhunter device.

Proposed Solution

There are two solutions i can see:

• We preinstall dropbear or our own telnetd on the device by default.
• rayhunter-daemon comes with its own telnet-like shell server.

I think the latter might be significantly easier to do and does not require many changes to the installer to have it guaranteed available on all devices.

If we go with the latter approach, we should be sure that rayhunter-daemon starts in a number of situations where it currently shuts down:

• sdcard not mounted (tplink)
• filesystem read-only or full
• /dev/diag is not available

When these things happen today, one can open a shell (somehow) to diagnose the issue. When rayhunter-daemon is the shell and it doesn't start up then, we take away the user's/dev's tools when they need them most.

Alternatives Considered

We refactor the installer internally (and impose more structure on the code) so that shell and file transfer support becomes more consistent that way. I think that will have to happen anyway for the GUI.

[BeigeBox]

I would vote for dropbear. I already have scripts to build it, and then install it on the Orbic I'm already planning on testing it on the Moxee too in the next few days.

[BeigeBox]

ADB on the Orbic is pretty locked down, so SSH gets around that issue.

[trevormjaymes]

I'm in the infancy of testing this, but its working so far - might take a look at the russh crate: https://gist.github.com/trevormjaymes/e7eb50cadb9ec96d66987d97733a43ed - This is a readme from an ssh module that interacts with the default dropbear config - there is no binary for dropbear out of box, but there is an xml config toggle as OEM leftover.

This is a much larger suggestion, and a bit of a thread hijack - I totally understand, and it does pull on the scope/focus of the project - but I want to make an argument in favor for it.

There is something to be said about a goal to secure these units for their intended purpose, they come with dozens if not on the order of hundreds of vulnerabilities. There's vulnerabilities within the tcp stack, TLS handshakes, and almost every binary. The very unfortunate part is most of the attack surface is on the carrier side, so if comms are hijacked - or there is a MITM - you are an open attack surface - to that end, a goal should likely be to secure the unit, and that brings the underlying firmware back into scope. To successfully achieve the goals/mission beyond academic paper exercises, there must be an intent to close the gaps and prevent further exploitation or circumvention, particurarly if/when nation states are involved. What's to stop mock data injection, denial of service, etc. ?

I am within a few weeks of publishing work-in-progress research that consolidates LCD display, HTTP API, network monitoring, power management, input handling, and service health into a single async rust process and I plan to replace/patch as much as feasible to reduce/mitigate/eliminate the attack surface.

It is an immense amount of work, but also tampering with system firmware - especially when persistence is involved - warrants strong backup/recovery. To that end, while I have successfully recovered and reflashed, when bad kernel modules caused bootloops and system level issues, etc., I've also irrecoverably altered partition data and flashed the wrong partitions, etc. hard bricking 2 units. The level of safety & guarantees are different and hard bricks are now in play, through Emergency DownLoad. I'm simply pointing out that the more that you modify the system, the more risk you inherit - the more of a need to be stroger about backup & recovery, and automation.

This is more just food for thought and commentary, I think there would/should be an OSS obligation to secure, but whether or not its a separate project or under the same umbrella is a good argument, of which I have no strong opinion. It will be hard to bring this level of research, and to ensure this level of capability for every supported device type - but its also a shame to leave them so damn vulnerable, these things need a lot of work.

Btw, you're going to run into another issue with diag and a few other, needing a multiplexer, so another argument for becoming/overtaking - this topic is the thread of a sweater, which is why I'm trying to pull on it:

        /dev/diag
            │
     ┌──────▼────────┐
     │  DiagDevice   │  Raw fd I/O, HDLC framing, NV commands
     └──────┬────────┘
            │
     ┌──────▼────────┐
     │  DiagServer   │  Async select! loop, broadcast, NV routing
     └──────┬────────┘
            │
   ┌────────┼────────┐
   │        │        │
┌──▼───┐ ┌──▼──┐ ┌───▼──────┐
│ HTTP │ │ IPC │ │ internal │
│ API  │ │     │ │  cache   │
└──────┘ └──┬──┘ └──────────┘
            │
     ┌──────▼──────┐
     │  rayhunter  │
     │  (client)   │
     └─────────────┘

                        ┌──────────────────────────────┐
     /dev/input/*  ───► │          Input Handler       │
                        └──────────────┬───────────────┘
                                       │ KeyInput, UserActivity
                                       ▼
┌──────────┐  status   ┌────────────────────────────────┐   events  ┌──────────┐
│ Status   │──────────►│           Event Bus            │◄──────────│Rayhunter │
│Collector │  events   │     (broadcast channel, 64)    │           │ Monitor  │
└──────────┘           └──┬──┬────┬───┬───┬───┬───┬─────┘           └──────────┘
                          │  │    │   │   │   │   │
             ┌────────────┘  │    │   │   │   │   └────────────┐
             ▼               ▼    │   ▼   │   ▼                ▼
        ┌────┴────┐    ┌─────┴┐   │ ┌─┴──┐│ ┌─┴────┐    ┌──────┴───┐
        │  LCD /  │    │ LED  │   │ │HTTP││ │Power │    │Shell/SSH │
        │  Menu   │    │      │   │ │ /WS││ │Mgr   │    │2323 / 22 │
        └────┬────┘    └──────┘   │ └──┬─┘│ └──────┘    └──────────┘
             │                    │    │  │
             ▼                    │    │  │
        ┌────┴────┐               │    ▼  ▼
        │orbic-fb │         ┌─────┴────┴──┴──────┐    ┌─────────────────┐
        │/dev/fb0 │         │  Config Manager    │    │  Service Health │
        └─────────┘         │  (XML read/write)  │◄───│    Monitor      │
                            └────────────────────┘    └───────┬─────────┘
                                                              │
                                          ┌───────────────────┼──────────────┐
                                          ▼                   ▼              ▼
                                     ┌─────────┐       ┌──────────┐   ┌──────────┐
                                     │ wlan.ko │       │ hostapd  │   │ dnsmasq  │
                                     │ insmod  │       │ AP daemon│   │ DHCP/DNS │
                                     └────┬────┘       └─────┬────┘   └──────────┘
                                          │                  │
                                          └──► wlan0 ◄───────┘
                                                │
                                              bridge0

[untitaker]

@trevormjaymes I do think you should make a separate thread for this, I don't think I follow all of what exactly you're doing.

As far as it relates to this issue, I think adding yet another network shell probably won't significantly change the security situation on these devices. I do hope that if/when we have a common interface to run installation routines against, people will be able to properly package things for ordinary users, like iptables rules to block OTA updates or scripts to disable wifi, and we can try those things at scale in the name of security hardening without having to ship them in core rayhunter from the start while they're still buggy. I'm not really sure though if that extension mechanism will work for replacing the entire firmware, that probably still requires a fork of rayhunter or full upstreaming.

[BeigeBox]

@trevormjaymes We should talk some time, there's a lot of big feature work I've been doing. Stuff like from creating a UI for the Orbic (replaces the stock ui on the devices screen), having the device join a wifi network in client mode, adding meshtastic USB support, host usb support, and ssh. I would hate for our work to conflict or cause major rewrites.

[trevormjaymes]

our work to conflict or cause major rewrites.

You should continue with your work. I only do art-of-the-possible and research, but it could certainly help inform/influence/inspire - I will make sure to reach out once its ready :)

[BeigeBox]

our work to conflict or cause major rewrites.

You should continue with your work. I only do art-of-the-possible and research, but it could certainly help inform/influence/inspire - I will make sure to reach out once its ready :)

And FYI I have a hardening script that I’ve been working on. It’s per device and something you would apply manually as an additional step after installing (optionally). It closes the critical and high severity issues at least on Moxee which is the first I started with. It is somewhat involved as it includes cross compiled safe versions of some daemons with known vulns; setting up iptables rules; and replacing some bins with a noop binary for unneeded daemons which are watchdoggdd among other things. Seems you’re familiar with that stuff. Lmk if you would like to collab on something like that.