Init script called multiple times at boot on TP-Link v9, launching duplicate rayhunter instances cause ioctl failure

[thatsnailguy]

Prerequisites

• I have read CONTRIBUTING.md

Bug Report Details

VIBE CODE WARNING

Rayhunter v0.10.1 logs appear broken on TP-Link M7350 v9 out of the box. The log shows 11 retries followed by DiagInitError. A working instance is actually running but its log was overwritten. It appears that the init is called multiple times each boot. Each call launches a new rayhunter-daemon via start-stop-daemon -b. The first instance claims the MD session (mask=31) and works. Subsequent instances fail the ioctl (MD session already taken) and their log truncation (> rayhunter.log) overwrites the first instance's success log. The user sees only -1 failures even though a working instance exists.

The port triggering rule that starts rayhunter:

triggerPort: "$(/etc/init.d/rayhunter_daemon start &)"

might be evaluated multiple times as network interfaces come up, each time executing the init script. Each call launches a new rayhunter-daemon because start-stop-daemon uses --startas /bin/sh without --exec

Related: #676, #683

Log output (stock init script)

R A Y H U N T E R 🐳
[2026-02-19T19:08:15Z INFO  rayhunter_daemon] Using configuration for device: Tplink
[2026-02-19T19:08:15Z INFO  rayhunter::diag_device] Diag device initialization failed 1 times, retrying in 100ms: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:15Z INFO  rayhunter::diag_device] Diag device initialization failed 2 times, retrying in 200ms: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:15Z INFO  rayhunter::diag_device] Diag device initialization failed 3 times, retrying in 400ms: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:15Z INFO  rayhunter::diag_device] Diag device initialization failed 4 times, retrying in 800ms: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:16Z INFO  rayhunter::diag_device] Diag device initialization failed 5 times, retrying in 1.6s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:18Z INFO  rayhunter::diag_device] Diag device initialization failed 6 times, retrying in 3.2s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:21Z INFO  rayhunter::diag_device] Diag device initialization failed 7 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:27Z INFO  rayhunter::diag_device] Diag device initialization failed 8 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:32Z INFO  rayhunter::diag_device] Diag device initialization failed 9 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:37Z INFO  rayhunter::diag_device] Diag device initialization failed 10 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:42Z INFO  rayhunter::diag_device] Diag device initialization failed 11 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:47Z ERROR rayhunter::diag_device] Failed to initialize diag device after 30s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
Error: DiagInitError(InitializationFailed("DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1"))

Workaround

Add --exec /data/rayhunter/rayhunter-daemon so start-stop-daemon detects the already-running instance:

     start-stop-daemon -S -b --make-pidfile --pidfile /tmp/rayhunter.pid \
+    --exec /data/rayhunter/rayhunter-daemon \
     --startas /bin/sh -- -c "RUST_LOG=info exec /data/rayhunter/rayhunter-daemon /data/rayhunter/config.toml > /data/rayhunter/rayhunter.log 2>&1"

Environment

• Device: TP-Link M7350 v9.0
• Firmware: 9.0.3 Build 241219 Rel.1089n
• Modem: Quectel EC25EUC (firmware EC25EUCGAR08A01M2G_OCPU_01.001.04)
• Chip: Qualcomm MDM9607
• Rayhunter: v0.10.1
• Host OS: Windows 11

EDIT Original context

## Summary

On the TP-Link M7350 v9.0, rayhunter fails to initialize /dev/diag on every boot with DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1. The cause is system daemons that hold /dev/diag open at boot, blocking rayhunter's ioctl. Killing these processes before starting rayhunter resolves the issue.

Related: #676, #683

Device

• TP-Link M7350 v9.0, firmware 9.0.3 Build 241219
• Rayhunter v0.10.1

Log output (stock init script)

R A Y H U N T E R 🐳
[2026-02-19T19:08:15Z INFO  rayhunter_daemon] Using configuration for device: Tplink
[2026-02-19T19:08:15Z INFO  rayhunter::diag_device] Diag device initialization failed 1 times, retrying in 100ms: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:15Z INFO  rayhunter::diag_device] Diag device initialization failed 2 times, retrying in 200ms: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:15Z INFO  rayhunter::diag_device] Diag device initialization failed 3 times, retrying in 400ms: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:15Z INFO  rayhunter::diag_device] Diag device initialization failed 4 times, retrying in 800ms: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:16Z INFO  rayhunter::diag_device] Diag device initialization failed 5 times, retrying in 1.6s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:18Z INFO  rayhunter::diag_device] Diag device initialization failed 6 times, retrying in 3.2s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:21Z INFO  rayhunter::diag_device] Diag device initialization failed 7 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:27Z INFO  rayhunter::diag_device] Diag device initialization failed 8 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:32Z INFO  rayhunter::diag_device] Diag device initialization failed 9 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:37Z INFO  rayhunter::diag_device] Diag device initialization failed 10 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:42Z INFO  rayhunter::diag_device] Diag device initialization failed 11 times, retrying in 5s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
[2026-02-19T19:08:47Z ERROR rayhunter::diag_device] Failed to initialize diag device after 30s: Failed to initialize /dev/diag: DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1
Error: DiagInitError(InitializationFailed("DIAG_IOCTL_SWITCH_LOGGING ioctl failed with error code -1"))

Note: Even though the log reports initialization failure, rayhunter was still producing QMDL entries with data. However, as with #676, the quality/reliability of that data is uncertain.

Root cause

On the M7350 v9.0, several system daemons hold /dev/diag open at boot. While they hold the device, rayhunter's DIAG_IOCTL_SWITCH_LOGGING ioctl fails. The 30-second retry window is not enough — these processes run permanently.

The processes I found holding /dev/diag on my device:

• diagrebootapp (respawned by /etc/inittab)
• services_monitor (respawned by /etc/inittab)
• qti
• qmuxd
• MCM_MOBILEAP_ConnectionManager
• quectel_daemon
• mobile
• ql_manager_server
• thermal-engine

Solution

Modified the init script to kill these processes before starting rayhunter. Two of them (diagrebootapp, services_monitor) are respawned by inittab, so a double-kill is needed: kill, wait for inittab to respawn, kill again. After rayhunter claims /dev/diag, the cellular daemons are restarted so connectivity is restored.

Log output (modified init script)

R A Y H U N T E R 🐳
[2026-02-19T19:11:03Z INFO  rayhunter_daemon] Using configuration for device: Tplink
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] Diag device initialization succeeded after 0 retries
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] retrieving diag logging capabilities...
[2026-02-19T19:11:03Z WARN  rayhunter::diag] warning: 329 leftover bytes when parsing Message
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] enabled logging for log type 1
[2026-02-19T19:11:03Z WARN  rayhunter::diag] warning: 298 leftover bytes when parsing Message
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] enabled logging for log type 4
[2026-02-19T19:11:03Z WARN  rayhunter::diag] warning: 140 leftover bytes when parsing Message
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] enabled logging for log type 5
[2026-02-19T19:11:03Z WARN  rayhunter::diag] warning: 168 leftover bytes when parsing Message
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] enabled logging for log type 7
[2026-02-19T19:11:03Z WARN  rayhunter::diag] warning: 122 leftover bytes when parsing Message
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] enabled logging for log type 10
[2026-02-19T19:11:03Z WARN  rayhunter::diag] warning: 73 leftover bytes when parsing Message
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] enabled logging for log type 11
[2026-02-19T19:11:03Z WARN  rayhunter::diag] warning: 72 leftover bytes when parsing Message
[2026-02-19T19:11:03Z INFO  rayhunter::diag_device] enabled logging for log type 13
[2026-02-19T19:11:03Z INFO  rayhunter_daemon] Starting Diag Thread
[2026-02-19T19:11:03Z INFO  rayhunter_daemon] Starting UI
[2026-02-19T19:11:03Z INFO  rayhunter_daemon::display::tplink] fallback to framebuffer
[2026-02-19T19:11:03Z INFO  rayhunter_daemon] Starting Key Input service
[2026-02-19T19:11:03Z INFO  rayhunter_daemon] create shutdown thread
[2026-02-19T19:11:03Z INFO  rayhunter_daemon] spinning up server
[2026-02-19T19:11:03Z INFO  rayhunter_daemon] The orca is hunting for stingrays...
[2026-02-19T19:11:03Z WARN  rayhunter_daemon::battery] Failed to get battery status: Error parsing file to determine battery level

Note: The battery warning only appears at startup. Battery status updates correctly on subsequent checks.

Init script fix (diff against installer-generated stock script)

--- a/dist/scripts/rayhunter_daemon
+++ b/dist/scripts/rayhunter_daemon
@@ -6,8 +6,30 @@
 case "$1" in
 start)
     echo -n "Starting rayhunter: "
     (mount /dev/mmcblk0p1 /media/sdcard || true) 2>&1 | tee /tmp/rayhunter-mount.log
+    # Kill all processes that hold /dev/diag and create competing MD sessions.
+    # diagrebootapp and services_monitor are respawned by inittab, so we must
+    # kill them, wait for init to respawn them, then kill again.
+    killall -9 services_monitor 2>/dev/null || true
+    killall -9 diagrebootapp 2>/dev/null || true
+    killall -9 MCM_MOBILEAP_ConnectionManager 2>/dev/null || true
+    killall -9 qti 2>/dev/null || true
+    killall -9 qmuxd 2>/dev/null || true
+    killall -9 quectel_daemon 2>/dev/null || true
+    killall -9 mobile 2>/dev/null || true
+    killall -9 ql_manager_server 2>/dev/null || true
+    killall -9 thermal-engine 2>/dev/null || true
+    sleep 1
+    killall -9 services_monitor 2>/dev/null || true
+    killall -9 diagrebootapp 2>/dev/null || true
+    killall -9 MCM_MOBILEAP_ConnectionManager 2>/dev/null || true
+    sleep 1
     start-stop-daemon -S -b --make-pidfile --pidfile /tmp/rayhunter.pid \
     --startas /bin/sh -- -c "RUST_LOG=info exec /data/rayhunter/rayhunter-daemon /data/rayhunter/config.toml > /data/rayhunter/rayhunter.log 2>&1"
+    # Restart killed daemons so cellular connectivity is restored
+    sleep 5
+    /etc/init.d/chgrp-diag start 2>/dev/null || true
+    /etc/init.d/quectel_daemon start 2>/dev/null || true
+    /etc/init.d/thermal-engine start 2>/dev/null || true
+    /etc/init.d/mobile-mgr start 2>/dev/null || true
+    /etc/init.d/start_ql_manager_server_le start 2>/dev/null || true
+    /etc/init.d/start_MCM_MOBILEAP_ConnectionManager_le start 2>/dev/null || true
     echo "done"
     ;;

Environment

• Device: TP-Link M7350 v9.0
• Firmware: 9.0.3 Build 241219 Rel.1089n
• Modem: Quectel EC25EUC (firmware EC25EUCGAR08A01M2G_OCPU_01.001.04)
• Chip: Qualcomm MDM9607
• Rayhunter: v0.10.1
• Host OS: Windows 11

[untitaker]

This has been mentioned elsewhere too (I forget where) but so far it seemed that all the other processes would eventually release their diag sessions. I wonder what the problem is now. Which processes have you seen hold /dev/diag and how far into the boot process/uptime? I'm basically wondering if retying indefinitely is a valid fix.

I'm not thrilled about the idea of hardcoding N different process names into the installer that we need to kill.

[thatsnailguy]

Replaced the init script with one that only mounts the SD card, then sampled fuser /dev/diag and /proc/*/fd at 6 intervals from uptime 81s through 457s (~7.5 minutes).

Result: 7 processes hold /dev/diag open for the entire observation period. Same PIDs, same file descriptors, zero variation across all 6 snapshots:

Process	Path	FD
qti	/usr/bin/qti	4
qmuxd	/usr/bin/qmuxd	4
thermal-engine	/usr/bin/thermal-engine	5
quectel_daemon	/usr/bin/quectel_daemon	4
mobile	/usr/bin/mobile	16
ql_manager_server	/usr/bin/ql_manager_server	7
diagrebootapp	/usr/bin/diagrebootapp	3

[untitaker]

I find it puzzling that all of these processes seem to get along fine with each other though. I see two possible explanations for this: Either their peripheral masks are non-overlapping (unlikely because surely at least two will use the modem), or most of them open diag without using it (yet) to open a debug logging session. afaik opening diag alone isn't acquiring a lock, it's the ioctl that does

[thatsnailguy]

😳You were right. Classic noob vibe coder barking up wrong tree. It is being blocked by itself. Updated body with better analysis.

[untitaker]

thanks, the fix for this one seems straightforward, I'll test it on orvic etc