Problem: when you "Stop" a bash/powershell script only the top process is stopped

[AndrewSav]

Issue

It's quite forseeable that your script use other native executables to accomplish what you need to do. Those executable can have children of their own. When we stop a task that is running a script, only the script itself is stopped, all the childern contine running in the background.

Impact

Other

Installation method

Docker

Database

Postgres

Browser

Chrome

Semaphore Version

2.16.45

Ansible Version

Available upon request, but does not seem to be relevant

Logs & errors

Available upon request, but it seems there is nothing relevant there.

Manual installation - system information

N/A

Configuration

Availabile upon request. Seems irrelevant to the issue.

Additional information

I had a brief look at the source code, and it does not appear that it attempts to kill proces group / children. If the process locks some resource remote, or local, then the fact that it does not die is a big issue. For example if it keeps a port open, then no subsequent task execution can succed if they are using the same port.

[AndrewSav]

For now, I use this as a work around to prevent new instances failing:

# ====================== Working Around Semaphore issue ======================
# https://github.com/semaphoreui/semaphore/issues/3512

function Test-ProcessHasScriptGroupId {
    param(
        [Parameter(Mandatory = $true)]
        [int] $targetPid,

        [Parameter(Mandatory = $true)]
        [string] $ExpectedValue
    )

    $procDir = "/proc/$targetPid"

    # /proc/<pid> must exist
    if (-not (Test-Path -LiteralPath $procDir)) {
        Write-Warning "Path '$procDir' does not exist"
        return $false
    }

    # Only consider processes owned by the same UID
    try {
        $procUid = (& stat -c '%u' $procDir 2>$null).Trim()
    }
    catch {
        Write-Warning "Failed getting UID from '$procDir'"
        return $false
    }

    if ($procUid -ne $currentUid) {
        return $false
    }

    # Read /proc/<pid>/environ (null-separated environment)
    $environPath = "$procDir/environ"
    if (-not (Test-Path -LiteralPath $environPath)) {
        Write-Warning "Path '$environPath' does not exist"
        return $false
    }

    try {
        $bytes = [System.IO.File]::ReadAllBytes($environPath)
    }
    catch {
        Write-Warning "Could not read '$environPath'"
        # Often not readable for other users / special processes; just skip
        return $false
    }

    if ($bytes.Length -eq 0) {
        return $false
    }

    $envString = [System.Text.Encoding]::UTF8.GetString($bytes)
    $entries = $envString -split "`0"

    # Look for exact "SCRIPT_GROUP_ID=<ExpectedValue>" match
    $targetEntry = "SCRIPT_GROUP_ID=$ExpectedValue"
    foreach ($entry in $entries) {
        if ($entry -eq $targetEntry) {
            return $true
        }
    }

    return $false
}

if ($IsLinux) {
    $env:SCRIPT_GROUP_ID = "yourGroupId"
    $selfPid = $PID
    # Current user's numeric UID (e.g. 1000)
    try {
        $currentUid = (& id -u).Trim()
    }
    catch {
        Write-Error "Failed to determine current UID via 'id -u'. Are you on Linux?"
        exit 1
    }

    Get-ChildItem -LiteralPath /proc -Directory |
    Where-Object { $_.Name -match '^\d+$' } |   # numeric PIDs only
    ForEach-Object {
        $targetPid = [int]$_.Name

        # Don't kill ourselves
        if ($targetPid -eq $selfPid) {
            return
        }

        if (Test-ProcessHasScriptGroupId -targetPid $targetPid -ExpectedValue $env:SCRIPT_GROUP_ID) {
            Write-Host "Killing PID $targetPid (same user, SCRIPT_GROUP_ID=$env:SCRIPT_GROUP_ID)..."

            try {
                # Send SIGKILL (9)
                & kill -9 -- $targetPid 2>$null
            }
            catch {
                Write-Warning "Failed to kill PID $targetPid`: $_"
            }
        }
    }
}

Of course this does not sovle the issue of processes left behind running forver.