Skip to content

Attach to specific multiprocessing subprocesses via the CLI? #1838

@garrett361

Description

@garrett361

Is it possible to attach to a specific multiprocessing subprocess using only the debugpy CLI?

Example:

import multiprocessing as mp

def target(rank):
    a = 2
    b = rank * a
    print(b)
    print(f"Done on {rank=}")

def test():
    procs = [
        mp.Process(target=target, name=f"rank-{rank}", args=(rank,))
        for rank in range(4)
    ]
    for p in procs:
        p.start()
    for p in procs:
        p.join()


if __name__ == "__main__":
    test()

with subProcess = True I can do python -m debugpy --listen 5678 --wait-for-client <path-to-file.py> and then attach to a (seemingly random, but usually rank-zero) process by specifying port = 5678 and debug that one process. But I can't figure out how to either attach to a specifc process or get any of the other ranks to step through their code.

Is it possible to achieve this just through the CLI?

One thing that does work is to explicitly insert the debugpy API calls into the source code:

import multiprocessing as mp

def target(rank):
    import debugpy

    debugpy.listen(5678 + rank)
    debugpy.wait_for_client()
    a = 2
    b = rank * a
    print(b)
    print(f"Done on {rank=}")

def test():
    procs = [
        mp.Process(target=target, name=f"rank-{rank}", args=(rank,))
        for rank in range(4)
    ]
    for p in procs:
        p.start()
    for p in procs:
        p.join()

if __name__ == "__main__":
    test()

Just calling python <path-to-file.py>, I can attach to any port in {5678, 5679, 5680, 5681} and connect to the expected rank. But, it would be great if there were a way to achieve this without needing to change the source code.

Metadata

Metadata

Assignees

Labels

needs reproIssue has not been reproduced yet

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions