Skip to content

Commit e23864a

Browse files
rismayrismay
committed
Expand and clarify documentation for CommonProcess
Added detailed explanations, diagrams, and examples to the architecture, problem space, solution, and technical design documentation. These updates clarify the motivation, design constraints, module boundaries, runner selection, and usage patterns for CommonProcess, providing better guidance for users and contributors.
1 parent d321b12 commit e23864a

File tree

5 files changed

+211
-7
lines changed

5 files changed

+211
-7
lines changed

‎sources/common-process/Documentation.docc/CommonProcess-Architecture.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@
22

33
This diagram summarizes the architecture of CommonProcess and its runner stack.
44

5+
Visual diagram (SVG)
6+
7+
![CommonProcess Architecture](commonprocess-architecture.svg)
8+
9+
ASCII fallback
10+
511
```text
612
┌───────────────────────────────────────────────────────────────────────────┐
713
│ Callers (CLIs, shells, tools) │

‎sources/common-process/Documentation.docc/MasterExecutable.md‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,6 @@ command concerns (env, cwd, timeouts, streaming) in `CommandSpec`.
55

66
## `ExecutableReference`
77

8-
### `ExecutableReference`
9-
108
```swift
119
public enum ExecutableReference: Sendable, Equatable, Codable {
1210
case name(String) // PATH lookup

‎sources/common-process/Documentation.docc/TheProblemSpace.md‎

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,40 @@
11
# The Problem Space
22

3-
Explore the motivation and constraints for this package. Clarify the user
4-
story, inputs/outputs, and the boundaries of responsibility.
3+
Modern developer tools frequently spawn external processes (swift, git, npm,
4+
xcodebuild, rsync). Doing that safely and portably is harder than it looks:
5+
6+
- Portability gaps
7+
- `Foundation.Process` behaves differently across macOS, Linux, and
8+
Mac Catalyst; streaming semantics and cancellation differ.
9+
- Runners like TSCBasic and swift-subprocess have different APIs and
10+
expectations.
11+
- Safety and observability
12+
- Ad‑hoc string building encourages quoting bugs and environment leaks.
13+
- Unbounded logging can dump giant outputs into terminals and CI logs.
14+
- Without previews and tags, runs are opaque to users and metrics systems.
15+
- Determinism and policy
16+
- Tooling often needs to replay, diff, and audit “what exactly ran”.
17+
- Codebases mix shells and process wrappers, making policy enforcement
18+
(timeouts, previews, deny/allow lists) inconsistent.
19+
- Streaming and interactivity
20+
- Some scenarios require incremental consumption (tailing logs, long builds)
21+
or interactive input (prompts). Others need simple buffered results.
22+
- Testing and cross‑platform CI
23+
- A single abstraction that works on macOS and Linux simplifies tests,
24+
reduces flakiness, and avoids platform forks.
25+
26+
Non‑goals
27+
28+
- CommonProcess does not try to wrap every CLI on earth. Package‑specific and
29+
user‑friendly wrappers live one layer up in `CommonShell`/`CommonCLI`.
30+
- It does not replace shell languages. It provides a safe, typed execution
31+
core for Swift libraries and CLIs that need to call out to tools.
32+
33+
Design constraints
34+
35+
- Codable, replayable description of a run for logs and automation.
36+
- Pluggable runners with clear fallback rules; prefer `swift‑subprocess` where
37+
available, otherwise use TSCBasic or Foundation.
38+
- Bounded previews and structured logs by policy, not by call‑site convention.
39+
- First‑class streaming and interactive flows alongside buffered runs.
40+
- Minimal, dependency‑light core; keep integrations optional.

‎sources/common-process/Documentation.docc/TheSolution.md‎

Lines changed: 83 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,85 @@
11
# The Solution
22

3-
Describe the chosen design at a high level and provide the smallest working
4-
examples. Explain how to adopt the library (and optional CLI).
3+
CommonProcess provides a small, focused execution core:
4+
5+
- CommandSpec (Codable)
6+
- A single value that describes a run: identity, arguments, environment,
7+
working directory, logging policy, timeout, preferred runner, and
8+
streaming mode. It is `Codable & Sendable` so you can persist and replay
9+
runs or pass them across concurrency domains.
10+
11+
- Pluggable runners with safe defaults
12+
- `RunnerControllerFactory` selects a runner by availability and the
13+
preferred kind: `.subprocess` → `.tscbasic` → `.foundation`.
14+
- You can override selection per run while keeping a deterministic policy.
15+
16+
- Unified results and events
17+
- Buffered: `run(command:)` returns `ProcessOutput` (stdout/stderr/status,
18+
with optional pid capture for audit).
19+
- Streaming: `stream(command:)` returns an async sequence of `ProcessEvent`
20+
with a `cancel()` closure. `interactive(command:)` also returns stdin
21+
`send` and `closeInput` hooks when supported by the runner.
22+
23+
- Observability built‑in
24+
- `ProcessLogOptions` bounds previews (bytes or head/tail line caps) and
25+
prints a concise command preview to stderr when exposure is enabled.
26+
- Instrumentation hooks (`ProcessInstrumentation`) and a metrics recorder
27+
(`ProcessMetricsRecorder`) enable start/finish events with previews.
28+
29+
- Host wrappers live above execution
30+
- Execution host selection (direct, env, shell, npm, npx) is modeled outside
31+
the core in the `ExecutionHostKind` and is composed in `CommonShell`. The
32+
core stays focused on correctness and portability.
33+
34+
Smallest working examples
35+
36+
Buffered run
37+
38+
```swift
39+
import CommonProcess
40+
import CommonProcessExecutionKit
41+
42+
let out = try await RunnerControllerFactory.run(
43+
command: CommandSpec(
44+
executable: .name("echo"),
45+
args: ["hello"],
46+
logOptions: .init(exposure: .summary)
47+
)
48+
)
49+
print(out.utf8Output())
50+
```
51+
52+
Streaming run
53+
54+
```swift
55+
var cmd = CommandSpec(
56+
executable: .path("/bin/sh"),
57+
args: ["-c", "for i in 1 2 3; do echo $i; sleep 0.1; done"],
58+
streamingMode: .passthrough
59+
)
60+
let (events, cancel) = RunnerControllerFactory.stream(command: cmd)
61+
for try await e in events {
62+
if case .stdout(let d) = e { print(String(decoding: d, as: UTF8.self)) }
63+
}
64+
```
65+
66+
Interactive run (when runner supports stdin)
67+
68+
```swift
69+
let (events, send, closeInput, cancel) =
70+
RunnerControllerFactory.interactive(command: CommandSpec(
71+
executable: .name("sh"),
72+
args: ["-lc", "read a; echo $a"],
73+
streamingMode: .passthrough
74+
))
75+
Task { send(Data("y\n".utf8)); closeInput() }
76+
for try await _ in events {}
77+
```
78+
79+
Adoption
80+
81+
1) Add CommonProcess to your package (or depend transitively via CommonShell).
82+
2) Construct a `CommandSpec` where you used to build an argv array.
83+
3) Prefer `run(command:)` for simple cases and `stream(command:)`/`interactive` for
84+
long‑running or chatty tools.
85+
4) Use `ProcessLogOptions` for previews and set a reasonable timeout.

‎sources/common-process/Documentation.docc/TheTechnicalDesign.md‎

Lines changed: 84 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,86 @@
11
# The Technical Design
22

3-
Explain the system design and module boundaries.
3+
Module boundaries
4+
5+
- Target `CommonProcess` (core)
6+
- Data model and policies: `CommandSpec`, `Executable`/`ExecutableReference`,
7+
`EnvironmentModel`, `ProcessLogOptions`, `ProcessOutput`,
8+
`ProcessInstrumentation`, `ProcessMetricsRecorder`, `ExecutionHostKind`,
9+
and error types.
10+
- No direct process spawning. No platform conditionals in public API.
11+
12+
- Target `CommonProcessExecutionKit` (runners)
13+
- Factories, controllers, and concrete backends.
14+
- Depends on Subprocess/TSCBasic/Foundation conditionally.
15+
16+
Runner selection and execution
17+
18+
- `RunnerControllerFactory`
19+
- `defaultKind()` evaluates availability at compile‑time.
20+
- `chooseBest(preferred:)` returns a supported kind with fallback.
21+
- `run(command:)` selects the runner, delegates to a controller, and returns
22+
a buffered `ProcessOutput`.
23+
- `stream(command:)` prefers streaming‑capable controllers and falls back to
24+
an error if none support streaming.
25+
- `interactive(command:)` adds stdin `send`/`close` when the runner supports
26+
interactive I/O; otherwise returns the streaming path only.
27+
28+
- `RunnerFactory` (per backend)
29+
- `FoundationRunnerFactory`, `SubprocessRunnerFactory`, `TSCBasicRunnerFactory`.
30+
- Each factory returns a backend‑neutral `RunnerExecutor` with a concrete
31+
lifecycle delegate.
32+
33+
Executor and delegates
34+
35+
- `RunnerExecutor<Delegate>` coordinates the run in three stages:
36+
1) Instrumentation – set up metrics and preview policy; print a concise
37+
`$ <cmd>` line when exposure is enabled.
38+
2) Environment – materialize merged environment from `EnvironmentModel`.
39+
3) Resolution – compute the concrete executable path + argv, then hand off to
40+
the delegate.
41+
- Delegates implement `launch`, `awaitResult`, and optional streaming hooks.
42+
- Streaming path tees output through a bounded preview capture so metrics can
43+
emit head/tail or byte previews without retaining the full output.
44+
45+
Executable resolution
46+
47+
- `Executable` + `ExecutableReference`
48+
- Identity is explicit and separate from call‑site args.
49+
- Resolution favors PATH lookup for `.name`, uses the provided path for
50+
`.path`, and for `.none` interprets the first `args` element as the token.
51+
52+
Policy and observability
53+
54+
- `ProcessLogOptions` controls:
55+
- Exposure (none/summary/verbose) and a short `$ …` preview line.
56+
- Head/tail line caps and byte caps for stdout/stderr previews.
57+
- Optional pid capture for audit when supported by the backend.
58+
- `ProcessInstrumentation` and `ProcessMetricsRecorder`
59+
- Start/finish callbacks include runner name, duration, and bounded previews.
60+
61+
Streaming and interactive semantics
62+
63+
- Buffered path returns a `ProcessOutput` with all data.
64+
- Streaming path returns `AsyncThrowingStream<ProcessEvent, Error>` and a
65+
cancel closure. Runners: Subprocess, Foundation, TSCBasic.
66+
- Interactive path (when supported by the runner) returns stdin `send` and
67+
`closeInput` closures in addition to the stream and cancel closure.
68+
69+
Thread‑safety and concurrency
70+
71+
- Core structs are `Sendable`. Executors and delegates carefully isolate state
72+
with `NSLock` or private dispatch queues when needed. Public APIs are async.
73+
74+
Extensibility
75+
76+
- Adding a runner:
77+
- Implement a lifecycle delegate and a factory that returns a
78+
`RunnerExecutor` parameterized by your delegate.
79+
- Provide streaming and interactive paths if available; otherwise rely on the
80+
buffered path.
81+
- Update `RunnerControllerFactory.supportedKinds()` and fallback logic.
82+
83+
Non‑goals revisited
84+
85+
- No shell DSL and no CLI wrappers in the core. Typed wrappers and adapters
86+
belong in `CommonShell`/`CommonCLI` to keep a clean separation of concerns.

0 commit comments

Comments
 (0)