Strict Blog
Load Balancing 8 min read · Updated November 2026

Shaping GitHub Actions Fleets Without a Proxy: Inside Strict’s Auto Load-Balancer

GitHub Actions assigns queued jobs directly to whichever matching self-hosted runner polls first—with zero awareness of a machine’s free disk, memory pressure, or CPU saturation.

Instead of rewriting workflows to route webhooks through a custom job broker, Strict steers placement by dynamically pausing and re-admitting idle runner slots on each host. Here is the exact algorithm running inside computeFleetDirective() and reconcile_drain().

1. Why supply-side control beats proxying webhooks

A job proxy sits in the critical path of every workflow dispatch: if the webhook consumer stalls or rate-limits, CI queues hang. By letting hosts run standard persistent actions-runner listeners and shaping available runner capacity cleanly via launchctl bootout/bootstrap (macOS) and systemctl --user stop/start (Linux):

  • Workflow YAML needs zero changes—GitHub matches runs-on labels natively against whichever slots are awake.
  • When every node is healthy, Strict adds zero scheduling latency.
  • If the Strict server reboots or becomes unreachable, hosts automatically recover their own runners after 5 minutes.
Strict 30-second auto load-balancing control loop 1. Host Telemetry Disk, RAM, load1, repos Every 30s 2. computeFleetDirective basePressure + load-skew poolKey & headroom check Returns drainRunners[] 3. reconcile_drain Skips busy workers bootout / stop + drained.json 4. Spillover GitHub picks healthy peers Auto re-admit once recovered (or after > 300s offline via recover_offline_drain)

2. Interactive Load-Balancer Simulator

Experiment with the exact TypeScript rules from computeFleetDirective(). Click a scenario preset or drag any host’s sliders to observe hysteresis, repo-pool safety guards, proportional load-skew shedding, and deterministic tie-breaking live.

Live Fleet Directive Evaluator

lowGiB=20 · targetGiB=40 · 2 slots/host
Computed HostDirective per node

3. Four balance signals with enter/exit hysteresis

Only online hosts (seen within FRESH_MS = 105,000 ms) participate in fleet evaluation. To keep runners from flapping back and forth near a threshold, every signal in basePressure() and computeFleetDirective() uses separate enter and exit bands:

Reason Enters pressure when Stays latched until cleared at Scope drained
disk diskFree < lowGiB diskFree >= targetGiB All covered slots + runCleanup
memory memUsedPct >= 85% memUsedPct <= 75% All covered slots
cpu load1 / cpuCores >= 1.4 load1 / cpuCores <= 0.9 All covered slots
load-skew busyCount >= 2 & idle healthy peer has >= 8 GiB more free RAM busyCount === 0 or peer free RAM gap < 4 GiB Half of covered slots (floor(n/2))

4. Pool matching, headroom checks & anti-thrash tie-breaking

Draining the only Mac capable of building acme/release just because its disk is filling up would strand jobs indefinitely in GitHub’s queue. Strict refuses to drain unless peerCoversPool() confirms another online peer covers that slot cleanly:

  • Repo-pool identity (poolKey): Runners are grouped by their configured repository (runnerRepos[runnerName]), falling back to stripping any <deviceId>- host prefix. Peers only count as coverage if they hold an idle or busy non-draining runner in the same pool.
  • Never cascade onto overloaded peers (peerHasHigherHeadroom): Strict never routes spillover onto a peer experiencing memory or cpu pressure. On disk-only pressure, a disk-pressured peer is only eligible if it has strictly more free disk. For all active reasons, the target peer must have strictly better headroom on every active pressure axis.
  • Deterministic symmetry breaking: If two hosts in the same pool hit the exact same free disk and free memory simultaneously, comparing peer.id < device.id ensures only the lexically higher ID drains—the lower ID stays online to absorb the pool so both machines never mutually take themselves offline.

5. Proportional shedding & agent-side safety guarantees

Once directives reach the host agent, three local invariants keep running jobs and fleet capacity safe:

  • Proportional shedding under load-skew: When a host has multiple slots busy and a roomier idle peer is waiting, Strict wants to balance concurrency—not empty the busy host. Under load-skew alone, it sheds only Math.floor(coveredCandidates.length / 2) slots, choosing candidates via drainCandidatePriority(): already-draining (0), then idle (1), then starting (2), then busy (3).
  • In-flight jobs are never aborted: Before executing launchctl bootout or systemctl --user stop, reconcile_drain() runs is_runner_active() to inspect live Runner.Worker processes. Active slots finish their current GitHub job untouched and are drained only once they go idle.
  • drained.json stops self_heal() interference: Intentionally drained runners are recorded in drained.json so the maintenance timer’s crash-recovery loop doesn’t immediately restart a runner the load balancer just paused.
  • 5-minute offline recovery fail-safe: If a host loses connectivity to the Strict control plane for more than 300 seconds (measured against last-sync-ok.json or drain timestamps), recover_offline_drain() clears drains and brings runners back online automatically—provided free disk is at least RESTART_SPACE (5 GiB).