Resource-aware scheduling

Queue a dispatch instead of piling onto a saturated host, and watch the queue with orch status.

Goal#

Keep a busy machine from getting worse: when the host is already loaded, a new dispatch waits its turn — visible as queued — instead of starting alongside everything else and slowing every worker down together. Off by default, so a quiet machine sees no change at all.

Steps#

  1. Set one or both thresholds in orch.yaml:

    scheduler:
  max_load: 0.85          # queue while CPU busy fraction is above this (0-1); 0 = off
  min_free_mem_mb: 512    # queue while free memory is under this many MB; 0 = off
  max_wait_sec: 0         # cap on the wait; 0 = this dispatch's own timeout

    Both thresholds are off unless you set them — a workspace that never touches this section dispatches exactly as it did before item 7.

  2. Dispatch as usual. When the host is over a configured threshold at launch time, the session is recorded queued (task list --status queued, session list, orch status) and the dispatch call blocks, polling the host every couple of seconds, until the load clears or max_wait_sec elapses.

  3. Watch it live:

    orch status                # host load + scheduler queue depth for this folder
orch task list --status queued

  4. Raising scheduler.max_load in orch.yaml while a dispatch is already queued takes effect on that dispatch's next poll — no restart needed.

What happens on a timeout#

A queued dispatch that never clears within max_wait_sec fails with scheduler: waited <duration> over max_load (or under min_free_mem_mb) instead of hanging forever; the session is marked failed, not left queued.

Platform coverage#

The load sampler reads /proc on Linux and GetSystemTimes / GlobalMemoryStatusEx on Windows. Where a platform has no cheap reading (macOS today), thresholds are skipped rather than checked against a guess — a dispatch is never queued on a reading nobody could actually take.

  • Evidence bundle — a queued wait shows up in the session's own log, not in the bundle's token/cost numbers.