Fortress Desk · living spec · source _md_files/desk/DESK_REALTIME_SPEC.md

Fortress Desk real-time: spec, decisions and where we are

The ONE place for the Desk real-time work: the goal, how the real-time path works, every owner decision (date, why, commit), what was deliberately not done, the open items, how to measure, and the rules for changing anything. Living doc: update it IN PLACE when a real-time decision is made or a number moves. Frozen thread dumps: history/2026-10-09_DESK_REALTIME_PASS_CONTEXT.md; new-thread brief (decisions + TODO): history/2026-10-13_DESK_REALTIME_NEW_THREAD.md. CF copy: /reference/DeskRealtimeSpec (re-render with 2026-10-13_realtime_bench/render_spec.py, then deploy_cf.sh). Per-tool mechanics live in _md_files/SCRIPT_CATALOG_DETAILS.md (the *_live.py sections); this file is the decisions and the map.

1. Goal

Owner, 2026-10-09: make the Desk app "as real-time as possible, just like TWS". The app is the surface the owner trades from (native-first; Classic Chrome pages are the fallback, CLAUDE.md "Desk: native-first").

2. Where we are (measured)

App = how old the data on the screen is, sampled every second. Chrome = the same tab's Classic page in headless Chrome, same window.

Tab	App tab data, p50 / p90	App spot + held-call ticks, p50 / p90	Chrome Classic, p50 / p90	Measured
Rebuild	1.2 / 3.4 s	0.5 / 0.9 s	49 / 75 s	2026-10-13 01:02-01:17 (app), 00:04 (Chrome)
FIGHT	1.2 / 3.5 s	0.7 / 0.8 s	41 / 73 s	2026-10-12 23:51-00:01
Scanner	1.1 / 2.0 s	0.5 / 0.9 s	10 / 23 s	2026-10-13 00:04
Coverage	1.2 / 3.3 s	not ticked	16 / 28 s	2026-10-13 00:04
Income	1.6 / 5.8 s	not ticked	7.9 / 13 s	2026-10-13 00:04

Watcher write to app delivery (hub SSE): 0.01-0.02 s p50.
A hidden Chrome tab freezes (about 10 min old after 10 min hidden); the app keeps updating.
The Desk index publish gap p90: Rebuild 2.5 s, FIGHT 5.0 s, Scanner 2.0 s, Coverage / Income 4.7 s.
Refused market-data lines (err 101): 0 in every RTH sample since the counter exists (2026-10-09).
Owner-facing report with both Chrome-vs-app readings: CF /reference/DeskRealtime_Oct09.

3. How the real-time path works (one paragraph per piece)

Desk state every cycle. Each watcher publishes its native tab's view model (state/index.json, module_desk_*) every loop cycle, not on the Classic render cadence; the hub (fortress_hub.py, :8930) pushes it to the app over SSE within ~0.02 s.
2 s loops. All five watchers run a 2 s live cycle (Rebuild, FIGHT, Scanner since 2026-10-09; Coverage / Income since 2026-10-10).
Publish first. Slow IBKR work (verify fetches, sweeps, Monte Carlo) runs AFTER the cycle's Desk publish, so a fetch delays the NEXT publish by at most max(interval, fetch), never interval + fetch.
1-second ticks. module_desk_ticks: spot, held-call mark and P&L leaf cells refresh every second from the already-streamed quotes (Rebuild, FIGHT, Scanner), on a timer that also fires while a watcher waits on IBKR. Display only; decisions keep the cycle's values.
LIVE option streams. Persistent streams on held calls, the Desk-followed position's band and the picks (Rebuild 120, FIGHT 80, Scanner 80 lines).
Split-phase windows. Verify pieces, fallbacks and the stale sweep are quote windows (subscribe now, collect next cycle) instead of blocking fetches: Rebuild and Scanner.
Render worker. The Classic HTML renders in a spawned child process so the loop never stands still for it: Rebuild (2026-10-09), FIGHT (2026-10-12).
Spot sharing. module_spot_seam: one market-data line per ticker for all five watchers (130 -> 42 lines); spot.quotes (module_spot_quote) is the one spot every native tab shows.
Quote seam. Rebuild publishes its streamed option quotes (module_quote_seam); Coverage / Income fold them (marks seconds old instead of up to 5 min) and skip a ticker's timed re-check when the seam covered it whole.
Focus. runs/hub/focus.json (written by the hub) names the positions / labels a Desk window follows: Rebuild streams the followed position's band; FIGHT builds Monte-Carlo detail panels only for a followed label.

4. Market-data lines (the budget behind every real-time choice)

Account cap 578 simultaneous lines (TWS dialog); TWS's own windows hold ~72; 50 kept free for order tickets. module_ibkr_fetch.MD_LINE_ALLOTMENT / MD_LINE_TWS_OWN / MD_LINE_BUDGET / MD_LINE_CAPS; line_budget_report().

Holder	Lines (caps today)
Spots, shared by all five watchers	~42
Rebuild: streams + windows + sweep	250 = 120 + 80 + 50
Scanner: streams + windows	130 = 80 + 50
FIGHT streams	80 cap (43-57 used)
Coverage / Income snapshot bursts	small
Order tickets: hot streams on the warm previewer (`150178f`, off switch)	owner line-budget test
TWS's own windows	~72

Peak seen: 472 watcher lines + ~72 TWS = 544 of 578 (2026-10-12 22:08-23:00). Every watcher publishes md_err_101 (refusals) and line counts in live.json stats.

5. Owner decisions (newest first)

Date (SGT)	Decision	Why	Commit(s)
10-13 01:2x	LEAVE Rebuild's remaining blocking fallback as is (~9 s/min: sweep DELAYED dead-contract verdicts + verify FROZEN/DELAYED leftovers; ~3 pauses of 10-11 s per 15 min)	ticks keep flowing, decisions unaffected, the fix (windows after a tidy step) costs hours for ~5 s off the worst pause; revisit only if the owner says the tab visibly freezes	none
10-13 01:01	No blocking SNAP tier on the LIVE feed in RTH	SNAP = 15+ min-old snapshot of strikes with no live market: 21 of 33 fallback calls, 27% quoted, 235 s blocked per 16 min	`98dbc8f`
10-13 00:34	On-demand grids (roll chain, cc / hatch re-plan) ask LIVE + FROZEN only in RTH	DELAYED / SNAP made 14-25 s blocks (COPX, AMZN)	`10bf96f`
10-13 00:04	Second Chrome-vs-app reading	owner asked for one more	reading in `2026-10-13_realtime_bench/readings/`
10-12 23:5x	FIGHT: display-only Monte Carlo off the loop (rung note without the ladder table; detail panels only for a Desk-followed label)	owner: "the main card is what I trust", the MC detail is rarely opened	`037bb07`
10-12 23:30	FIGHT Classic pages in a render worker; orphan-chain yardstick `'pos'` fix	render 1.9 s p50 on the loop; tick stalls 7.4 s	`4e28657`
10-12 22:20	Open checks for Monday (launch fix proof, err 101, Coverage/Income loop, held legs, Rebuild)	first session after the weekend ships	all passed; see section 7
10-10 11:21	Coverage / Income: 2 s loop + publish first (no 1 s ticks for them)	tab age 3.3 s p50 vs ~1 s elsewhere; ticks would add little once the loop is 2 s	`8598ea2`
10-10 04:07	Launch fix: Homebrew python pinned, watchers at PRI 31 from any caller (launchd Interactive), Launchpad KILL stops them, FortressList Interactive	cron / Launchpad starts ran at PRI 20 (renders ~2x slower), Xcode 3.9 once	`f889003`
10-10 01:58	Sweep backoff for unquotable rows (`_sweep_tried`)	the faster sweep re-asked dead rows every 15 s	`14e6398`
10-10 01:13	Rebuild fallback item waits up to 45 s for window lines (owner "go ahead with change 2")	fewer demotes to blocking	`fabb705`
10-10 00:2x	Rebuild sweep gets its OWN 50 lines (Rebuild 200 -> 250)	sweep waited for lines; owner: "give the sweep 50 more lines"	`52fbb9e`, `fb9baf9`, `bfd0b40`
10-10 00:53	Scanner split-phase windows, 80 streams + 50 window lines (owner "A: 80 + 50", another thread)	Scanner p90 tail = blocking verify	`87ab8b9`
10-09 23:0x	Held legs: option (a) skip covered re-checks (not (b) focus-band slice, not (c) +88 lines)	no lines; smaller win	`4659166`
10-09 23:0x	Speed-ups, ship during RTH: Scanner MC / verify after publish; Rebuild fallback as windows; sweep at full 50	no budget change	`8209bb2`, `1b3ac12`, `c2d646f`, `1dcf5a2`
10-09 23:0x	FIGHT verify budget: NO change (5 s p90 is fine)	owner	none
10-09 22:3x	Fix FIGHT's per-cycle Monte Carlo regression (from `6d5d6bc`) and ship	Desk publish 1.6 s per cycle	`77469e6`
10-09 22:1x	Rebuild "120 streams + 80 windows" (inside 200)	windows replace blocking verify bursts	`1ec01d6` and follow-ups
10-09 10:2x	MD_LINE_ALLOTMENT 578 + TWS-own term; spot de-dup	the real cap; ~100 lines freed with no cut	`e1b5ad3`
10-09	"Keep full limits. Do NOT trim until you confirm with me." On err 101: report numbers + escalation order, wait for the OK	owner rule (memory `feedback_line_caps_confirm_before_trim`)	`9c37802`
10-09	No "wait N min after a TWS restart" anywhere; a wedged TWS needs a restart	owner rule	`1c5be86`, `b18668b`, `78c2125`
10-09 01:20	Restarts during RTH approved for this work	owner	none

Standing owner preferences that shape this work: live pick flipping is fine (picks are read as real time); the Rebuild routing verdict (ROLL CC / SKIP / WAIT) is not used; light mode only; center-aligned tables.

6. Deliberately not done (and when to revisit)

Item	Why not	Revisit when
Rebuild fallback calls as windows	owner 2026-10-13: leave it	owner reports the Rebuild tab freezing
FIGHT split-phase windows	needs window lines; peak 544 of 578	owner frees or raises lines
Coverage / Income 1-second ticks	2 s loop already ~1.2-1.6 s; spot is live via spot.quotes	their p90 matters to the owner
Held legs (b) / (c)	(a) chosen	(a)'s skips prove too few (cc_check had 27 skips / 10 min on 10-12)
Lower the Classic pages' reload throttle	the app does not need it; Classic is the fallback	owner uses Chrome again
Any cap / budget cut	owner rule: confirm first	err 101 in RTH, with numbers

7. Open items / watch

Rebuild verify coverage: 98% on 10-10, 110% on 10-12 (fine; keep an eye).
IBKR code 354 ("market data not subscribed"): 123 on Rebuild 10-09 after 23:00, 0 on 10-12.
rh_gate: the 3 replay cases fail since f23adde (pre-existing); need a fresh evening recording (touch ~/fortress/harness/rebuild/RECORD_NEXT_EVENING).
scripts/com.abhi.fortress.fortresslist.plist is untracked (a Desk session's work); it must keep ProcessType=Interactive when committed.
Order-ticket hot streams (150178f) add lines while tickets are warm: count them in any line-budget reading.

8. Rules for changing real-time code

Measure first, in RTH, 10-15 min, read-only (section 9). Compare against section 2's numbers, never against memory.
Never block the loop on a fetch you can window, and never put display-only work (Monte Carlo detail, HTML) on the per-cycle path.
Line budget: a new stream / window / burst goes through MD_LINE_BUDGET / MD_LINE_CAPS; any cap change needs the owner's OK; on err 101 report and ask, never trim.
Rebuild: tidy-as-you-go (CLAUDE.md) plus rh_gate check; re-capture only the cases whose differences you confirmed.
Restarts: a PLAIN restart that resumes from the watcher's own dump (launch_live.sh --only <tool> --action restart); never delete the dump unless fetch planning changed AND a batch bundle under 30 min exists (2026-10-09: Rebuild dark 13 min). Check no order ticket is WORKING first.
Ship Python changes to a running watcher outside RTH unless the owner says ship now.
Commit only your own hunks; other sessions edit the same tree (stage HEAD + your hunk via git hash-object -w / git update-index --cacheinfo).
Record the decision here (section 5) and the mechanics in SCRIPT_CATALOG_DETAILS.

9. How to measure (tools in `2026-10-13_realtime_bench/`)

Tool	What it measures	Run
`chrome_classic.mjs` + `freshness.py`	Chrome vs app on-screen age, same window, 10 min all visible + 10 min one visible	`node chrome_classic.mjs OUT.json 600 SCRATCH_DIR rebuild` then `python3 freshness.py OUT.json`
`beat_sampler.py` + `beat_report.py`	a watcher's Desk beat (live.json stats + index writes, 1 s)	`python3 beat_sampler.py OUT.jsonl MINUTES [runs_subdir]`
`fight_sample.py` / `rb_sample.py`	FIGHT / Rebuild Desk gap, data age, tick age, phase timings	`python3 fight_sample.py SECONDS OUT.json`
`open_checks.py`	open-of-session checks: launch PRI, err 101, line totals, publish gaps	`python3 open_checks.py OUT_DIR` (edit the 21:50 / 22:20 / 23:00 gates)

All read-only: GET only, never :8924, no IBKR connection. Chrome uses a scratch profile.

Rendered from the repo spec (last edit 2026-10-13 01:24 SGT). Edit the .md, then re-run render_spec.py and deploy_cf.sh.