chore(task): archive 09-12-mind-send-liveness

This commit is contained in:
YBF
2026-09-12 15:35:15 +08:00
parent ec28fd746b
commit 1aa33f035f
6 changed files with 294 additions and 0 deletions
@@ -0,0 +1,4 @@
{"file":".trellis/spec/project/architecture.md","reason":"Review ownership, dependency direction, and narrow module responsibilities."}
{"file":".trellis/spec/project/async-state-boundaries.md","reason":"Review timer cleanup, terminal states, and stale callback handling."}
{"file":".trellis/spec/server/backend/quality-guidelines.md","reason":"Apply Server static and compilation validation guidance without adding tests."}
{"file":".trellis/spec/mind-test-harness/development/boundary.md","reason":"Verify the change remains a development-only manual tool without test/build orchestration changes."}
@@ -0,0 +1,104 @@
# Design: Harden Mind send liveness
## Objective
Maintain one authoritative Server-side answer to “is a plugin currently
routable?”, while ensuring the local Harness page never treats the absence of a
response as continued health or an indefinitely pending operation.
## Server: the in-memory lease is the authority
`createOneTalkConnectionStore` remains the sole owner of connection membership,
generation, heartbeat timestamps, and lease duration. It will expose one narrow
freshness query for an already registered connection. The query is true only
when the supplied connection is still canonical and its `lastHeartbeatAtMs` is
strictly newer than `now() - heartbeatTimeoutMs`.
`requestSend` will use that query twice:
1. while selecting the unique candidate, before reserving a send; and
2. after its asynchronous authorization checks, immediately before dispatching
`send.command`.
This prevents a lease that expires during authorization from being dispatched.
The periodic sweep remains responsible for eventual removal, close, and status
notification; it no longer creates a window in which expired membership can be
admitted for a send.
The existing canonical pre-dispatch result remains
`rejected_before_send / waiting_for_page`. No new public result code is needed:
the contract already expresses that no exact routable page is available, and
introducing `plugin_offline` would widen the shared protocol for no behavioral
gain.
No server-initiated heartbeat is added before each send. A successful preflight
ACK would only prove a point in time before dispatch, adds latency and a second
timeout state, and does not repair a broken Server-to-Mind return path.
## Harness page: local liveness and terminal states
The Harness page does not become a second plugin-presence authority. Its
`plugin.status` remains a last-observed display/precheck. It adds a local
Mind-to-Server liveness guard:
- one heartbeat may be awaiting an ACK at a time;
- starting that heartbeat starts a fixed 75-second deadline;
- a matching ACK clears the request record and deadline;
- an expired deadline invalidates the current socket, marks the connection
unavailable, disables all send controls, and directs the operator to the
existing manual recovery/reconnect flow.
If a send is pending when local liveness ends, the page clears only its local
pending marker and reports an unconfirmed/unknown outcome. It must not emit or
pretend to have received a `send.result`, mark it confirmed, or resend it.
The same local settlement is used for an explicit socket close while a send is
pending. Late server frames are ignored because the local pending ID has ended
or the socket is no longer current.
The existing state object remains the only Harness-page state owner. Timers and
their request identity are added there and are cleared by the existing socket
cleanup path. A focused helper local to `harness/websocket.ts` owns the
"pending send became locally unknown" transition, avoiding duplicate UI updates
across ACK timeout and close handling.
## Harness HTTP deadlines
The Harness page adds a shared, page-local `fetch` wrapper in the runtime
script. Each invocation owns an `AbortController` and 30-second timer; aborts
become a stable `request_timeout` error for the existing error-text mapping.
Both read flows and the upload flow use it. Their existing `catch/finally`
paths therefore render the visible error and release upload controls. This does
not add retries, change server routes, or add a global loading coordinator.
## State transitions
```text
Server: registered plugin + fresh lease
└─ send request → recheck fresh lease → authorize → recheck fresh lease
├─ invalid → rejected_before_send / waiting_for_page
└─ valid → send.command → existing confirmation / timeout flow
Harness: accepted socket
└─ heartbeat pending → matching ACK → continue
└─ 75s deadline → local connection unavailable
├─ no pending send → disable controls; manual reconnect
└─ pending send → local unknown; disable controls; manual reconnect
Harness: HTTP request
└─ response → existing parse/result path
└─ 30s → abort → request_timeout → existing visible error/finally path
```
## Compatibility and risk
- Server behavior becomes stricter only for expired leases; valid routes retain
existing send, authorization, confirmation, and timeout semantics.
- The 75-second Harness deadline intentionally matches the current default
lease, but remains page-local. Changing Server configuration later does not
silently change the manual page; this is accepted to keep the dev tool simple.
- A local unknown outcome may coexist with a later real-world send. This is the
required safe representation: no automatic resend or synthetic terminal
Server frame is permitted.
- This task deliberately does not add tests or a fault-injection harness per
user decision and package boundary. Compiler/static validation cannot prove a
real half-open network path and will be reported as such.
@@ -0,0 +1,4 @@
{"file":".trellis/spec/project/architecture.md","reason":"Shared ownership and async-state boundary rules for the Server lease and Harness page changes."}
{"file":".trellis/spec/project/async-state-boundaries.md","reason":"Required lifecycle guidance for timers, socket invalidation, pending requests, and late acknowledgements."}
{"file":".trellis/spec/server/backend/index.md","reason":"Server package boundary and WebSocket implementation baseline."}
{"file":".trellis/spec/mind-test-harness/development/boundary.md","reason":"Development-only Harness scope and explicit prohibition on new test infrastructure."}
@@ -0,0 +1,49 @@
# Implementation plan: Harden Mind send liveness
## Scope owners
- `apps/server/src/websocket/connection-store.ts`: canonical lease freshness
query and existing presence query semantics.
- `apps/server/src/websocket/pending-send-coordinator.ts`: use lease freshness
before reservation and immediately before `send.command`.
- `apps/mind-test-harness/src/harness/runtime.ts`: fixed page-local timeout
constants and timer state.
- `apps/mind-test-harness/src/harness/websocket.ts`: single outstanding
heartbeat, ACK deadline, local unknown send settlement, and cleanup.
- `apps/mind-test-harness/src/harness/reading.ts`, `upload.ts`, `errors.ts`:
shared deadline-aware fetch and visible timeout text.
## Ordered work
1. Run GitNexus impact analysis for every Server/Harness symbol selected for
editing; report any high/critical finding before proceeding.
2. Extend the connection-store public interface with a canonical, time-aware
freshness query. Keep lease math and membership identity in the store.
3. Apply the query at both Server send admission fences. Preserve all existing
result shapes and the current pending-send coordinator ownership.
4. Add the 75-second heartbeat ACK deadline and 30-second HTTP deadline to the
Harness page runtime. Do not add configuration plumbing.
5. Implement single-flight heartbeat acknowledgement tracking. On deadline or
socket close, settle any local pending send as unconfirmed/unknown, invalidate
the socket, disable sending, and retain manual recovery/reconnect.
6. Route reads and uploads through a 30-second `AbortController` wrapper and
map the abort to a stable visible timeout error.
7. Review the complete diff for duplicated state ownership, timeout/retry
behavior, synthetic success, and any accidental protocol expansion.
## Validation and review
- No test files or test commands are added, per user decision and Harness
package boundary.
- Run targeted TypeScript compilation for the edited Server and Harness
packages if local dependencies permit; run formatting and `git diff --check`.
- Inspect the final source paths to verify the Server checks the same owned
lease at both send fences, and the Harness clears every timer on socket
invalidation.
- Record that static/compile checks do not simulate a half-open network.
## Rollback
- The change is limited to in-memory lease admission and the development-only
Harness page. Reverting this task restores previous admission/indefinite-wait
behavior without a schema, migration, or protocol migration.
@@ -0,0 +1,107 @@
# Harden Mind send liveness
## Goal
Make Server send admission fail closed on an expired plugin lease, and make the
local Harness page end all waiting states visibly. At the instant Server receives
a Mind `send.request`, it must use its own in-memory connection lease rather
than a stale `WebSocket.OPEN` entry to decide whether a plugin is currently
routable. When the Harness page cannot prove that its WebSocket or HTTP request
is still live, it must not continue to present a healthy or indefinitely pending
state.
## Terminology
- **Harness page**: the local browser-facing manual integration page served by
`apps/mind-test-harness` (normally port 8788). It represents the Mind page
for this task and owns the WebSocket/send/read/upload UI.
- **Mind authorization mock**: the separate local service in the same package
(normally port 8787). It is not part of this liveness work.
## Confirmed Facts
- The Server owns plugin presence in process memory, not in a database or Redis:
`connections` retains `lastHeartbeatAtMs`; `activePlugins` maps the full Mind
scope to its unique plugin socket. A new plugin replaces the old one.
- Server heartbeat processing refreshes the timestamp, and a 25-second sweep
closes entries older than the configured 75-second default lease
(`apps/server/src/websocket/connection-store.ts:118-120,204-267`).
- `requestSend` currently selects a candidate from live connections by scope,
binding, permission, and `WebSocket.OPEN`, but does not synchronously require
that the recorded lease remains unexpired
(`apps/server/src/websocket/pending-send-coordinator.ts:139-154`).
- The Harness page enables sending from its last received `plugin.status=online`
and `WebSocket.OPEN`. It records heartbeat IDs, but a missing correlated ACK
has no deadline or state transition (`apps/mind-test-harness/src/harness/websocket.ts:63-87,137-163,227-235`).
- The Harness clears its pending send only after a matching `send.result` or
`onclose`; its `onerror` only changes text. A half-open Server-to-Mind path
can therefore leave the page permanently in "sending"
(`apps/mind-test-harness/src/harness/websocket.ts:250-265`).
- Harness HTTP reads and OSS uploads call `fetch` without cancellation or a
deadline, so their UI can remain loading/uploading forever
(`apps/mind-test-harness/src/harness/reading.ts:50-63`,
`apps/mind-test-harness/src/harness/upload.ts:21-63`).
## Requirements
1. Treat the Server's in-memory plugin lease as the sole send-admission fact.
At `send.request` time, a candidate must have the correct identity,
permission, live socket, and a non-expired lease; otherwise do not dispatch
`send.command`.
2. Preserve fail-closed send semantics. An unavailable plugin produces an
explicit pre-dispatch rejection; a dispatched send without a confirmed
terminal result remains `delivery_unknown`, with no automatic resend.
3. Do not use per-send Server-initiated plugin heartbeats as the normal
admission mechanism. They add a round trip without eliminating the race
after acknowledgement; continuous plugin lease renewal plus an exact
admission check is the chosen model.
4. Keep `plugin.status` as a connection snapshot and online/offline transition,
not a periodic source of truth for the Mind page.
5. Keep the same semantic rule in the Harness page: every locally initiated
wait must reach a visible terminal state. A missing heartbeat ACK disables
sending and locally reports any pending send as unconfirmed/unknown; it does
not invent a server terminal result or retry.
6. Bound Harness HTTP reads and uploads with cancellation. A timeout must be
shown as an explicit failure and release the relevant UI controls.
7. Keep implementation cost intentionally narrow: use the existing modules,
status widgets, manual recovery path, and current single-process Server
memory store. Do not add Harness test files or a browser/network
fault-injection framework.
## Acceptance Criteria
- [ ] A plugin whose `lastHeartbeatAtMs` is past the configured lease deadline
is rejected synchronously at send admission even if the sweep has not yet
run; no `send.command` reaches that plugin.
- [ ] Existing non-expired lease dispatch and confirmation behavior, including
text/media timeout semantics, is retained.
- [ ] A Harness heartbeat ACK deadline disables all send controls, changes the
WebSocket UI from healthy to failed/unavailable, and converts any local
pending send from "sending" to an explicitly unconfirmed/unknown outcome.
- [ ] A later `send.result` is ignored once the local send has ended; no local
timeout is represented as `confirmed_sent` or retried automatically.
- [ ] Timed-out Harness reads and uploads show an explicit error and restore
their controls rather than remaining loading/uploading.
- [ ] No new test files, package scripts, root quality-gate changes, or
browser-fault injection are added. Validation is limited to proportionate
static/compilation checks and manual code-path review.
## Out of Scope
- Redis/database-backed presence, multi-instance WebSocket routing, and new
cross-instance coordination.
- Periodic `plugin.status` broadcasts or a per-send plugin preflight heartbeat
protocol.
- Changes to plugin identity, authorization policy, message persistence,
delivery acknowledgement, or automatic send retries.
- New automated Harness tests, a browser/network fault-injection framework, or
changes to root test/build/typecheck orchestration.
## Decisions
- The Harness page uses a fixed 75-second missing-heartbeat-ACK deadline,
aligned with the current Server lease default.
- Harness HTTP reads and uploads use a fixed 30-second deadline.
- The user explicitly accepts no new automated tests or fault-injection work;
the task remains a narrow implementation with proportionate compile/static
validation only.
@@ -0,0 +1,26 @@
{
"id": "mind-send-liveness",
"name": "mind-send-liveness",
"title": "Harden Mind send liveness",
"description": "Fail closed on expired plugin leases and converge the Mind harness after lost acknowledgements",
"status": "completed",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "ybf",
"assignee": "ybf",
"createdAt": "2026-09-12",
"completedAt": "2026-09-12",
"branch": "09-12-mind-send-liveness",
"base_branch": "main",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": null,
"relatedFiles": [],
"notes": "",
"meta": {}
}