mirror of
https://github.com/sinanyuntu/trade-message-center.git
synced 2026-09-17 13:22:11 +08:00
feat: index OneTalk sync ledger queries
This commit is contained in:
@@ -84,7 +84,7 @@ OneTalkContactProfileStore.discardPendingProfile(input: {
|
||||
|
||||
### Durable lifecycle
|
||||
|
||||
- `ONE_TALK_SYNC_DATABASE_VERSION` 为 `8`。独立的 `onetalk_contact_profiles` store 以账号/会话对为键;消息、candidate、checkpoint、anomaly 各 store 保持独立。
|
||||
- `ONE_TALK_SYNC_DATABASE_VERSION` 为 `9`。独立的 `onetalk_contact_profiles` store 以账号/会话对为键;消息、candidate、checkpoint、anomaly 各 store 保持独立。v8→v9 只建立查询 index 并保留 ledger record;`hasProfileRecord(account)` 走账号 index count,pending flush 以 `[channelAccountId, pending.observedAtMs]` 的合法账号前缀范围读取。无 `pending.observedAtMs` 的 uploaded/rejected record 不得出现在该 index 结果中,也不得回退到 object-store 全表扫描。
|
||||
- durable profile 指纹是上传去重的唯一闸门。每收到一条 profile,先在 `putPendingProfile()` 之前读取它的 `[channelAccountId, conversationId]` 记录。若 `pending.fingerprint` 或 `lastUploadedFingerprint` 与来入指纹相同,读后即结束:不更新时间戳/高水位、不写 IndexedDB、不发 `profile_observed` 诊断、不 flush、不发送。指纹不同时走既有的 durable-first pending 写入。
|
||||
- 同键观察对 `getProfile → putPendingProfile` 转换做串行化。并发的相同指纹可以各自执行读取,但最多一个能写入、诊断或 flush。读取失败对调用方保持可见;tracker 清理不得制造第二个未处理 rejection。durable 读进行期间协调器被 dispose 时,其完成不产生写入、诊断或 flush。
|
||||
- 只有 readwrite transaction 完成且当前 pending 的 fingerprint 与 `observedAtMs` 仍然匹配时,ACK 才标记 uploaded。更旧、未知、重复或不匹配的 ACK 一律 no-op。future-skew 错误只移除精确匹配的 pending snapshot,并记录其被拒观察水位。
|
||||
@@ -123,7 +123,7 @@ OneTalkContactProfileStore.discardPendingProfile(input: {
|
||||
|
||||
- Contract:精确 profile/frame 字段、协议版本、方向/scope、direct discovery 类型、敏感/未知字段拒绝、空/超限批次和 `256 KiB` 字节上限。
|
||||
- Observer:initial snapshot、可重复的 `syncData` 发布、精确 snapshot/collect payload 校验、targeted direct 查找、群聊/未知目标排除、登录身份、登出/切账号、CRM 客户匹配和头像 URL 校验。
|
||||
- Ledger:pre-v7(`oldVersion < 7`)升级先清空全部旧 OneTalk store 再重建当前 profile 与消息同步状态,v7→v8 升级保留五个既有 store;账号/会话键、durable-first 顺序、同 pending/已上传指纹一次读零写、同键并发写、读中 dispose、显式读取失败、ACK/CAS、future-skew 丢弃、重连与重启恢复。升级必须保留 configuration/deviceId,且不得 rekey 或重试旧 pending ledger。
|
||||
- Ledger:pre-v7(`oldVersion < 7`)升级先清空全部旧 OneTalk store 再重建当前 profile 与消息同步状态,v7→v8 升级保留五个既有 store,v8→v9 逐条保留六个 OneTalk store record 并创建 scope index;账号/会话键、durable-first 顺序、同 pending/已上传指纹一次读零写、同键并发写、读中 dispose、显式读取失败、ACK/CAS、future-skew 丢弃、重连与重启恢复。测试必须区分 profile account/pending index 与 `objectStore.getAll()`,并确认 uploaded record 不会混入 pending flush。升级必须保留 configuration/deviceId,且不得 rekey 或重试旧 pending ledger。
|
||||
- Service Worker:既有 Bright binding/read/sync 授权、page-first 与 auth-first 两种顺序的首次认证 setup、ledger 读取后授权丢失、实时 sent/received targeted collect、历史排除、50/51 条 profile 批次、ACK 计数、future 错误映射和过期回调/页面身份 fence;direct discovery 始终携带 `conversationType: "direct"`。
|
||||
- 必须执行定向 typecheck、contract/extension focused 与全量测试、format check 和 `git diff --check`。真实 Chromium、Bright PostgreSQL 和生产 Mind 集成是另行执行的外部检查。
|
||||
|
||||
|
||||
@@ -61,10 +61,18 @@ rejected
|
||||
|
||||
### Profile ledger (independent state machine)
|
||||
|
||||
联系人资料不使用消息 candidate/checkpoint/anomaly store。`ONE_TALK_SYNC_DATABASE_VERSION=8`:从 `oldVersion < 7` 升级时,transaction 必须清空五个 OneTalk store(消息、candidate、checkpoint、anomaly、`onetalk_contact_profiles`),不重键或回补旧 ledger;从 v7 升级到 v8 时只新增 `onetalk_conversation_bootstraps` store,必须保留五个既有 store 的记录。Chrome 配置、deviceId、binding 与其它渠道存储不属于该删除范围;首次升级后必须重新采集 profile 并从 clean state 执行 full sync,v7→v8 不得因为新增 bootstrap store 清理既有事实。v7 checkpoint 新增 OneTalk 会话列表来源的 `latestMessageAtMs`,用于同步完成后更新服务端会话活动时间。
|
||||
联系人资料不使用消息 candidate/checkpoint/anomaly store。`ONE_TALK_SYNC_DATABASE_VERSION=9`:从 `oldVersion < 7` 升级时,transaction 必须先清空五个 OneTalk store(消息、candidate、checkpoint、anomaly、`onetalk_contact_profiles`),不重键或回补旧 ledger;从 v7 升级到 v8 时只新增 `onetalk_conversation_bootstraps` store,必须保留五个既有 store 的记录;从 v7/v8 升级到 v9 时只为既有消息、checkpoint、candidate、anomaly、profile records 建立账号、会话与状态查询索引,六个 store 的记录逐条保留。legacy clear 完成后才建 index,避免为必然删除的 records 建索引。Chrome 配置、deviceId、binding 与其它渠道存储不属于该删除范围;首次 pre-v7 升级后必须重新采集 profile 并从 clean state 执行 full sync,v7→v8/v9 不得因为 schema 演进清理既有事实。v7 checkpoint 新增 OneTalk 会话列表来源的 `latestMessageAtMs`,用于同步完成后更新服务端会话活动时间。
|
||||
|
||||
清空后的 `onetalk_contact_profiles` 业务键为 `channelAccountId + conversationId`;记录包含 key、账号、conversationId、资料字段中的 aliId、lastUploadedFingerprint、uploaded/rejected observed high-water mark、updatedAt、lastUploadedAt 和最新 pending 清洗 profile。资料上传到 Bright persistence,不调用 Mind profile HTTP。
|
||||
|
||||
### v9 scope-bounded reads
|
||||
|
||||
- `checkpoint` 使用 `by_account`;candidate 使用 `by_account_status`、`by_account_conversation` 与 `by_account_conversation_status`;message/anomaly 使用 `by_account_conversation`;profile 使用 `by_account` 和 `by_account_pending_observed_at`。bootstrap 继续只按稳定主键读取,不为无消费者的查询建 index。
|
||||
- 已知 ACK `candidateKey` 必须直接主键读取,且只接纳仍为 `pending_ack` 的 candidate;没有 request key 时才兼容查找 `(channelAccountId, conversationId, messageId)`。确认后只比较 checkpoint 当前 `latestMessageId` candidate 与本次 confirmed candidate 的 `candidateOrder`,不得重建全会话候选列表。
|
||||
- profile pending 读取必须使用 `[channelAccountId, pending.observedAtMs]` 的合法账号前缀 `IDBKeyRange`;缺少 `pending.observedAtMs` 的 uploaded/rejected record 没有该 index entry,不得由调用方再过滤整表。
|
||||
- 会话重建必须在一个 readwrite transaction 中,对目标 scope index 执行 `getAllKeys()` 并删除 message/candidate/anomaly,再直接删除该会话 checkpoint;不得在 transaction 外读后删,也不得使用 object-store `getAll()`。
|
||||
- v9 profile 一旦打开,v8 binary 打开同一数据库会因版本较低失败。发布回滚只能通过 v10 前向迁移,或保留 v9 schema 并仅禁用非 schema 行为;不得重发 v8 或清除 durable ledger。
|
||||
|
||||
Observation 先写最新 pending,再由现有 Bright WebSocket 发送 contact.profile.observed。同 fingerprint 且无 pending 时只推进更高 observed high-water mark;任何不高于 uploaded/rejected high-water mark 的不同 fingerprint 也跳过。断线、Service Worker 重启或新页面连接只从 pending 重建发送。收到 contact.profile.ack 后,必须等待 readwrite transaction oncomplete,且只确认仍匹配的 fingerprint 和 observedAtMs;迟到旧 ACK 不得删除新 pending。收到 `ws.error` `profile_observed_at_future` 时只丢弃该 request 的精确 pending,避免无限重试。
|
||||
|
||||
消息与 profile 的耐久状态仍然分离:历史/实时消息中的 `business_card` 只持久化 `{ version: 1, kind: "business_card" }` marker。由于 `__conversationListData__` 与消息采集可能异步,消息候选不得等待或读取 profile ledger;当前客户资料只在服务端读取时按 `[channelAccountId, conversationId]` 以内存方式组合到对外 view。没有 profile 时保留 marker,不能 fallback 到登录人资料。
|
||||
@@ -171,7 +179,7 @@ Service Worker 重启后必须从 IndexedDB 恢复:
|
||||
- 断线恢复会重新发现会话并恢复未确认事实和 completion 声明。
|
||||
- \`delivery_unknown\` 不创建发送任务、不自动重发;迟到消息继续进入普通 observation。
|
||||
- 页面同步路由、Bright ACK 和 IndexedDB 事务顺序在重启/断线下保持一致。
|
||||
- `oldVersion < 7` 升级到 v8 必须断言五个 OneTalk store 均为空、没有 profile 重键/重试分支,且下一次 bootstrap 重新采集 profile、会话活动时间并执行 full sync;v7→v8 必须断言新增 bootstrap store 且五个既有 store 逐条保留;配置、deviceId 与其它渠道数据在两条路径都保持不变。
|
||||
- `oldVersion < 7` 升级到 v9 必须断言五个 OneTalk store 均为空、没有 profile 重键/重试分支,且下一次 bootstrap 重新采集 profile、会话活动时间并执行 full sync;v7→v8 必须断言新增 bootstrap store 且五个既有 store 逐条保留;v8→v9 必须断言六个 OneTalk store 的 records 逐条保留、目标 index 存在且读取不调用 `objectStore.getAll()`;配置、deviceId 与其它渠道数据在所有升级路径保持不变。
|
||||
|
||||
## 7. Wrong vs Correct
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ OneTalk 插件同时需要以下能力时,遵循本总览和对应子规范:
|
||||
| --- | --- |
|
||||
| [OneTalk 页面桥、Port 与命令路由](./page-bridge.md) | MAIN/ISOLATED/SW 页面桥、Port 注册、页面身份和 command 路由 |
|
||||
| [OneTalk 耐久同步与连接生命周期](./durable-sync.md) | IndexedDB、full/incremental/live、ACK、checkpoint、重启恢复和连接生命周期 |
|
||||
| [OneTalk 联系人资料 Bright 持久化](./contact-profile-sync.md) | profile 白名单、Bright profile frame、v7 清空后重采集 ledger、ACK/HWM/future-skew 和账号/epoch 隔离 |
|
||||
| [OneTalk 联系人资料 Bright 持久化](./contact-profile-sync.md) | profile 白名单、Bright profile frame、pre-v7 清空后重采集、v9 scope index、ACK/HWM/future-skew 和账号/epoch 隔离 |
|
||||
| [OneTalk 扩展安装实例设备身份](./device-identity.md) | deviceId 生成、迁移、独立存储、配置清除和生命周期 |
|
||||
| [OneTalk Service Worker 状态与诊断](./runtime-diagnostics.md) | getSnapshot、错误投影、敏感信息脱敏和 development 构建 |
|
||||
| [OneTalk PWA 出站发送 SOP](./send-sop.md) | sendUIMessages 输入、SDK-only 发送和 WebSocket 旁路事实确认 |
|
||||
@@ -36,7 +36,7 @@ OneTalk MAIN world
|
||||
-> Bright WebSocket upload
|
||||
-> per-message ACK
|
||||
|
||||
联系人资料事实使用独立路径:OneTalk MAIN snapshot/syncData → safe profile envelope + page identity → Service Worker v7 clean-state profile ledger → existing Bright plugin WebSocket `contact.profile.observed` → Bright guarded profile transaction → `contact.profile.ack`。
|
||||
联系人资料事实使用独立路径:OneTalk MAIN snapshot/syncData → safe profile envelope + page identity → Service Worker pre-v7 clean-state / v9 scope-indexed profile ledger → existing Bright plugin WebSocket `contact.profile.observed` → Bright guarded profile transaction → `contact.profile.ack`。
|
||||
\`\`\`
|
||||
|
||||
服务端命令:
|
||||
@@ -247,7 +247,7 @@ writer.sendSendConfirmation(result);
|
||||
- `authorization_unavailable` 表示授权依赖暂时不可用,只关闭当前 Bright socket 并沿既有连接退避自动重连;只有凭证、授权版本、binding、scope 或协议版本等确定性错误才阻断自动重连并进入 unauthorized。
|
||||
- Service Worker 重启从 IndexedDB 恢复 checkpoint、候选和模式,不信任旧内存 cursor。
|
||||
- live `messageType: "new"`(sent 或 received)只触发所属 conversation 的 profile collect;history 不触发。profile 相同 fingerprint 在 coordinator 的唯一 durable read 后静默结束,不能通过 MAIN `seen`、时间水位或 timer 再建第二去重状态。
|
||||
- `oldVersion < 7` 的数据库先执行 v7 全量 OneTalk state 清空(v7→v8 只新增 bootstrap store、保留既有事实),再重新采集 profile、会话活动时间并执行 full sync;之后的重启/重连才从当前 ledger 恢复 pending。ACK 只在当前 `[channelAccountId, conversationId, fingerprint, observedAtMs]` 的 IndexedDB transaction commit 后生效;future-skew 整批拒绝并只丢弃匹配 pending。
|
||||
- `oldVersion < 7` 的数据库先执行全量 OneTalk state 清空(v7→v8 只新增 bootstrap store、保留既有事实;v8→v9 只新增 scope index、逐条保留 records),再重新采集 profile、会话活动时间并执行 full sync;之后的重启/重连才从当前 ledger 恢复 pending。v9 profile pending flush 只读取 `[channelAccountId, pending.observedAtMs]` index,不得扫描其它账号或已上传 record。ACK 只在当前 `[channelAccountId, conversationId, fingerprint, observedAtMs]` 的 IndexedDB transaction commit 后生效;future-skew 整批拒绝并只丢弃匹配 pending。
|
||||
|
||||
### Send and protocol boundaries
|
||||
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
{"file":".trellis/spec/project/architecture.md","reason":"Review state ownership, transaction boundaries, source layout, and no duplicate derived source."}
|
||||
{"file":".trellis/spec/chrome-extension/frontend/onetalk/durable-sync.md","reason":"Verify durable ledger, migration, ACK, and recovery behavior against the canonical contract."}
|
||||
{"file":".trellis/spec/chrome-extension/frontend/onetalk/contact-profile-sync.md","reason":"Verify profile pending and ACK behavior remains independent from message ledger changes."}
|
||||
@@ -0,0 +1,74 @@
|
||||
# OneTalk IndexedDB 查询索引设计
|
||||
|
||||
## Design decision
|
||||
|
||||
采用一次 v9 schema migration、scope/status 索引和精确 ACK 主键读取。保留所有 ledger records 和 JSON 序列化主键;将“整表传输到 JS 后过滤”改为 IndexedDB 内部按索引范围限制。confirmed candidate 不是临时队列,继续作为可恢复、可审计的 durable ledger。
|
||||
|
||||
## Storage schema
|
||||
|
||||
`ensureSyncStores` 拆成“确保 store 存在”和“确保 index 存在”两部分,只在 `onupgradeneeded` 的 versionchange transaction 执行。v9 的私有 index:
|
||||
|
||||
| Store | Index | Key path | Consumer |
|
||||
| --- | --- | --- | --- |
|
||||
| message | `by_account_conversation` | `[channelAccountId, conversationId]` | rebuild delete keys |
|
||||
| checkpoint | `by_account` | `channelAccountId` | resume、retry completion、bootstrap snapshot |
|
||||
| candidate | `by_account_status` | `[channelAccountId, status]` | account pending resume |
|
||||
| candidate | `by_account_conversation_status` | `[channelAccountId, conversationId, status]` | drain、awaiting-anchor、pending count |
|
||||
| candidate | `by_account_conversation` | `[channelAccountId, conversationId]` | rebuild delete keys |
|
||||
| anomaly | `by_account` | `channelAccountId` | account anomaly read |
|
||||
| anomaly | `by_account_conversation` | `[channelAccountId, conversationId]` | conversation anomaly/read-delete |
|
||||
| contact profile | `by_account` | `channelAccountId` | record count |
|
||||
| contact profile | `by_account_pending_observed_at` | `[channelAccountId, pending.observedAtMs]` | pending flush |
|
||||
|
||||
bootstrap 继续只有主键,因为只按稳定 `(channelAccountId, migrationId)` 读取。profile pending index 使用已有嵌套标量;没有 pending 的 record 没有 index entry,v8 的 pending record 会在创建 index 时自动进入,不增加平行 `hasPending` 字段或 backfill source of truth。
|
||||
|
||||
## Migration and compatibility
|
||||
|
||||
1. `ONE_TALK_SYNC_DATABASE_VERSION` 从 8 升至 9。
|
||||
2. 先确保六个 stores 存在;若 `oldVersion < 7`,按既有契约清空五个 legacy stores;最后通过 `indexNames.contains` 幂等建 index。
|
||||
3. v7/v8 → v9 保留 records,浏览器从现有 records 建 index;不重键、不回放、不清库。
|
||||
4. 保留 `onblocked` rejection,不增加 silent fallback。
|
||||
|
||||
先清 legacy 再建 index,避免为必然删除的数据建索引。v9 数据库不能由 v8 代码打开,因此线上回滚只能是 v10 前向兼容修复,不能直接重发旧扩展。
|
||||
|
||||
## Query API and call-site refactor
|
||||
|
||||
以带范围的 `readByIndex`、`readKeysByIndex`、`countByIndex` 和既有 `readOne` 取代通用 `readAll`。所有生产调用已有账号 scope,收紧 optional account 参数,不保留跨账号全表枚举。
|
||||
|
||||
| Current path | Replacement | Result bound |
|
||||
| --- | --- | --- |
|
||||
| `listCheckpoints(account)` | checkpoint `by_account.getAll(only(account))` | account checkpoints |
|
||||
| `listPendingCandidates(account)` | candidate `by_account_status.getAll(only([account, pending_ack]))` | account pending |
|
||||
| conversation candidate list + filter | `by_account_conversation_status` | conversation and status |
|
||||
| request key → pending list → find | `getCandidateByKey(requestKey)` + `pending_ack` guard | one candidate |
|
||||
| latest confirmed from all candidates | checkpoint `latestMessageId` candidate vs just-confirmed candidate by `candidateOrder` | at most two candidates |
|
||||
| completion pending check | status index `count` | one count |
|
||||
| `listAnomalies` | account or scope index | account or conversation |
|
||||
| `listPendingProfiles(account)` | profile pending compound index range | account pending profiles |
|
||||
| `hasProfileRecord(account)` | profile account-index `count` | one count |
|
||||
| `clearConversationHistory` | scope-index `getAllKeys`; checkpoint direct delete | target conversation keys only |
|
||||
|
||||
`AckCompletionCoordinator` 保持协议语义的具体变更:
|
||||
|
||||
- `activateAwaitingCandidates` 精确取 anchor,只列出 `awaiting_anchor`。
|
||||
- `drainConversation` 只列出 `pending_ack`,保留排序、批次和 in-flight guards。
|
||||
- 有 request key 的 ACK 直接读取 candidate,并要求它仍是 `pending_ack`;没有 key 时保留 `(account, conversation, messageId)` 回退。
|
||||
- durable status 更新后,以 checkpoint 的 `latestMessageId` 精确读取当前最新 candidate,并与刚确认 candidate 用 `candidateOrder` 比较;因此乱序 ACK 仍得到相同最大 confirmed message,无需物化整会话。
|
||||
- `tryComplete` 用 pending index count 代替全 candidate `some`。explicit-start gate 改为发送/ACK 调用携带精确的 in-memory 标记,不扫描 durable history 寻找该标记;现有 reset/clear 时同步清理。
|
||||
|
||||
不解析或依赖 JSON key 的字符串排序,不改变既有 `getCandidate(account, conversation, messageId)`。
|
||||
|
||||
## Transaction and failure semantics
|
||||
|
||||
会话重建仍使用同一个四 store readwrite transaction:每个 scope index `getAllKeys(range)` 后对返回 key `delete`,最后等待该 transaction `oncomplete`。不在 transaction 外先读后删,不引入异常吞没或 fallback。
|
||||
|
||||
ACK 顺序仍为 durable candidate 读取 → candidate 状态写入 → checkpoint 写入(若需要)→ completion 判断。既有 lifecycle、generation、quiesce 和 authorization guards 均不因缩小查询范围而绕开。
|
||||
|
||||
## Test design
|
||||
|
||||
1. 扩展 storage/profile tests 的 Fake IndexedDB,使其可观测 `objectStore.getAll`,并最小实现 `createIndex`、`index`、`getAll`、`getAllKeys`、`count`;加入 v8 seed → v9 upgrade。
|
||||
2. 以两账号、两会话、不同 candidate status、账号级 anomaly、pending/non-pending profile 断言精确结果,并断言目标读取未调用 object-store `getAll`。
|
||||
3. 覆盖会话清理只删除目标会话;覆盖精确 request key、批量/乱序 ACK、pending resume、awaiting-anchor、completion gate 和 profile flush。
|
||||
4. 保留并重跑 `<7` clear、v7→v8 保留断言;新增 v8→v9 preservation。
|
||||
|
||||
功能测试证明查询范围,不将 fake timing 当性能证据。真实 Chromium profile 的 IDB latency、index-build 时间、Worker memory/termination 采样作为后续独立验证。
|
||||
@@ -0,0 +1,3 @@
|
||||
{"file":".trellis/spec/project/architecture.md","reason":"Preserve state ownership, async boundaries, and source responsibilities during storage/query refactor."}
|
||||
{"file":".trellis/spec/chrome-extension/frontend/onetalk/durable-sync.md","reason":"Preserve durable-first, candidate ACK, checkpoint, restart, and migration contracts."}
|
||||
{"file":".trellis/spec/chrome-extension/frontend/onetalk/contact-profile-sync.md","reason":"Preserve independent profile ledger and ACK semantics while indexing profile reads."}
|
||||
@@ -0,0 +1,41 @@
|
||||
# Implementation plan: OneTalk IndexedDB scoped queries
|
||||
|
||||
## Scope and ownership
|
||||
|
||||
Primary implementation owner is `apps/chrome-extension/src/onetalk/service-worker/storage.ts`. Affected consumers are `sync-engine/ack-completion.ts` and existing test doubles under `apps/chrome-extension/test/`; Bright, Mind, page-bridge and authorization contracts are out of scope.
|
||||
|
||||
## Ordered work
|
||||
|
||||
1. Before each production-symbol edit, run GitNexus impact analysis and load the target frontend/durable-sync specs.
|
||||
2. Add v9 index constants/helpers and versionchange migration; preserve old-version clear order and existing open/blocked failure behavior.
|
||||
3. Replace generic all-store reader with bounded primary/index helpers. Add exact candidate-key, status-scoped candidate, scoped anomaly/profile, and transaction-local `getAllKeys` delete APIs.
|
||||
4. Refactor `AckCompletionCoordinator` to use direct request-key reads, status-scoped lists, checkpoint-vs-candidate latest comparison, and indexed pending completion checks. Keep queue/generation/ACK order unchanged.
|
||||
5. Update type-conforming test fakes and focused tests; add v8→v9 compatibility and query-path instrumentation.
|
||||
6. Run focused tests, typecheck, build, full tests and format check; review for retention, protocol or fallback regressions.
|
||||
7. Before commit, run GitNexus `detect_changes()` and inspect affected flows. Commit/release remain separately authorized.
|
||||
|
||||
## Validation commands
|
||||
|
||||
```bash
|
||||
pnpm --filter @trade-message-center/chrome-extension test -- onetalk-sync-storage.test.js
|
||||
pnpm --filter @trade-message-center/chrome-extension test -- onetalk-contact-profile-storage.test.js
|
||||
pnpm --filter @trade-message-center/chrome-extension test -- onetalk-sync-engine.test.js
|
||||
pnpm typecheck
|
||||
pnpm build
|
||||
pnpm test
|
||||
pnpm format:check
|
||||
```
|
||||
|
||||
Use the repository-supported focused-test argument form confirmed at implementation time. Any unexpectedly affected backend unit test keeps the project 60-second timeout rule.
|
||||
|
||||
## Review gates
|
||||
|
||||
- Every target query carries an account/conversation/status IndexedDB range; target code has no `objectStore.getAll()`.
|
||||
- No confirmed candidate deletion, primary-key shape change, raw payload index or Bright frame change.
|
||||
- v8 preservation and `<7` cleanup are verified independently.
|
||||
- New reads/writes stay in IndexedDB transactions and `oncomplete` remains the durable boundary.
|
||||
- Release notes state the no-binary-rollback v9 constraint.
|
||||
|
||||
## Rollback
|
||||
|
||||
Before release, source rollback is normal Git reversal. After a profile receives v9, do not deploy v8 against that profile; publish a v10 forward migration that restores compatible behavior, or retain v9 schema code and explicitly disable only a non-schema behavior. Never clear durable ledger data as rollback.
|
||||
@@ -0,0 +1,67 @@
|
||||
# 修复 OneTalk IndexedDB 查询全表扫描
|
||||
|
||||
## Goal
|
||||
|
||||
在不改变 OneTalk durable-first、候选 ACK、锚点推进、profile ledger 或历史保留语义的前提下,消除同步账本按整个对象仓库 `getAll()` 后在内存过滤的查询路径。目标是让 ACK、重连恢复、会话重建和 profile flush 的读取范围由账号、会话和状态键限定,而不是由历史总量决定。
|
||||
|
||||
## Confirmed facts
|
||||
|
||||
- `storage.ts` 当前数据库版本为 8;`ensureSyncStores` 仅为六个 OneTalk store 建立 `{ keyPath: "key" }`,没有二级索引(`apps/chrome-extension/src/onetalk/service-worker/storage.ts:16,275-288`)。
|
||||
- `readAll` 调用 `objectStore.getAll()`;`listCheckpoints`、`listCandidates`、`listPendingCandidates`、`listAnomalies`、`listPendingProfiles`、`hasProfileRecord` 都经由该路径读取整表(`storage.ts:384-390,605-611,644-671,722-730,841-862`)。
|
||||
- `clearConversationHistory` 对 message、checkpoint、candidate、anomaly 四个 store 各调用一次 `getAll()`,再逐记录删除(`storage.ts:740-765`)。
|
||||
- 每条有效观察都会持久化 candidate,确认后保留;仅显式会话重建清理候选。因此 candidate 表的规模会随历史同步积累(`storage.ts:514-555`)。
|
||||
- 已关联的 ACK 请求携带精确 `candidateKey`,但 `handleAcknowledgementResult` 仍先 `listPendingCandidates(account)` 再 `.find(key)`;批量 ACK 对每个结果串行执行(`ack-completion.ts:377-405,545-548`)。
|
||||
- 认证/重连会调用 `resumePendingCandidates` 和 `retrySentCompletions`(`sync-engine.ts:340-350,367-378`)。现有调用点都传入 `channelAccountId`;没有生产调用需要跨账号列出 pending candidate 或 pending profile。
|
||||
- v7→v8 必须保留既有 OneTalk ledger;`oldVersion < 7` 才会清空五个 legacy store。该兼容契约不可倒退(`.trellis/spec/chrome-extension/frontend/onetalk/durable-sync.md`)。
|
||||
|
||||
## Requirements
|
||||
|
||||
### R1. Schema migration
|
||||
|
||||
- 将同步数据库升级到 v9;为既有和新建 store 幂等创建查询索引。
|
||||
- v8→v9 只建立索引,必须逐条保留 message、checkpoint、candidate、anomaly、profile 和 bootstrap 记录;不得重键、清库或回放观察。
|
||||
- `oldVersion < 7` 的既有清库规则保持不变;清理完成后再建立索引,避免为必然删除的 legacy 记录建立索引。
|
||||
- index 名称和 key path 是 `storage.ts` 私有 schema,不能泄露到 Bright、Mind 或页面桥合同。
|
||||
|
||||
### R2. Scope-bounded reads
|
||||
|
||||
- candidate:支持按精确 key、`[channelAccountId, conversationId]` 和候选状态读取;按账号恢复 pending 时只读取该账号的 `pending_ack`。
|
||||
- checkpoint:按账号列出恢复/完成声明所需 checkpoint;按精确主键删除单会话 checkpoint。
|
||||
- message、candidate、anomaly:会话重建只读取匹配 `[channelAccountId, conversationId]` 的 keys,再在同一 readwrite transaction 删除;不得物化不相关记录。
|
||||
- profile:`hasProfileRecord` 使用账号索引计数;pending flush 仅从该账号的 pending profile 索引读取。
|
||||
- anomaly 的现有账号/会话查询保持结果和账号级 anomaly 的排除语义。
|
||||
|
||||
### R3. ACK and sync semantics
|
||||
|
||||
- ACK 已有 `candidateKey` 时必须以主键直接读取,且只接受 `pending_ack` candidate;保留没有 request key 时按 `(account, conversation, messageId)` 的兼容查找。
|
||||
- 按会话 drain 只读取 `pending_ack`;激活增量候选只读取 `awaiting_anchor` 及精确 anchor candidate。
|
||||
- ACK 后最新 confirmed message 的选择继续遵守 `candidateOrder`。不得因索引重构把较旧、乱序 ACK 覆盖成最新锚点。
|
||||
- `tryComplete` 只查询是否仍有 pending candidate;现有 explicit-start/boundary gate 语义不得变化。
|
||||
- 不删除 confirmed candidate,不改业务主键、不改 Bright frame、不改变 ACK 先 durable write 后推进的顺序。
|
||||
|
||||
### R4. Verification
|
||||
|
||||
- 测试必须证明 v8→v9 记录保留和 index 创建;`oldVersion < 7` 的清理行为继续成立。
|
||||
- 存储层测试必须能区分 `objectStore.getAll()` 与 `IDBIndex.getAll()` / `getAllKeys()` / `count()`,并断言目标查询未走全表读取。
|
||||
- 同步引擎测试必须覆盖精确 request key ACK、批量 ACK、断线恢复、增量 anchor 和 completion 路径,确认 observable 协议行为未变。
|
||||
|
||||
## Out of scope
|
||||
|
||||
- 不引入 confirmed candidate 过期/压缩策略,不删除历史消息或改变 retention。
|
||||
- 不变更 IndexedDB database name、JSON 序列化主键、OneTalk/Bright/Mind frame、页面桥或授权逻辑。
|
||||
- 不承诺本任务已在真实用户 profile 的数万条数据上测得 worker 卡顿或内存下降;该问题需要单独的真实浏览器性能采样。
|
||||
- 不为只按主键读取的 bootstrap store 增加无消费者的索引。
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] AC1: v9 upgrade 为当前 records 建立所需索引,v8→v9 六个 OneTalk store 的记录数量与内容保持不变;`oldVersion < 7` 仍按现有契约清空五个 legacy store。
|
||||
- [ ] AC2: `storage.ts` 不再存在用于业务查询或会话删除的 `objectStore.getAll()`;所有目标读取均由主键或相应 `IDBIndex` 限定。
|
||||
- [ ] AC3: 正常 ACK 不再调用 `listPendingCandidates` 来按已知 request key 找 candidate;乱序 ACK 下 `latestMessageId` 仍是 `candidateOrder` 最大的 confirmed candidate。
|
||||
- [ ] AC4: 重连恢复、pending drain、awaiting-anchor 激活、completion、profile flush 和会话重建的结果与现有 focused tests 一致,且不读取其他账号/会话记录。
|
||||
- [ ] AC5: 目标存储、profile、同步引擎测试以及 typecheck、build、全量 test 和 format check 通过。
|
||||
|
||||
## Risks and operational constraints
|
||||
|
||||
- 建立新索引需要在首次打开 v9 数据库时遍历现有记录,成本为一次性迁移;升级被阻塞时必须继续显式失败,不能伪造成功或回退到全表扫描。
|
||||
- IndexedDB 版本不可降级。v9 已写入后,旧的 v8 扩展不能打开同一数据库;发布回滚必须使用前向 v10 修复或等待客户端更新,不能直接回退扩展包。
|
||||
- 测试中的轻量 Fake IndexedDB 当前没有 `IDBIndex` 行为。测试 harness 需要扩展为最小的可观测 index 模型,或替换为能验证真实 index API 的测试依赖;不得仅靠 mock 返回值宣称查询走了索引。
|
||||
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"id": "onetalk-indexeddb-query-indexes",
|
||||
"name": "onetalk-indexeddb-query-indexes",
|
||||
"title": "修复 OneTalk IndexedDB 查询全表扫描",
|
||||
"description": "为 OneTalk 同步账本设计索引、迁移与查询重构方案,消除 candidate、checkpoint 和 profile 的全表读取。",
|
||||
"status": "in_progress",
|
||||
"dev_type": null,
|
||||
"scope": null,
|
||||
"package": null,
|
||||
"priority": "P2",
|
||||
"creator": "ybf",
|
||||
"assignee": "ybf",
|
||||
"createdAt": "2026-09-14",
|
||||
"completedAt": null,
|
||||
"branch": "09-14-onetalk-indexeddb-query-indexes",
|
||||
"base_branch": "main",
|
||||
"worktree_path": null,
|
||||
"commit": null,
|
||||
"pr_url": null,
|
||||
"subtasks": [],
|
||||
"children": [],
|
||||
"parent": null,
|
||||
"relatedFiles": [],
|
||||
"notes": "",
|
||||
"meta": {}
|
||||
}
|
||||
@@ -222,11 +222,14 @@ export type OneTalkSyncStore = {
|
||||
conversationId: string,
|
||||
messageId: string,
|
||||
) => Promise<OneTalkSyncCandidate | null>;
|
||||
getCandidateByKey: (key: string) => Promise<OneTalkSyncCandidate | null>;
|
||||
listCandidates: (
|
||||
channelAccountId: string,
|
||||
conversationId: string,
|
||||
status?: OneTalkSyncCandidateStatus,
|
||||
) => Promise<OneTalkSyncCandidate[]>;
|
||||
listPendingCandidates: (channelAccountId?: string) => Promise<OneTalkSyncCandidate[]>;
|
||||
listPendingCandidates: (channelAccountId: string) => Promise<OneTalkSyncCandidate[]>;
|
||||
hasPendingCandidates: (channelAccountId: string, conversationId: string) => Promise<boolean>;
|
||||
putCandidate: (candidate: OneTalkSyncCandidate) => Promise<void>;
|
||||
updateCandidate: (
|
||||
candidate: OneTalkSyncCandidate,
|
||||
@@ -246,9 +249,7 @@ export type OneTalkContactProfileStore = {
|
||||
channelAccountId: string,
|
||||
conversationId: string,
|
||||
) => Promise<OneTalkContactProfileLedgerRecord | null>;
|
||||
listPendingProfiles: (
|
||||
channelAccountId?: string,
|
||||
) => Promise<OneTalkContactProfileLedgerRecord[]>;
|
||||
listPendingProfiles: (channelAccountId: string) => Promise<OneTalkContactProfileLedgerRecord[]>;
|
||||
hasProfileRecord: (channelAccountId: string) => Promise<boolean>;
|
||||
putPendingProfile: (
|
||||
channelAccountId: string,
|
||||
@@ -337,6 +338,21 @@ const toStoredObservation = (
|
||||
};
|
||||
};
|
||||
|
||||
const ONE_TALK_MESSAGE_BY_ACCOUNT_CONVERSATION_INDEX = "by_account_conversation";
|
||||
const ONE_TALK_CHECKPOINT_BY_ACCOUNT_INDEX = "by_account";
|
||||
const ONE_TALK_CANDIDATE_BY_ACCOUNT_STATUS_INDEX = "by_account_status";
|
||||
const ONE_TALK_CANDIDATE_BY_ACCOUNT_CONVERSATION_STATUS_INDEX = "by_account_conversation_status";
|
||||
const ONE_TALK_CANDIDATE_BY_ACCOUNT_CONVERSATION_INDEX = "by_account_conversation";
|
||||
const ONE_TALK_ANOMALY_BY_ACCOUNT_INDEX = "by_account";
|
||||
const ONE_TALK_ANOMALY_BY_ACCOUNT_CONVERSATION_INDEX = "by_account_conversation";
|
||||
const ONE_TALK_CONTACT_PROFILE_BY_ACCOUNT_INDEX = "by_account";
|
||||
const ONE_TALK_CONTACT_PROFILE_BY_ACCOUNT_PENDING_OBSERVED_AT_INDEX =
|
||||
"by_account_pending_observed_at";
|
||||
|
||||
const ensureIndex = (store: IDBObjectStore, name: string, keyPath: string | string[]): void => {
|
||||
if (!store.indexNames.contains(name)) store.createIndex(name, keyPath);
|
||||
};
|
||||
|
||||
const ensureSyncStores = (database: IDBDatabase): void => {
|
||||
const stores = [
|
||||
ONE_TALK_MESSAGE_STORE_NAME,
|
||||
@@ -354,6 +370,49 @@ const ensureSyncStores = (database: IDBDatabase): void => {
|
||||
}
|
||||
};
|
||||
|
||||
const ensureSyncIndexes = (transaction: IDBTransaction): void => {
|
||||
ensureIndex(
|
||||
transaction.objectStore(ONE_TALK_MESSAGE_STORE_NAME),
|
||||
ONE_TALK_MESSAGE_BY_ACCOUNT_CONVERSATION_INDEX,
|
||||
["channelAccountId", "conversationId"],
|
||||
);
|
||||
ensureIndex(
|
||||
transaction.objectStore(ONE_TALK_CHECKPOINT_STORE_NAME),
|
||||
ONE_TALK_CHECKPOINT_BY_ACCOUNT_INDEX,
|
||||
"channelAccountId",
|
||||
);
|
||||
const candidateStore = transaction.objectStore(ONE_TALK_CANDIDATE_STORE_NAME);
|
||||
ensureIndex(candidateStore, ONE_TALK_CANDIDATE_BY_ACCOUNT_STATUS_INDEX, [
|
||||
"channelAccountId",
|
||||
"status",
|
||||
]);
|
||||
ensureIndex(candidateStore, ONE_TALK_CANDIDATE_BY_ACCOUNT_CONVERSATION_STATUS_INDEX, [
|
||||
"channelAccountId",
|
||||
"conversationId",
|
||||
"status",
|
||||
]);
|
||||
ensureIndex(candidateStore, ONE_TALK_CANDIDATE_BY_ACCOUNT_CONVERSATION_INDEX, [
|
||||
"channelAccountId",
|
||||
"conversationId",
|
||||
]);
|
||||
ensureIndex(
|
||||
transaction.objectStore(ONE_TALK_ANOMALY_STORE_NAME),
|
||||
ONE_TALK_ANOMALY_BY_ACCOUNT_INDEX,
|
||||
"channelAccountId",
|
||||
);
|
||||
ensureIndex(
|
||||
transaction.objectStore(ONE_TALK_ANOMALY_STORE_NAME),
|
||||
ONE_TALK_ANOMALY_BY_ACCOUNT_CONVERSATION_INDEX,
|
||||
["channelAccountId", "conversationId"],
|
||||
);
|
||||
const profileStore = transaction.objectStore(ONE_TALK_CONTACT_PROFILE_STORE_NAME);
|
||||
ensureIndex(profileStore, ONE_TALK_CONTACT_PROFILE_BY_ACCOUNT_INDEX, "channelAccountId");
|
||||
ensureIndex(profileStore, ONE_TALK_CONTACT_PROFILE_BY_ACCOUNT_PENDING_OBSERVED_AT_INDEX, [
|
||||
"channelAccountId",
|
||||
"pending.observedAtMs",
|
||||
]);
|
||||
};
|
||||
|
||||
const clearLegacyOneTalkStores = (transaction: IDBTransaction): void => {
|
||||
for (const storeName of [
|
||||
ONE_TALK_MESSAGE_STORE_NAME,
|
||||
@@ -377,6 +436,7 @@ const openSyncDatabase = (factory: IDBFactory): Promise<IDBDatabase> => {
|
||||
if ((event?.oldVersion ?? 0) < 7 && request.transaction) {
|
||||
clearLegacyOneTalkStores(request.transaction);
|
||||
}
|
||||
if (request.transaction) ensureSyncIndexes(request.transaction);
|
||||
};
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error ?? new Error("IndexedDB open failed"));
|
||||
@@ -651,14 +711,36 @@ const readRequest = <T>(request: IDBRequest<T>): Promise<T> => {
|
||||
});
|
||||
};
|
||||
|
||||
const readAll = async <T>(database: IDBDatabase, storeName: string): Promise<T[]> => {
|
||||
const readByIndex = async <T>(
|
||||
database: IDBDatabase,
|
||||
storeName: string,
|
||||
indexName: string,
|
||||
query: IDBValidKey | IDBKeyRange,
|
||||
): Promise<T[]> => {
|
||||
const transaction = database.transaction(storeName, "readonly");
|
||||
const completion = transactionResult(transaction);
|
||||
const result = await readRequest(transaction.objectStore(storeName).getAll());
|
||||
const result = await readRequest(
|
||||
transaction.objectStore(storeName).index(indexName).getAll(query),
|
||||
);
|
||||
await completion;
|
||||
return result as T[];
|
||||
};
|
||||
|
||||
const countByIndex = async (
|
||||
database: IDBDatabase,
|
||||
storeName: string,
|
||||
indexName: string,
|
||||
query: IDBValidKey | IDBKeyRange,
|
||||
): Promise<number> => {
|
||||
const transaction = database.transaction(storeName, "readonly");
|
||||
const completion = transactionResult(transaction);
|
||||
const result = await readRequest(
|
||||
transaction.objectStore(storeName).index(indexName).count(query),
|
||||
);
|
||||
await completion;
|
||||
return result;
|
||||
};
|
||||
|
||||
const readOne = async <T>(
|
||||
database: IDBDatabase,
|
||||
storeName: string,
|
||||
@@ -874,11 +956,12 @@ export const createOneTalkSyncStore = (
|
||||
|
||||
const listCheckpoints = async (channelAccountId: string): Promise<OneTalkSyncCheckpoint[]> => {
|
||||
const database = await getDatabase();
|
||||
const checkpoints = await readAll<OneTalkSyncCheckpoint>(
|
||||
return readByIndex<OneTalkSyncCheckpoint>(
|
||||
database,
|
||||
ONE_TALK_CHECKPOINT_STORE_NAME,
|
||||
ONE_TALK_CHECKPOINT_BY_ACCOUNT_INDEX,
|
||||
channelAccountId,
|
||||
);
|
||||
return checkpoints.filter((checkpoint) => checkpoint.channelAccountId === channelAccountId);
|
||||
};
|
||||
|
||||
const putCheckpoint = async (checkpoint: OneTalkSyncCheckpoint): Promise<void> => {
|
||||
@@ -911,34 +994,56 @@ export const createOneTalkSyncStore = (
|
||||
);
|
||||
};
|
||||
|
||||
const getCandidateByKey = async (key: string): Promise<OneTalkSyncCandidate | null> => {
|
||||
const database = await getDatabase();
|
||||
return (
|
||||
(await readOne<OneTalkSyncCandidate>(database, ONE_TALK_CANDIDATE_STORE_NAME, key)) ??
|
||||
null
|
||||
);
|
||||
};
|
||||
|
||||
const listCandidates = async (
|
||||
channelAccountId: string,
|
||||
conversationId: string,
|
||||
status?: OneTalkSyncCandidateStatus,
|
||||
): Promise<OneTalkSyncCandidate[]> => {
|
||||
const database = await getDatabase();
|
||||
const candidates = await readAll<OneTalkSyncCandidate>(
|
||||
return readByIndex<OneTalkSyncCandidate>(
|
||||
database,
|
||||
ONE_TALK_CANDIDATE_STORE_NAME,
|
||||
);
|
||||
return candidates.filter(
|
||||
(candidate) =>
|
||||
candidate.channelAccountId === channelAccountId &&
|
||||
candidate.conversationId === conversationId,
|
||||
status === undefined
|
||||
? ONE_TALK_CANDIDATE_BY_ACCOUNT_CONVERSATION_INDEX
|
||||
: ONE_TALK_CANDIDATE_BY_ACCOUNT_CONVERSATION_STATUS_INDEX,
|
||||
status === undefined
|
||||
? [channelAccountId, conversationId]
|
||||
: [channelAccountId, conversationId, status],
|
||||
);
|
||||
};
|
||||
|
||||
const listPendingCandidates = async (
|
||||
channelAccountId?: string,
|
||||
channelAccountId: string,
|
||||
): Promise<OneTalkSyncCandidate[]> => {
|
||||
const database = await getDatabase();
|
||||
const candidates = await readAll<OneTalkSyncCandidate>(
|
||||
return readByIndex<OneTalkSyncCandidate>(
|
||||
database,
|
||||
ONE_TALK_CANDIDATE_STORE_NAME,
|
||||
ONE_TALK_CANDIDATE_BY_ACCOUNT_STATUS_INDEX,
|
||||
[channelAccountId, "pending_ack"],
|
||||
);
|
||||
return candidates.filter(
|
||||
(candidate) =>
|
||||
candidate.status === "pending_ack" &&
|
||||
(channelAccountId === undefined || candidate.channelAccountId === channelAccountId),
|
||||
};
|
||||
|
||||
const hasPendingCandidates = async (
|
||||
channelAccountId: string,
|
||||
conversationId: string,
|
||||
): Promise<boolean> => {
|
||||
const database = await getDatabase();
|
||||
return (
|
||||
(await countByIndex(
|
||||
database,
|
||||
ONE_TALK_CANDIDATE_STORE_NAME,
|
||||
ONE_TALK_CANDIDATE_BY_ACCOUNT_CONVERSATION_STATUS_INDEX,
|
||||
[channelAccountId, conversationId, "pending_ack"],
|
||||
)) > 0
|
||||
);
|
||||
};
|
||||
|
||||
@@ -992,11 +1097,13 @@ export const createOneTalkSyncStore = (
|
||||
conversationId?: string,
|
||||
): Promise<OneTalkSyncAnomaly[]> => {
|
||||
const database = await getDatabase();
|
||||
const anomalies = await readAll<OneTalkSyncAnomaly>(database, ONE_TALK_ANOMALY_STORE_NAME);
|
||||
return anomalies.filter(
|
||||
(anomaly) =>
|
||||
anomaly.channelAccountId === channelAccountId &&
|
||||
(conversationId === undefined || anomaly.conversationId === conversationId),
|
||||
return readByIndex<OneTalkSyncAnomaly>(
|
||||
database,
|
||||
ONE_TALK_ANOMALY_STORE_NAME,
|
||||
conversationId === undefined
|
||||
? ONE_TALK_ANOMALY_BY_ACCOUNT_INDEX
|
||||
: ONE_TALK_ANOMALY_BY_ACCOUNT_CONVERSATION_INDEX,
|
||||
conversationId === undefined ? channelAccountId : [channelAccountId, conversationId],
|
||||
);
|
||||
};
|
||||
|
||||
@@ -1015,24 +1122,20 @@ export const createOneTalkSyncStore = (
|
||||
] as const;
|
||||
const transaction = database.transaction([...storeNames], "readwrite");
|
||||
const completion = transactionResult(transaction);
|
||||
for (const storeName of storeNames) {
|
||||
for (const [storeName, indexName] of [
|
||||
[ONE_TALK_MESSAGE_STORE_NAME, ONE_TALK_MESSAGE_BY_ACCOUNT_CONVERSATION_INDEX],
|
||||
[ONE_TALK_CANDIDATE_STORE_NAME, ONE_TALK_CANDIDATE_BY_ACCOUNT_CONVERSATION_INDEX],
|
||||
[ONE_TALK_ANOMALY_STORE_NAME, ONE_TALK_ANOMALY_BY_ACCOUNT_CONVERSATION_INDEX],
|
||||
] as const) {
|
||||
const store = transaction.objectStore(storeName);
|
||||
const request = store.getAll();
|
||||
const request = store.index(indexName).getAllKeys([channelAccountId, conversationId]);
|
||||
request.onsuccess = () => {
|
||||
for (const record of request.result as Array<{
|
||||
key: string;
|
||||
channelAccountId: string;
|
||||
conversationId?: string;
|
||||
}>) {
|
||||
if (
|
||||
record.channelAccountId === channelAccountId &&
|
||||
record.conversationId === conversationId
|
||||
) {
|
||||
store.delete(record.key);
|
||||
}
|
||||
}
|
||||
for (const key of request.result) store.delete(key);
|
||||
};
|
||||
}
|
||||
transaction
|
||||
.objectStore(ONE_TALK_CHECKPOINT_STORE_NAME)
|
||||
.delete(syncKey(channelAccountId, conversationId));
|
||||
await completion;
|
||||
};
|
||||
|
||||
@@ -1042,8 +1145,10 @@ export const createOneTalkSyncStore = (
|
||||
putCheckpoint,
|
||||
persistObservedBatch: persistBatch,
|
||||
getCandidate,
|
||||
getCandidateByKey,
|
||||
listCandidates,
|
||||
listPendingCandidates,
|
||||
hasPendingCandidates,
|
||||
putCandidate,
|
||||
updateCandidate,
|
||||
recordAnomaly,
|
||||
@@ -1110,27 +1215,27 @@ export const createOneTalkContactProfileStore = (
|
||||
};
|
||||
|
||||
const listPendingProfiles = async (
|
||||
channelAccountId?: string,
|
||||
channelAccountId: string,
|
||||
): Promise<OneTalkContactProfileLedgerRecord[]> => {
|
||||
const database = await getDatabase();
|
||||
const records = await readAll<OneTalkContactProfileLedgerRecord>(
|
||||
return readByIndex<OneTalkContactProfileLedgerRecord>(
|
||||
database,
|
||||
ONE_TALK_CONTACT_PROFILE_STORE_NAME,
|
||||
);
|
||||
return records.filter(
|
||||
(record) =>
|
||||
record.pending !== undefined &&
|
||||
(channelAccountId === undefined || record.channelAccountId === channelAccountId),
|
||||
ONE_TALK_CONTACT_PROFILE_BY_ACCOUNT_PENDING_OBSERVED_AT_INDEX,
|
||||
IDBKeyRange.bound([channelAccountId], [channelAccountId, []], false, true),
|
||||
);
|
||||
};
|
||||
|
||||
const hasProfileRecord = async (channelAccountId: string): Promise<boolean> => {
|
||||
const database = await getDatabase();
|
||||
const records = await readAll<OneTalkContactProfileLedgerRecord>(
|
||||
database,
|
||||
ONE_TALK_CONTACT_PROFILE_STORE_NAME,
|
||||
return (
|
||||
(await countByIndex(
|
||||
database,
|
||||
ONE_TALK_CONTACT_PROFILE_STORE_NAME,
|
||||
ONE_TALK_CONTACT_PROFILE_BY_ACCOUNT_INDEX,
|
||||
channelAccountId,
|
||||
)) > 0
|
||||
);
|
||||
return records.some((record) => record.channelAccountId === channelAccountId);
|
||||
};
|
||||
|
||||
const putPendingProfile = async (
|
||||
|
||||
@@ -18,9 +18,7 @@ import {
|
||||
candidateOrder,
|
||||
candidateTimestamp,
|
||||
isNonEmptyString,
|
||||
latestConfirmedMessageId,
|
||||
nowOr,
|
||||
pendingCandidate,
|
||||
scopedKey,
|
||||
} from "./helpers.ts";
|
||||
import { CheckpointCoordinator } from "./checkpoint-coordinator.ts";
|
||||
@@ -59,7 +57,7 @@ export class AckCompletionCoordinator {
|
||||
private readonly checkpoints: CheckpointCoordinator;
|
||||
private readonly isBootstrapRunning: () => boolean;
|
||||
private readonly requestCandidates = new Map<string, CandidateRequest>();
|
||||
private readonly explicitlySentCandidates = new Set<string>();
|
||||
private readonly explicitlySentConversations = new Set<string>();
|
||||
private readonly discovered = new Set<string>();
|
||||
private readonly pendingDiscoveries = new Map<
|
||||
string,
|
||||
@@ -89,6 +87,7 @@ export class AckCompletionCoordinator {
|
||||
|
||||
public resetConnection(): void {
|
||||
this.requestCandidates.clear();
|
||||
this.explicitlySentConversations.clear();
|
||||
this.discovered.clear();
|
||||
this.pendingDiscoveries.clear();
|
||||
this.pendingIncompletes.clear();
|
||||
@@ -97,15 +96,8 @@ export class AckCompletionCoordinator {
|
||||
/** 清掉单会话的易失 ACK/discovery 状态,不能影响其它会话。 */
|
||||
public clearConversation(conversationId: string): void {
|
||||
const key = scopedKey(this.scope.channelAccountId, conversationId);
|
||||
const candidatePrefix = `${JSON.stringify([
|
||||
this.scope.channelAccountId,
|
||||
conversationId,
|
||||
]).slice(0, -1)},`;
|
||||
this.discovered.delete(key);
|
||||
this.explicitlySentCandidates.forEach((candidateKey) => {
|
||||
if (candidateKey.startsWith(candidatePrefix))
|
||||
this.explicitlySentCandidates.delete(candidateKey);
|
||||
});
|
||||
this.explicitlySentConversations.delete(key);
|
||||
for (const [requestId, request] of this.requestCandidates) {
|
||||
if (request.conversationId === conversationId) this.requestCandidates.delete(requestId);
|
||||
}
|
||||
@@ -153,9 +145,13 @@ export class AckCompletionCoordinator {
|
||||
conversationId: string,
|
||||
anchorMessageId: string | null,
|
||||
): Promise<void> {
|
||||
const candidates = await this.store.listCandidates(channelAccountId, conversationId);
|
||||
const candidates = await this.store.listCandidates(
|
||||
channelAccountId,
|
||||
conversationId,
|
||||
"awaiting_anchor",
|
||||
);
|
||||
const anchorCandidate = anchorMessageId
|
||||
? candidates.find((candidate) => candidate.messageId === anchorMessageId)
|
||||
? await this.store.getCandidate(channelAccountId, conversationId, anchorMessageId)
|
||||
: undefined;
|
||||
const anchorTime = anchorCandidate ? candidateTimestamp(anchorCandidate) : null;
|
||||
|
||||
@@ -295,9 +291,9 @@ export class AckCompletionCoordinator {
|
||||
});
|
||||
return;
|
||||
}
|
||||
const candidates = (await this.store.listCandidates(channelAccountId, conversationId))
|
||||
.filter(pendingCandidate)
|
||||
.sort(candidateOrder);
|
||||
const candidates = (
|
||||
await this.store.listCandidates(channelAccountId, conversationId, "pending_ack")
|
||||
).sort(candidateOrder);
|
||||
for (const candidate of candidates) {
|
||||
if (this.lifecycle.isDisposed()) return;
|
||||
if (!this.bright.isOnline()) return;
|
||||
@@ -327,17 +323,16 @@ export class AckCompletionCoordinator {
|
||||
if (!this.lifecycle.getHistoryGeneration(conversationId)) return;
|
||||
const checkpoint = await this.checkpoints.getCheckpoint(channelAccountId, conversationId);
|
||||
if (!checkpoint || !checkpoint.historyComplete || checkpoint.phase !== "uploading") return;
|
||||
const candidates = await this.store.listCandidates(channelAccountId, conversationId);
|
||||
if (candidates.some(pendingCandidate)) return;
|
||||
const explicitCompletion = candidates.some((candidate) =>
|
||||
this.explicitlySentCandidates.has(candidate.key),
|
||||
if (await this.store.hasPendingCandidates(channelAccountId, conversationId)) return;
|
||||
const explicitCompletion = this.explicitlySentConversations.has(
|
||||
scopedKey(channelAccountId, conversationId),
|
||||
);
|
||||
if (!this.lifecycle.isBusinessFrameGateOpen() && !explicitStart && !explicitCompletion) {
|
||||
return;
|
||||
}
|
||||
if (!this.bright.isOnline()) return;
|
||||
if (checkpoint.completionSent && !forceResend) return;
|
||||
const latestMessageId = latestConfirmedMessageId(candidates) ?? checkpoint.anchorMessageId;
|
||||
const latestMessageId = checkpoint.latestMessageId ?? checkpoint.anchorMessageId;
|
||||
const reportAnchorNotFound =
|
||||
checkpoint.anomalyCode !== undefined &&
|
||||
!(checkpoint.mode === "full" && checkpoint.anchorState === "not_found");
|
||||
@@ -401,8 +396,7 @@ export class AckCompletionCoordinator {
|
||||
): Promise<void> {
|
||||
let candidate: OneTalkSyncCandidate | null = null;
|
||||
if (requestKey) {
|
||||
const all = await this.store.listPendingCandidates(this.scope.channelAccountId);
|
||||
candidate = all.find((item) => item.key === requestKey) ?? null;
|
||||
candidate = await this.store.getCandidateByKey(requestKey);
|
||||
}
|
||||
if (
|
||||
!candidate &&
|
||||
@@ -415,7 +409,7 @@ export class AckCompletionCoordinator {
|
||||
result.messageId,
|
||||
);
|
||||
}
|
||||
if (!candidate) return;
|
||||
if (!candidate || candidate.status !== "pending_ack") return;
|
||||
const nextStatus =
|
||||
result.status === "accepted" || result.status === "duplicate"
|
||||
? "confirmed"
|
||||
@@ -423,18 +417,9 @@ export class AckCompletionCoordinator {
|
||||
? "anomaly"
|
||||
: "rejected";
|
||||
await this.queue.enqueue(candidate.channelAccountId, candidate.conversationId, async () => {
|
||||
const durableCandidate = await this.store.getCandidate(
|
||||
candidate.channelAccountId,
|
||||
candidate.conversationId,
|
||||
candidate.messageId,
|
||||
);
|
||||
if (!durableCandidate) return;
|
||||
const durableStatus =
|
||||
durableCandidate.status === "confirmed" ||
|
||||
durableCandidate.status === "anomaly" ||
|
||||
durableCandidate.status === "rejected"
|
||||
? durableCandidate.status
|
||||
: nextStatus;
|
||||
const durableCandidate = await this.store.getCandidateByKey(candidate.key);
|
||||
if (!durableCandidate || durableCandidate.status !== "pending_ack") return;
|
||||
const durableStatus = nextStatus;
|
||||
await this.store.updateCandidate(durableCandidate, durableStatus);
|
||||
traceOneTalkObservedMessages([durableCandidate.message], {
|
||||
stage: "bright_ack",
|
||||
@@ -467,13 +452,23 @@ export class AckCompletionCoordinator {
|
||||
if (checkpointForAck !== checkpoint) {
|
||||
await this.store.putCheckpoint(checkpointForAck);
|
||||
}
|
||||
const candidates = await this.store.listCandidates(
|
||||
candidate.channelAccountId,
|
||||
candidate.conversationId,
|
||||
);
|
||||
const previousLatest = checkpoint.latestMessageId
|
||||
? await this.store.getCandidate(
|
||||
candidate.channelAccountId,
|
||||
candidate.conversationId,
|
||||
checkpoint.latestMessageId,
|
||||
)
|
||||
: null;
|
||||
const latestConfirmed = [
|
||||
previousLatest?.status === "confirmed" ? previousLatest : null,
|
||||
durableStatus === "confirmed" ? durableCandidate : null,
|
||||
]
|
||||
.filter((item): item is OneTalkSyncCandidate => item !== null)
|
||||
.sort(candidateOrder)
|
||||
.at(-1);
|
||||
await this.checkpoints.checkpointLatestMessageId(
|
||||
checkpointForAck,
|
||||
latestConfirmedMessageId(candidates),
|
||||
latestConfirmed?.messageId ?? null,
|
||||
);
|
||||
}
|
||||
await this.tryComplete(candidate.channelAccountId, candidate.conversationId);
|
||||
@@ -582,7 +577,9 @@ export class AckCompletionCoordinator {
|
||||
);
|
||||
if (sent) {
|
||||
if (explicitStart)
|
||||
candidates.forEach((candidate) => this.explicitlySentCandidates.add(candidate.key));
|
||||
this.explicitlySentConversations.add(
|
||||
scopedKey(first.channelAccountId, first.conversationId),
|
||||
);
|
||||
return true;
|
||||
}
|
||||
this.lifecycle.emitDiagnostic({
|
||||
|
||||
@@ -32,6 +32,10 @@ const profile = (
|
||||
observationStatus: "confirmed",
|
||||
});
|
||||
|
||||
globalThis.IDBKeyRange = {
|
||||
bound: (lower, upper) => ({ lower, upper }),
|
||||
};
|
||||
|
||||
class Request {
|
||||
constructor(result, schedule = true) {
|
||||
this.result = result;
|
||||
@@ -47,6 +51,11 @@ class Store {
|
||||
constructor(name) {
|
||||
this.name = name;
|
||||
this.records = new Map();
|
||||
this.objectStoreGetAllCalls = 0;
|
||||
this.indexNames = {
|
||||
values: new Set(),
|
||||
contains: (indexName) => this.indexNames.values.has(indexName),
|
||||
};
|
||||
}
|
||||
|
||||
get(key) {
|
||||
@@ -54,9 +63,40 @@ class Store {
|
||||
}
|
||||
|
||||
getAll() {
|
||||
this.objectStoreGetAllCalls += 1;
|
||||
return new Request([...this.records.values()]);
|
||||
}
|
||||
|
||||
createIndex(name, keyPath) {
|
||||
this.indexNames.values.add(name);
|
||||
this.indexes ??= new Map();
|
||||
this.indexes.set(name, keyPath);
|
||||
}
|
||||
|
||||
index(name) {
|
||||
const keyPath = this.indexes?.get(name);
|
||||
if (keyPath === undefined) throw new Error(`Missing index: ${name}`);
|
||||
const keyFor = (record) => {
|
||||
const valueFor = (path) =>
|
||||
path.split(".").reduce((value, segment) => value?.[segment], record);
|
||||
return Array.isArray(keyPath) ? keyPath.map(valueFor) : valueFor(keyPath);
|
||||
};
|
||||
const matches = (record, query) => {
|
||||
const key = keyFor(record);
|
||||
if (query?.lower !== undefined && Array.isArray(key)) {
|
||||
return key[0] === query.lower[0] && key.every((value) => value !== undefined);
|
||||
}
|
||||
return JSON.stringify(key) === JSON.stringify(query);
|
||||
};
|
||||
const matchingEntries = (query) =>
|
||||
[...this.records.entries()].filter(([, record]) => matches(record, query));
|
||||
return {
|
||||
getAll: (query) => new Request(matchingEntries(query).map(([, record]) => record)),
|
||||
getAllKeys: (query) => new Request(matchingEntries(query).map(([key]) => key)),
|
||||
count: (query) => new Request(matchingEntries(query).length),
|
||||
};
|
||||
}
|
||||
|
||||
put(record) {
|
||||
this.transaction.stage(this.records, record.key, record);
|
||||
}
|
||||
@@ -230,11 +270,44 @@ test("resets legacy OneTalk stores before v7 while opening the current database
|
||||
});
|
||||
|
||||
test("reports whether an account has any durable profile ledger record", async () => {
|
||||
const store = createOneTalkContactProfileStore(new Factory(), () => 100);
|
||||
const factory = new Factory();
|
||||
const store = createOneTalkContactProfileStore(factory, () => 100);
|
||||
assert.equal(await store.hasProfileRecord("account-1"), false);
|
||||
await store.putPendingProfile("account-1", profile());
|
||||
assert.equal(await store.hasProfileRecord("account-1"), true);
|
||||
assert.equal(await store.hasProfileRecord("account-2"), false);
|
||||
assert.equal(
|
||||
factory.database.stores.get(ONE_TALK_CONTACT_PROFILE_STORE_NAME).objectStoreGetAllCalls,
|
||||
0,
|
||||
);
|
||||
});
|
||||
|
||||
test("lists only an account's pending profiles through the pending-observed index", async () => {
|
||||
const factory = new Factory();
|
||||
const store = createOneTalkContactProfileStore(factory, () => 100);
|
||||
const pending = profile("pending-account-1", null, 100, "conversation-1");
|
||||
const uploaded = profile("uploaded-account-1", null, 101, "conversation-2");
|
||||
const otherAccount = profile("pending-account-2", null, 102, "conversation-3");
|
||||
|
||||
await store.putPendingProfile("account-1", pending);
|
||||
await store.putPendingProfile("account-1", uploaded);
|
||||
await store.markProfileUploaded({
|
||||
channelAccountId: "account-1",
|
||||
conversationId: uploaded.conversationId,
|
||||
fingerprint: uploaded.profileFingerprint,
|
||||
observedAtMs: uploaded.observedAtMs,
|
||||
uploadedAt: 200,
|
||||
});
|
||||
await store.putPendingProfile("account-2", otherAccount);
|
||||
|
||||
assert.deepEqual(
|
||||
(await store.listPendingProfiles("account-1")).map((record) => record.key),
|
||||
[contactProfileKey("account-1", pending.conversationId)],
|
||||
);
|
||||
assert.equal(
|
||||
factory.database.stores.get(ONE_TALK_CONTACT_PROFILE_STORE_NAME).objectStoreGetAllCalls,
|
||||
0,
|
||||
);
|
||||
});
|
||||
|
||||
test("does not return true before the readwrite transaction commits", async () => {
|
||||
|
||||
@@ -35,6 +35,10 @@ class FakeRequest {
|
||||
class FakeStore {
|
||||
constructor() {
|
||||
this.records = new Map();
|
||||
this.indexNames = {
|
||||
values: new Set(),
|
||||
contains: (name) => this.indexNames.values.has(name),
|
||||
};
|
||||
}
|
||||
|
||||
put(record) {
|
||||
@@ -49,6 +53,10 @@ class FakeStore {
|
||||
return new FakeRequest([...this.records.values()].map((record) => structuredClone(record)));
|
||||
}
|
||||
|
||||
createIndex(name) {
|
||||
this.indexNames.values.add(name);
|
||||
}
|
||||
|
||||
clear() {
|
||||
this.records.clear();
|
||||
}
|
||||
|
||||
@@ -56,6 +56,7 @@ class MemoryStore {
|
||||
this.checkpoints = new Map();
|
||||
this.candidates = new Map();
|
||||
this.anomalies = new Map();
|
||||
this.listPendingCandidateCalls = 0;
|
||||
}
|
||||
|
||||
async putBatch() {}
|
||||
@@ -137,15 +138,21 @@ class MemoryStore {
|
||||
);
|
||||
}
|
||||
|
||||
async listCandidates(channelAccountId, conversationId) {
|
||||
async getCandidateByKey(key) {
|
||||
return this.candidates.get(key) ?? null;
|
||||
}
|
||||
|
||||
async listCandidates(channelAccountId, conversationId, status) {
|
||||
return [...this.candidates.values()].filter(
|
||||
(candidate) =>
|
||||
candidate.channelAccountId === channelAccountId &&
|
||||
candidate.conversationId === conversationId,
|
||||
candidate.conversationId === conversationId &&
|
||||
(status === undefined || candidate.status === status),
|
||||
);
|
||||
}
|
||||
|
||||
async listPendingCandidates(channelAccountId) {
|
||||
this.listPendingCandidateCalls += 1;
|
||||
return [...this.candidates.values()].filter(
|
||||
(candidate) =>
|
||||
candidate.status === "pending_ack" &&
|
||||
@@ -153,6 +160,15 @@ class MemoryStore {
|
||||
);
|
||||
}
|
||||
|
||||
async hasPendingCandidates(channelAccountId, conversationId) {
|
||||
return [...this.candidates.values()].some(
|
||||
(candidate) =>
|
||||
candidate.channelAccountId === channelAccountId &&
|
||||
candidate.conversationId === conversationId &&
|
||||
candidate.status === "pending_ack",
|
||||
);
|
||||
}
|
||||
|
||||
async putCandidate(candidate) {
|
||||
const { requestId: _requestId, ...durableCandidate } = candidate;
|
||||
this.candidates.set(candidate.key, { ...durableCandidate });
|
||||
@@ -553,6 +569,150 @@ test("ignores an ACK with a stale generation even when its request has no echoed
|
||||
);
|
||||
});
|
||||
|
||||
test("resolves a request-keyed ACK without enumerating account pending candidates", async () => {
|
||||
const store = new MemoryStore();
|
||||
const bright = new FakeBright();
|
||||
const engine = createOneTalkSyncEngine({
|
||||
scope,
|
||||
bright,
|
||||
store,
|
||||
pageRuntime: { routePageCommand: async () => ({ status: "completed" }) },
|
||||
now: () => 500,
|
||||
createRequestId: (kind) => `${kind}-request-keyed-ack`,
|
||||
});
|
||||
authenticate(bright, [{ conversationId: "conversation-1", latestMessageId: null }]);
|
||||
|
||||
await engine.ingestObservedBatch({
|
||||
channelAccountId: scope.channelAccountId,
|
||||
conversationId: "conversation-1",
|
||||
observationSource: "live",
|
||||
messages: [rawMessage("request-keyed-ack", 500)],
|
||||
});
|
||||
await waitFor(() => bright.messages.length === 1, "candidate was not sent");
|
||||
store.listPendingCandidateCalls = 0;
|
||||
|
||||
bright.emit({
|
||||
type: "message.ack",
|
||||
requestId: bright.messages[0].requestId,
|
||||
payload: {
|
||||
status: "accepted",
|
||||
conversationId: "conversation-1",
|
||||
messageId: "request-keyed-ack",
|
||||
},
|
||||
});
|
||||
await tick();
|
||||
await tick();
|
||||
|
||||
assert.equal(
|
||||
(await store.getCandidate(scope.channelAccountId, "conversation-1", "request-keyed-ack"))
|
||||
.status,
|
||||
"confirmed",
|
||||
);
|
||||
assert.equal(store.listPendingCandidateCalls, 0);
|
||||
});
|
||||
|
||||
test("resolves every result in a batch ACK by its request key", async () => {
|
||||
const store = new MemoryStore();
|
||||
const bright = new FakeBright();
|
||||
const engine = createOneTalkSyncEngine({
|
||||
scope,
|
||||
bright,
|
||||
store,
|
||||
pageRuntime: { routePageCommand: async () => ({ status: "completed" }) },
|
||||
now: () => 500,
|
||||
createRequestId: (kind) => `${kind}-batch-ack`,
|
||||
});
|
||||
authenticate(bright, [{ conversationId: "conversation-1", latestMessageId: null }]);
|
||||
|
||||
await engine.ingestObservedBatch({
|
||||
channelAccountId: scope.channelAccountId,
|
||||
conversationId: "conversation-1",
|
||||
observationSource: "history",
|
||||
mode: "full",
|
||||
messages: [rawMessage("batch-first", 100), rawMessage("batch-second", 200)],
|
||||
});
|
||||
await waitFor(() => bright.batches.length === 1, "candidate batch was not sent");
|
||||
store.listPendingCandidateCalls = 0;
|
||||
|
||||
bright.emit({
|
||||
type: "messages.ack",
|
||||
requestId: bright.batches[0].requestId,
|
||||
payload: {
|
||||
results: [
|
||||
{
|
||||
status: "accepted",
|
||||
conversationId: "conversation-1",
|
||||
messageId: "batch-first",
|
||||
},
|
||||
{
|
||||
status: "duplicate",
|
||||
conversationId: "conversation-1",
|
||||
messageId: "batch-second",
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
await tick();
|
||||
await tick();
|
||||
|
||||
assert.equal(
|
||||
(await store.getCandidate(scope.channelAccountId, "conversation-1", "batch-first")).status,
|
||||
"confirmed",
|
||||
);
|
||||
assert.equal(
|
||||
(await store.getCandidate(scope.channelAccountId, "conversation-1", "batch-second")).status,
|
||||
"confirmed",
|
||||
);
|
||||
assert.equal(store.listPendingCandidateCalls, 0);
|
||||
});
|
||||
|
||||
test("keeps the candidate-order maximum checkpoint after out-of-order ACKs", async () => {
|
||||
const store = new MemoryStore();
|
||||
const bright = new FakeBright();
|
||||
const engine = createOneTalkSyncEngine({
|
||||
scope,
|
||||
bright,
|
||||
store,
|
||||
pageRuntime: { routePageCommand: async () => ({ status: "completed" }) },
|
||||
now: () => 500,
|
||||
createRequestId: (kind) => `${kind}-${bright.messages.length}`,
|
||||
});
|
||||
authenticate(bright, [{ conversationId: "conversation-1", latestMessageId: null }]);
|
||||
|
||||
await engine.ingestObservedBatch({
|
||||
channelAccountId: scope.channelAccountId,
|
||||
conversationId: "conversation-1",
|
||||
observationSource: "live",
|
||||
messages: [rawMessage("first", 100)],
|
||||
});
|
||||
await engine.ingestObservedBatch({
|
||||
channelAccountId: scope.channelAccountId,
|
||||
conversationId: "conversation-1",
|
||||
observationSource: "live",
|
||||
messages: [rawMessage("second", 200)],
|
||||
});
|
||||
await waitFor(() => bright.messages.length === 2, "candidates were not sent");
|
||||
|
||||
for (const message of [...bright.messages].reverse()) {
|
||||
bright.emit({
|
||||
type: "message.ack",
|
||||
requestId: message.requestId,
|
||||
payload: {
|
||||
status: "accepted",
|
||||
conversationId: "conversation-1",
|
||||
messageId: message.message.messageId,
|
||||
},
|
||||
});
|
||||
await tick();
|
||||
await tick();
|
||||
}
|
||||
|
||||
assert.equal(
|
||||
(await store.getCheckpoint(scope.channelAccountId, "conversation-1")).latestMessageId,
|
||||
"second",
|
||||
);
|
||||
});
|
||||
|
||||
test("does not replace an anchor generation with a late discovery generation", () => {
|
||||
const bright = new FakeBright();
|
||||
const lifecycle = new OneTalkSyncLifecycle(
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
ONE_TALK_CHECKPOINT_STORE_NAME,
|
||||
ONE_TALK_CONTACT_PROFILE_STORE_NAME,
|
||||
ONE_TALK_CONVERSATION_BOOTSTRAP_STORE_NAME,
|
||||
ONE_TALK_SYNC_DATABASE_VERSION,
|
||||
ONE_TALK_MESSAGE_STORE_NAME,
|
||||
ONE_TALK_RENDERED_CARD_LEDGER_STORE_NAME,
|
||||
createOneTalkRenderedCardLedgerStore,
|
||||
@@ -43,6 +44,11 @@ class FakeRequest {
|
||||
class FakeStore {
|
||||
constructor() {
|
||||
this.records = new Map();
|
||||
this.objectStoreGetAllCalls = 0;
|
||||
this.indexNames = {
|
||||
values: new Set(),
|
||||
contains: (name) => this.indexNames.values.has(name),
|
||||
};
|
||||
}
|
||||
|
||||
put(record) {
|
||||
@@ -54,9 +60,43 @@ class FakeStore {
|
||||
}
|
||||
|
||||
getAll() {
|
||||
this.objectStoreGetAllCalls += 1;
|
||||
return new FakeRequest([...this.records.values()].map((record) => structuredClone(record)));
|
||||
}
|
||||
|
||||
createIndex(name, keyPath) {
|
||||
this.indexNames.values.add(name);
|
||||
this.indexes ??= new Map();
|
||||
this.indexes.set(name, keyPath);
|
||||
}
|
||||
|
||||
index(name) {
|
||||
const keyPath = this.indexes?.get(name);
|
||||
if (keyPath === undefined) throw new Error(`Missing index: ${name}`);
|
||||
const keyFor = (record) => {
|
||||
const valueFor = (path) =>
|
||||
path.split(".").reduce((value, segment) => value?.[segment], record);
|
||||
return Array.isArray(keyPath) ? keyPath.map(valueFor) : valueFor(keyPath);
|
||||
};
|
||||
const matches = (record, query) => {
|
||||
const key = keyFor(record);
|
||||
if (query?.lower !== undefined && Array.isArray(key)) {
|
||||
return key[0] === query.lower[0];
|
||||
}
|
||||
return JSON.stringify(key) === JSON.stringify(query);
|
||||
};
|
||||
const matchingEntries = (query) =>
|
||||
[...this.records.entries()].filter(([, record]) => matches(record, query));
|
||||
return {
|
||||
getAll: (query) =>
|
||||
new FakeRequest(
|
||||
matchingEntries(query).map(([, record]) => structuredClone(record)),
|
||||
),
|
||||
getAllKeys: (query) => new FakeRequest(matchingEntries(query).map(([key]) => key)),
|
||||
count: (query) => new FakeRequest(matchingEntries(query).length),
|
||||
};
|
||||
}
|
||||
|
||||
clear() {
|
||||
this.records.clear();
|
||||
}
|
||||
@@ -140,6 +180,7 @@ class FakeFactory {
|
||||
ONE_TALK_CHECKPOINT_STORE_NAME,
|
||||
ONE_TALK_ANOMALY_STORE_NAME,
|
||||
ONE_TALK_CONTACT_PROFILE_STORE_NAME,
|
||||
ONE_TALK_CONVERSATION_BOOTSTRAP_STORE_NAME,
|
||||
]);
|
||||
request.transaction = transaction;
|
||||
transaction.oncomplete = () => request.onsuccess?.();
|
||||
@@ -280,6 +321,46 @@ test("preserves existing stores while adding new stores from v7 and v8", async (
|
||||
}
|
||||
});
|
||||
|
||||
test("preserves v8 ledger records while creating v9 query indexes", async () => {
|
||||
const factory = new FakeFactory(8);
|
||||
for (const storeName of [
|
||||
ONE_TALK_MESSAGE_STORE_NAME,
|
||||
ONE_TALK_CANDIDATE_STORE_NAME,
|
||||
ONE_TALK_CHECKPOINT_STORE_NAME,
|
||||
ONE_TALK_ANOMALY_STORE_NAME,
|
||||
ONE_TALK_CONTACT_PROFILE_STORE_NAME,
|
||||
ONE_TALK_CONVERSATION_BOOTSTRAP_STORE_NAME,
|
||||
]) {
|
||||
factory.database.createObjectStore(storeName).put({ key: storeName, v8: true });
|
||||
}
|
||||
|
||||
const store = createOneTalkSyncStore(factory, () => 500);
|
||||
await store.listCheckpoints("account-1");
|
||||
|
||||
assert.equal(factory.database.version, ONE_TALK_SYNC_DATABASE_VERSION);
|
||||
for (const storeName of factory.database.stores.keys()) {
|
||||
assert.equal(factory.database.stores.get(storeName).records.get(storeName).v8, true);
|
||||
}
|
||||
assert.equal(
|
||||
factory.database.stores
|
||||
.get(ONE_TALK_CANDIDATE_STORE_NAME)
|
||||
.indexNames.contains("by_account_status"),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
factory.database.stores
|
||||
.get(ONE_TALK_CONTACT_PROFILE_STORE_NAME)
|
||||
.indexNames.contains("by_account_pending_observed_at"),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
factory.database.stores
|
||||
.get(ONE_TALK_ANOMALY_STORE_NAME)
|
||||
.indexNames.contains("by_account_conversation"),
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
test("clears only one conversation history ledger after its transaction commits", async () => {
|
||||
const factory = new FakeFactory();
|
||||
const store = createOneTalkSyncStore(factory, () => 500);
|
||||
@@ -357,7 +438,16 @@ test("clears only one conversation history ledger after its transaction commits"
|
||||
assert.deepEqual(await store.listAnomalies("account-1", "conversation-1"), []);
|
||||
assert.notEqual(await store.getCheckpoint("account-1", "conversation-2"), null);
|
||||
assert.equal((await store.listCandidates("account-1", "conversation-2")).length, 1);
|
||||
assert.equal((await store.listPendingCandidates("account-1")).length, 1);
|
||||
assert.equal((await store.listAnomalies("account-1")).length, 1);
|
||||
for (const storeName of [
|
||||
ONE_TALK_MESSAGE_STORE_NAME,
|
||||
ONE_TALK_CHECKPOINT_STORE_NAME,
|
||||
ONE_TALK_CANDIDATE_STORE_NAME,
|
||||
ONE_TALK_ANOMALY_STORE_NAME,
|
||||
]) {
|
||||
assert.equal(factory.database.stores.get(storeName).objectStoreGetAllCalls, 0);
|
||||
}
|
||||
});
|
||||
|
||||
test("stores bootstrap markers by account and stable migration identifier", async () => {
|
||||
|
||||
Reference in New Issue
Block a user