tracker

15.9 KB

1. # STATUS — tracker.worldapi.org
3. ## 2026-09-27 — ticket #2: old data migrated (took over a previous session's rewritten tool)
5. Continued from an earlier session that had rewritten `tools/migrate.hl` to avoid a real data-loss
6. bug (see below) but never run it even once, and had left the real `storage/mpackdb/` holding data
7. from the OLDER, buggy two-pass version of the tool (put a bare record, then `update()` it once the
8. other side of a circular reference existed). Took over: moved that buggy data aside
9. (`.scratch/pre-migrate-buggy-backup`, since removed), kept `storage/mpackdb/users.db` (step 1's
10. login table, untouched by the migration), and ran the already-fixed tool for the first time.
12. **The fix already on disk (kept as is):** `Show.seasons`/`Season.show` and `Show.cast`/
13. `Person.shows` are two-way references, so the id on one side must exist before the record on the
14. other side can be built. The old approach — `put()` a bare record, come back with `update()` once
15. the far side's id exists — loses rows: reopening a table in a fresh process after an `update()` had
16. touched it read back FEWER rows than were ever `put()` (seen directly: `persons.db`'s `count()`
17. dropped 15157 → 15152 after one round of `update()` — a real `hl:mpackdb` bug, reported to Hybriel,
18. not something an app is allowed to patch around). The fix assigns every new id itself (8 random
19. bytes, `hl:crypto`) in one pass over each export file BEFORE building any record, so by the time a
20. record is actually built every reference is already a known id — one `put()` per row, `update()`
21. never called.
23. **Run** (`tools/migrate.hl`, `TRACKER_OLD_DATA=.../old-tracker/mongo-export`,
24. `TRACKER_STORAGE=$PWD/storage/mpackdb`):
26. ```
27. migrate: genres 27 new, 0 already there (old 27 -> new 27)
28. migrate: persons 15157 new, 0 already there (old 15157 -> new 15157), show refs skipped 0
29. migrate: shows 9453 new, 0 already there (old 9453 -> new 9453), cast refs skipped 0, genre refs skipped 0, season refs skipped 0
30. migrate: seasons 6430 new, 0 already there (old 6430 -> new 6430), show refs skipped 0, episode refs skipped 0
31. migrate: episodes 231584 new, 0 already there (old 231584 -> new 231584), show refs skipped 0, season refs left null (pre-existing) 1
32. migrate: follows 128 new, 0 already there (old 128 -> new 128), show refs skipped 0
33. migrate: watches 11692 new, 0 already there (old 11692 -> new 11692), target refs left null (pre-existing) 3
34. migrate: 0 failed
35. ```
37. Every count equals the old export's own document count (`old-tracker/README-RECONSTRUCTED.md`:
38. Show 9453, Season 6430, Episode 231584, Person 15157, Genre 27, Follow 128, Watch 11692). Ran a
39. second time straight after (idempotency: the tool's `oldId` index finds each row again) — every
40. table printed `0 new`, all rows `already there`, counts unchanged, 0 failed: the id-loss bug does
41. not resurface across a real close-and-reopen.
43. **Proof beyond the tool's own counters** (`tools/verify.hl`, new): the ticket asks for a check that
44. does not just trust `migrate.hl`'s own "skipped" tallies. It runs against a plain filesystem COPY
45. of `storage/mpackdb/` (hl:mpackdb rewrites a file's data on open, so the live store is never opened
46. a second time) and independently re-derives, from the persisted records alone: every table's
47. `count()` against the export's own document count, and — by building id sets for every table and
48. walking every reference field (`Show.genres/cast/seasons`, `Season.show/episodes`,
49. `Episode.show/season`, `Person.shows`, `Follow.show/user`, `Watch.target/user`) — that every
50. non-null reference resolves to a real id. It also follows one show end to end (`Raised by Wolves`,
51. 2 seasons, 18 episodes, 1 follow, 2 season-level watches — all cross-checked back to the show).
53. ```
54. $ cp -r storage/mpackdb .scratch/verify-copy && TRACKER_VERIFY_COPY=$PWD/.scratch/verify-copy ./bin/hybriel tools/verify.hl
55. count ok genres: 27 (expected 27)
56. count ok persons: 15157 (expected 15157)
57. count ok shows: 9453 (expected 9453)
58. count ok seasons: 6430 (expected 6430)
59. count ok episodes: 231584 (expected 231584)
60. count ok follows: 128 (expected 128)
61. count ok watches: 11692 (expected 11692)
62. users: 1 (expected 1, the creator)
63. dangling references: 0 (episodes with a pre-existing null season: 36194, watches with a pre-existing null target: 3 — not dangling, the export already had no target)
64. END-TO-END show 105ad9c91b14d663 "Raised by Wolves" seasons=2
65. season 1 (be2965a5ab7165ca) episodes=10 show-back-ref-ok=true
66. season 2 (0fe4e53157ed0367) episodes=8 show-back-ref-ok=true
67. total episodes across seasons: 18
68. follows on this show: 1, season-level watches touching it: 2
69. VERIFY PASS
70. ```
72. (36194 episodes with no season = 36193 that never had one in the old export, plus the 1 the tool's
73. own count already named as a pre-existing dangling `season` reference in the source data — neither
74. is a reference this migration broke; `dangling references: 0` covers exactly that.)
76. The single old user (`User.jsonl`, `[email protected]`, password not taken over) is this app's
77. user for ident short id `az5b2` (`storage/mpackdb/users.db`, the same table step 1's login uses) —
78. confirmed in the copy: one record, `identity=az5b2`.
80. **Two Hybriel bugs found while building this** (both already filed as issues on this ticket by an
81. earlier session, not re-filed): `hl:fs readFile` silently truncates at 10 MiB (worked around here
82. with `split`, no shell, no JS, see `tools/migrate.hl`'s header comment) and `hl:mpackdb` loses rows
83. across `update()` + reopen (the reason this tool never calls `update()`).
85. **Open**: nothing shown yet — this ticket is data-only, on purpose (the next step, from the
86. creator, is showing the data in the UI).
88. ## 2026-09-27 — ticket #3: no border on the header or filled buttons
90. Design notes from the first test (architect): "the application-header should have no bottom
91. border" and "the buttons also no border except they are inverted". Two changes in
92. `styles.hl`:
94. - `applicationHeader`: dropped `borderBottom`.
95. - `ident-selector::part(button)` (the signed-out "choose ident" button): added
96. `border = 'none'` — the element's own CSS gave it a 1px border the same colour as its
97. background (`--_accent`), invisible but still a real border in the computed style; this is
98. the app-side restyling hook ident's README documents (`::part(button)`), not a change to
99. ident's vendored `selector.js`.
101. Left unchanged, already correct: the native `button` rule (filled, e.g. no other filled
102. buttons on this page yet) already has `border = '0'`; `button.quiet` ("Log out") and the
103. selector's dropdown `[part="identity"]` rows keep their border (transparent background —
104. inverted style); `a.button` ("Log in with ident") has no border rule and browsers give `<a>`
105. none by default. The selector's signed-in "logged in with ident" status pill
106. (`::part(status)`) is not one of the buttons the ticket names and isn't a button element
107. (a `<span>`) — left with its border.
109. **Gate** (`tests/browser.mjs`, now 16 checks): added computed-style checks — the header's
110. `borderBottomWidth` is `0px`; `#loginbutton`'s border is `0px`; the selector's
111. `[part~="button"]` (inside its shadow root) border is `0px`; `#logout`'s border is NOT `0px`
112. (still inverted-bordered) once signed in.
114. ```
115. node tests/browser.mjs
116. 16 passed, 0 failed
117. ```
119. ## 2026-09-27 — ticket #1 reopened: login redone to match calendar exactly (header selector, empty page)
121. The architect rejected the first build of ticket #1: the sign-in sat centered on the page
122. (its own "Sign in with ident" card, identity + sign-out shown in a `homeCard`) instead of
123. the header identity selector calendar/gitoria use, and there was no `<ident-selector>` at
124. all. Rebuilt the login by copying calendar.worldapi.org's own files (unchanged in shape, per
125. the architect's instruction "Copy calendar.worldapi.org's login unchanged"):
127. - **`users.hl`** (new): the ident exchange + a `usersTable` (`storage/mpackdb/users.db`,
128. `identity` → this app's user id), copied from calendar's `users.hl` with `TRACKER_KEY` /
129. `TRACKER_SECRET` / `TRACKER_URL` / `TRACKER_STORAGE` in place of calendar's names (kept
130. the names already in this app's `docker-compose.yml`/README/DECISIONS rather than
131. switching to calendar's generic `IDENT_API_KEY`/`IDENT_API_SECRET`).
132. - **`login.js`** (new): calendar's bridge between `<ident-selector>`'s `ident-login` DOM
133. event and the hidden `#identcode` input, copied verbatim (creator: "login.js is correct,
134. as thats for externals in general").
135. - **`components/main.hl`**: now the header carries `userBox` (`<ident-selector>` + "Log in
136. with ident" / "Log out"), the `trackerLogin`/`trackerLogOut` faces and the
137. `trackerSignedIn`/`trackerSignedOut` client push — calendar's shell renamed to this app's
138. event names.
139. - **`components/home.hl`**: now truly empty (`View { home {} }`) — no sign-in link, no
140. identity/sign-out duplicated on the page; that is entirely the header's job now.
141. - **`components/loginfailed.hl`** (new) + **`project.hl`**: the login routes are now
142. `/login/callback` (function route, the button's return AND the code exchange) and
143. `/login/failed`, copied from calendar's `project.hl` (`safePath`, `failed()`,
144. `loginCallback`) in place of the old `/login` + `/callback` + `/logout` routes; the old
145. `/logout` POST route is gone (logout is a face, like calendar's).
146. - **`styles.hl`**: added the header selector styles (`userBox`, `identSelector`,
147. `ident-selector::part(...)`, sticky header) from calendar's `styles.hl`; dropped the
148. now-unused `homeCard`/`accountBar`/`userName` rules from the rejected centered sign-in.
150. **Gate rewritten** (`tests/browser.mjs`, 12 checks): replaced the old gate (which drove
151. ident's full email-code UI through a vendored dev copy of ident at `.scratch/ident-dev`)
152. with calendar's own pattern — `tests/identkit.mjs` (copied from calendar unchanged) starts a
153. throwaway ident (a fresh copy of `/media/STORAGE/projects/ident.worldapi.org`'s code, no
154. `.env`/storage copied, codes to a mail sink, never the live ident), signs in over its REST
155. API and registers this app, then the gate proves the header in a real headless Chrome:
156. signed out → `#selector` + `#loginbutton` in the header, **`main` is empty** → sign in
157. (`/login/callback?ident_code=`, the same exchange the selector would trigger) → `#selector`
158. shows `class="in"`, `#logout` appears, **`main` still empty** → sign out → signed out again
159. → a reload stays signed out. No console errors.
161. ```
162. node tests/browser.mjs
163. 12 passed, 0 failed
164. ```
166. `./deploy.sh --dry-run --target .scratch/deploy-preview --url http://127.0.0.1:0/`: gate
167. passed as step [1/4], rsync preview held none of `storage/`, `.sessions/`, `.env`,
168. `.scratch/`, `server.*`, logs. Removed the stale `.scratch/ident-dev` (the old gate's vendored
169. ident copy) and `.scratch/browser-gate` (the old gate's storage) — unused by the new gate.
171. **Still open**: the app is registered in ident's dev copies only (this gate's own throwaway
172. ident). The LIVE ident registration (`TRACKER_KEY`/`TRACKER_SECRET` in `.env` next to
173. `docker-compose.yml`) is the architect's first-deploy step (README "Deploy"), not built here.
175. ## 2026-09-27 — ticket #2: data migration — blocked twice on sandbox access, nothing done
177. Two workers in a row (`s-20260927T1029-f57b0f`, then this session) could not start: the
178. migration source the ticket names, `/media/STORAGE/projects/old-tracker/mongo-export/*.jsonl`
179. + its `README-RECONSTRUCTED.md`, is not reachable from this worker's sandbox. Verified fresh
180. this session:
182. ```
183. $ ls -la /media/STORAGE/projects/
184. antcolony-docs hybriel ident.worldapi.org tracker.worldapi.org # no old-tracker
185. $ mount | grep STORAGE
186. /dev/nvme1n1p6 on /media/STORAGE/projects/tracker.worldapi.org type btrfs (rw,...)
187. /dev/nvme1n1p6 on /media/STORAGE/projects/hybriel type btrfs (ro,...)
188. /dev/nvme1n1p6 on /media/STORAGE/projects/ident.worldapi.org type btrfs (ro,...)
189. /dev/nvme1n1p6 on /media/STORAGE/projects/antcolony-docs type btrfs (ro,...)
190. # old-tracker is not among the mounted subvolumes at all
191. ```
193. So this is not a data problem (the export is "complete" per the ticket) — it's the worker
194. sandbox for this ticket that never got `old-tracker` mounted (read-only, like `hybriel` /
195. `ident.worldapi.org` / `antcolony-docs` above). Nothing was built or changed this session:
196. no `storage/`, no `tools/`, no id-mapping — writing the migration tool blind, guessing the
197. JSONL shape from the ticket text alone, risks silently wrong data and was avoided on purpose
198. (see the project's "Build it properly" rule). Filed as an antcolony issue so the next worker's
199. sandbox includes `old-tracker` before another attempt.
201. ## 2026-09-27 — ticket #1: the empty shell, ident login
203. Built the app from scratch: vendored `bin/hybriel` + `plugins/` + `shared/tokens.hl` from
204. `ident.worldapi.org` (newest local build, hybriel master 837fe120, no local patch — see
205. README "Vendored Hybriel"). `project.hl` (routes `/login`, `/callback`, `/logout`, `/`),
206. `components/home.hl` (the empty homepage), `components/main.hl` (shell), `styles.hl`
207. (accent green `#4ec9b0`, per `antcolony/README.md` "look and style" / `CONCEPT.md`).
208. `docker-compose.yml` / `Dockerfile` / `deploy.sh` following ident's own house pattern
209. (port 45008, container `tracker.worldapi.org`).
211. **Login**: the ident login BUTTON flow (ident.worldapi.org README "How apps use ident"),
212. server-side exchange (`POST <ident>/api/exchange`), this app's own framework session
213. (`session.user = { identity }`, cookie `trackersid`). ident hands over only the identity's
214. public **short id** — no display name yet (ident#11, properties handover, is on hold) — so
215. "your name" on the homepage is `Signed in as <shortid>`, the same identifier every other
216. worldapi app would show; see `decided` in the report.
218. **Lesson (Hybriel)**: a face that mutates `session.user` (e.g. `signOut`) does NOT
219. automatically re-render the page — a component's reactive members (`signedIn`, `identity`,
220. …) are computed once at mount from the framework's read-only session snapshot; a client
221. handler that calls `emit server X()` must ALSO flip the affected members itself afterwards
222. (ident's own `components/home.hl` `on doSignOut(e)` does exactly this: `emit server
223. signOut()` then `signedIn = false` …). Missing that made the sign-out button silently do
224. nothing client-side (the ack came back `ok:true`, the server-side session really was
225. cleared — proven by a reload — but the DOM never updated) until copied.
227. **Gate** (`tests/browser.mjs`, 13 checks): a fresh copy of `ident.worldapi.org`'s code (no
228. `.env`, no `storage/`, no `.sessions/` — verified empty of secrets/data before first use)
229. runs as this gate's own ident, on its own storage and its own mail sink (no live ident, no
230. real mail). The gate registers this app in that dev ident once over the `/__hl/emit` API
231. (the same one-time step a person does by hand in ident's `/apps` page), then drives the
232. REAL login-button flow in a real headless Chrome: signed out → click "Sign in with ident" →
233. ident's email form → the mail-sink code → the first-login optional-names form (Skip) → the
234. one-identity choice (one click) → back on tracker, "Signed in as `<shortid>`" → "Sign out" →
235. signed out again → a reload stays signed out. No console errors. Ran twice for stability, 0
236. failures; no leftover Chrome/hybriel processes after either run.
238. ```
239. node tests/browser.mjs
240. 13 passed, 0 failed
241. ```
243. `./deploy.sh --dry-run --target .scratch/deploy-preview --url http://127.0.0.1:0/` (a local
244. directory, per ident's own convention for testing deploy.sh without touching Byrodin): gate
245. passed, rsync preview held none of `storage/`, `.sessions/`, `.env`, `.scratch/`, `server.*`,
246. logs — confirmed by the same grep the script refuses on.
248. **Open**: the app is not registered with the LIVE ident yet — `TRACKER_KEY`/`TRACKER_SECRET`
249. are unset until the architect's first deploy does that (README "Deploy"); until then
250. `/login` on the live site answers a plain error page instead of redirecting (`/` itself
251. still renders). This is normal for a brand-new app, the same as ident's own SMTP `.env`
252. before its first deploy — not something this ticket's worker can set (no `.env` access,
253. and it is a LIVE ident registration).