tracker

120.8 KB

1. # tracker.worldapi.org
3. The new tracker: a Hybriel app that will track the TV shows (and later movies) the creator
4. follows and watches. **`CONCEPT.md` (the creator's) is the source of truth** — read it first;
5. nothing is built that it does not describe. Built step by step, one ticket per step
6. (tracker.worldapi.org#1, #2, …).
8. **Built so far — step 1 (ticket #1)**: the shell. The header carries the ident login (an
9. identity selector, like calendar/gitoria) and sign-out; the homepage itself stays empty. No
10. design carried over from the old app.
12. **Step 2 (ticket #2)**: the old tracker's MongoDB export is now in `storage/mpackdb/` (new
13. mpackdb ids, every reference re-pointed) — see "Data" and `STATUS.md`. Nothing is shown in the
14. UI yet; the homepage still renders empty. That is a later step, from the creator.
16. **Step 4 (ticket #4)**: the show page, `/shows/:slug` (`components/show.hl`) — poster, title,
17. genre pills, plot, cast (as described in "What it does (step 4)" below), every season with its
18. episodes, and the watch check (episode and season, season bulk-toggles its episodes).
20. **Step 5 (ticket #5)**: `/unwatched` (`components/unwatched.hl`) — every already-released,
21. unwatched episode of a show the signed-in user follows, newest release first (see "What it does
22. (step 5)" below).
24. **Step 6 (ticket #6)**: `/schedule` (`components/schedule.hl`) — every not-yet-released episode
25. of a show the signed-in user follows, soonest first, no check icons (see "What it does (step 6)"
26. below).
28. **Step 7 (ticket #7)**: `/my/shows` (`components/myshows.hl`) — every show the signed-in user
29. follows, newest follow first: small poster, title, last watched episode (`S08E35`) (see "What it
30. does (step 7)" below). The homepage itself is still empty.
32. **Step 8 (ticket #8)**: the personal lists all live under `/my/`: `/my/shows`, `/my/unwatched`,
33. `/my/schedule` (old `/unwatched`, `/schedule` answer 301). Episodes everywhere as `S01E01`, show
34. titles in lists with the year (`Doctor Who (2005)`), "1 episode" singular, and every `/my/` page
35. under 1 s full load on the real data (see "What it does (step 8)" below).
37. **Step 9 (ticket #9)**: the TMDB sync — every show someone follows gets its new seasons/episodes
38. (existing ones updated: title, summary, release) and its poster from TMDB, once a day inside the app
39. (04:00 UTC) and on demand with `tools/sync-tmdb.hl` (see "What it does (step 9)" below).
41. **Step 10 (ticket #10)**: an installable app (PWA) with its own icon — a TV with a check mark, tracker green
42. on dark; also the favicon. The shell (`/`) opens without network and says it is offline (see "What it does
43. (step 10)" below).
45. **Step 12 (ticket #12)**: the show page's header has small link icons to the show on TMDB, IMDb, TheTVDB and
46. TVmaze (a movie: TMDB + IMDb), only for the ids the show has, each in a new tab; the TMDB sync fills missing
47. ids (TMDB `external_ids`, TVmaze lookup) and never overwrites one (see "What it does (step 12)" below).
49. **Step 13 (ticket #13)**: the homepage has content — signed in, four big icons (Unwatched, Schedule, Shows,
50. Movies → the `/my/` pages); for everyone a short text about the site, the 10 latest released movies and the 10
51. series with the newest released episode (headings link to `/movies` and `/shows`: everything, 24 per page). Plus
52. `/my/movies` (the movies you follow); `/my/shows` is series only now (see "What it does (step 13)" below).
54. **Step 15 (ticket #15)**: the daily sync also asks TVmaze for every series and adds what TVmaze has but TMDB not
55. yet — new episodes/seasons, missing titles and air dates; TMDB values are never overwritten (see "What it does
56. (step 15)" below).
58. **Step 14 (ticket #14)**: the search — a magnifier in the header opens `/search`; typing searches OUR shows, movies and
59. people at once (in-memory index), "Fetch from web" lists TMDB titles we don't have yet, "Add" imports one like the daily
60. sync and opens its page (signed out: the sign-in modal) (see "What it does (ticket #14)" below).
62. **Mission 054**: adult titles never appear in the public lists (homepage rows, `/movies`, `/shows`) or the search (ours and
63. "Fetch from web") — only titles TMDB says are NOT adult are listed, unknown counts as hidden. A followed title stays on the
64. user's `/my/` pages and its page. A background job in the app (the "adult backfill") asks TMDB once per title for the
65. adult flag, the poster and the external ids (see "What it does (mission 054)" below).
66. **Step 16 (ticket #16)**: person pages — `/person/<slug>` (the show page's cast and the search's people link there): name,
67. photo, born/died, a short bio, the filmography (poster, title, year, role, newest first). A person we don't have complete yet is
68. fetched from TMDB on the first visit (person + all credits, every missing title added), the page fills in without a reload
69. (see "What it does (ticket #16)" below).
71. **Mission 060 (ticket #27)**: every movie, series and person has a 5-character short id (`shortId`, like `a9s9a`) shown
72. under its poster / photo; the 702 titles the old tracker numbered keep theirs (`100jh`…); `/<shortId>` answers 301 to the page
73. (see "What it does (mission 060)" below).
75. **Ticket #26**: titles a person page adds from a filmography (#16: minimal records) are completed — opening one shows skeletons and
76. fills in poster, links, cast, seasons/episodes without a reload; a background job (the "details repair") completes the rest (see
77. "What it does (ticket #26)" below).
79. **Ticket #28**: a title page shows its WHOLE cast (a series: every season's, main cast first by episode count, then "Guest stars")
80. with the characters, collapsed after 20 with "Show all (N)", and its key crew — Created by, Directed by, Written by, Screenplay,
81. Story, Music — every name linked to its person page. Stored by the daily sync, the search import and the completion of #26 from the
82. details request they make anyway; every other title gets them once from a background job (the "credits job") (see "What it does
83. (ticket #28)" below).
85. **Ticket #18**: the daily sync asks TMDB's and TVmaze's CHANGE LISTS first (what changed since the last run) and syncs only the
86. titles of ours that changed — followed or not, so every title stays current; it refreshes the title's own record too (title,
87. TMDB text, tagline, status, genres …). `summary` is the creator's own text and never written by the sync: the page shows
88. `summary`, else TMDB's `tmdbSummary`, else TVmaze's `tvmazeSummary` (see "What it does (ticket #18)" below).
89. **Mission 068 (tickets #20, #21)**: every page heading says its type first, in the type's colour, then the name in white
90. ("Series | Reacher", "Movie | …", "Show | …", "Genre | Action", "Person | …"). TV titles are split into **Series** (scripted,
91. mini-series, documentary) under `/series` and **Shows** (reality, talk, news, video) under `/shows`; movies live at
92. `/movies/<slug>`. The old `/shows/<slug>` of a series or movie answers 301. Home, search, `/my/series` + `/my/shows` follow the
93. split. A background job asks TMDB once per TV title for its type (details: [docs/kinds.md](docs/kinds.md)).
95. **Ticket #19**: franchises and timelines — `/franchises`, `/franchises/<slug>`, `/timelines/<slug>`, a `‹ Prequel | Timeline |
96. Sequel ›` widget on title pages, the creator's editor, and a background seed from TMDB movie collections (see "What it does
97. (ticket #19)" below, [docs/franchises.md](docs/franchises.md)).
99. **Mission 070 (2026-10-02)**: #18, #20/#21 and #19 merged into main; the sync never destroys migrated data (copied summaries are
100. MOVED to `migratedSummary`, a replaced title/genres/homepage/tagline is kept in `migrated*`); Hybriel re-vendored to 8efba065
101. (memory fix #126) with `&` on read-only lambda parameters (#48); `HL_GC_BYTES` stays Hybriel's default (see "Deploy" and
102. "Vendored Hybriel" below, STATUS.md mission 070).
104. Written in **Hybriel** on **hl:web**, same stack and conventions as `ident.worldapi.org`
105. (vendored plugins/binary copied from there, see "Vendored Hybriel").
107. ## Run (dev, Loreana)
109. ```bash
110. cd /media/STORAGE/projects/tracker.worldapi.org
111. TRACKER_PORT=8700 TRACKER_SYNC=0 setsid nohup ./bin/hybriel project.hl > server.log 2>&1 < /dev/null & echo $! > server.pid
112. # TRACKER_SYNC=0: the runtime loads .env (real TMDB token) by itself — without it a dev server syncs from TMDB at 04:00 UTC
113. # stop: kill $(cat server.pid)
114. ```
116. * Config (env; the real environment outranks nothing here — there is no `.env` reader in
117. project.hl, unlike ident's SMTP settings — set these in the shell or `docker-compose.yml`):
119. | Variable | Default | |
120. |---|---|---|
121. | `TRACKER_PORT` | 45008 | |
122. | `TRACKER_URL` | `http://127.0.0.1:<port>` | this app's own origin — the ident login button's `return=` is built from it |
123. | `IDENT_URL` | `https://ident.worldapi.org` | |
124. | `TRACKER_KEY` / `TRACKER_SECRET` | `''` | this app's API key + secret, from ident's `/apps` (register once, origin = `TRACKER_URL`) — **not set yet on Byrodin**: the first deploy (architect) registers the app in the live ident and puts these in `.env` next to `docker-compose.yml`, exactly like ident's own `.env` holds its SMTP settings |
125. | `TRACKER_WATCH` | on | `0` = no dev watcher (the container) |
126. | `TRACKER_STORAGE` | `./storage/mpackdb` | the `usersTable` directory |
127. | `TRACKER_SESSIONS` | `./.sessions` (hl:web's own default) | this app's own session store |
128. | `HL_HOST` (or `HOST`) | 0.0.0.0 | interface to bind; `127.0.0.1` on Byrodin |
129. | `TMDB_READ_TOKEN` | — | TMDB v4 read token (in `.env`, never print it). Unset → the daily sync is off. **The runtime loads the `.env` beside `project.hl` by itself** (the real environment wins): every dev server on Loreana has the real token — start dev servers with `TRACKER_SYNC=0` |
130. | `TRACKER_SYNC` | on | `0` = no daily TMDB run inside the app |
131. | `TRACKER_SYNC_HOUR` | 4 | UTC hour of the daily run (hl:time has no time zones; the container is UTC) — #18: by change lists since the day in `storage/mpackdb/sync-state.txt` |
132. | `TMDB_BASE_URL` / `TMDB_IMAGE_URL` | `https://api.themoviedb.org/3` / `https://image.tmdb.org/t/p` | the gate points both at its fake TMDB |
133. | `TVMAZE_BASE_URL` | `https://api.tvmaze.com` | #12: the TVmaze id lookup; the gate points it at its fake (`<fake>/tvmaze`) |
134. | `TRACKER_BACKFILL` | on | mission 054: `0` = no adult-flag backfill (off too without `TMDB_READ_TOKEN`). **A dev server with the real token runs it** — it only writes the storage it was given, but asks real TMDB |
135. | `TRACKER_BACKFILL_LOG_EVERY` | 500 | a progress line every N titles |
136. | `TRACKER_OLD_SHORT_IDS` | `./data/old-short-ids.json` | mission 060: the old tracker's short ids (`oldId` → id), part of the code (deployed with it) |
137. | `TRACKER_SHORTIDS` | on | mission 060: `0` = no short-id backfill (the gate's resume check; ids for NEW records are still given) |
138. | `TRACKER_SHORTID_BUDGET_MS` | 40 | mission 060: ms of work per backfill batch (one batch per 0.1 s tick) |
139. | `TRACKER_SHORTID_LOG_EVERY` | 5000 | mission 060: a progress line every N records |
140. | `TRACKER_REPAIR` | on | #26: `0` = no details repair (off too without `TMDB_READ_TOKEN`). Runs after the adult backfill is done. **A dev server with the real token runs it** (writes only its storage, asks real TMDB/TVmaze) |
141. | `TRACKER_REPAIR_LOG_EVERY` | 100 | a progress line every N titles |
142. | `TRACKER_REPAIR_FAST` | off | `1` = 20 ms pause between titles instead of the TMDB pacing — the gate only (fake TMDB) |
143. | `TRACKER_CREDITS` | on | #28: `0` = no credits job (off too without `TMDB_READ_TOKEN`). Runs after the backfill and the repair are done. **A dev server with the real token runs it** (writes only its storage, asks real TMDB) |
144. | `TRACKER_CREDITS_LOG_EVERY` | 500 | a progress line every N titles |
145. | `TRACKER_CREDITS_FAST` | off | `1` = 20 ms pause between titles — the gate only (fake TMDB) |
146. | `TRACKER_CREDITS_MAX_MB` | 16000 | #28: the credits job pauses (for this process; goes on after the next restart) when the app's RSS is above this — the runtime never frees fetched/decoded data |
147. | `TRACKER_CAST_MAX` | 300 | #28: at most this many cast per title (TMDB's order) — a talk show / soap lists 3,000+ one-episode guests |
148. | `TRACKER_KINDS` | on | mission 068: `0` = no kind backfill (TMDB's TV type per title; off too without `TMDB_READ_TOKEN`). **A dev server with the real token runs it** (~1,600 requests on the real data) |
149. | `TRACKER_KINDS_LOG_EVERY` | 500 | mission 068: a progress line every N titles |
150. | `HL_GC_BYTES` | 64 MiB (Hybriel's default — **not set** in docker-compose.yml) | mission 070: the runtime also collects garbage after this many bytes of plugin data (#126). Smaller = less memory, slower big pages; `0` = only by count (the old binary's memory). Real copy, 200 page loads after the jobs: default → `/my/unwatched` 4.8 s, RSS 2.1 → 2.8 GB; 256 MiB → 2.2 s, 2.7 → 3.5 GB but the first-start jobs grew to 12+ GB (the old binary: 15 GB); 0 → 1.2 s, 6.2 → 8.7 GB |
151. | `TRACKER_COLLECTIONS` | on | #19: `0` = no TMDB collection seed (off too without `TMDB_READ_TOKEN`; the main gate sets `0`). **A dev server with the real token runs it** against the storage it was given |
152. | `TRACKER_COLLECTIONS_LOG_EVERY` / `TRACKER_COLLECTIONS_PERSIST_EVERY` | 500 / 25 | #19: a progress line / a persist every N movies |
153. | `TRACKER_EDITORS` | `az5b2` | #19: ident ids (comma list) allowed to edit franchises and timelines (the gate sets its own) |
155. Without `TRACKER_KEY`/`TRACKER_SECRET`, `/` still renders (signed out, the selector and "Log
156. in with ident" show in the header); a login attempt answers `'login is not set up on this
157. server (TRACKER_KEY / TRACKER_SECRET missing)'` instead of exchanging a code — so the app is
158. never a dead 500 while waiting for that one-time setup.
160. ## What it does (step 1)
162. * **Login, copied unchanged from calendar.worldapi.org** (rejected once for a centered
163. sign-in and no header selector — architect, 2026-09-27): ident only, no own passwords
164. (ident's login button flow, ident.worldapi.org README "How apps use ident"), the identity
165. selector (`<ident-selector>`, ident's `/selector.js`) sits in the header
166. (`components/main.hl`, `userBox`), a plain "Log in with ident" link beside it when signed
167. out. Choosing an identity in the selector fires `ident-login` (one-time code); `login.js`
168. (an external-component bridge, allowed per the creator: "login.js is correct, as thats for
169. externals in general") hands the code to a hidden input, whose `change` calls the server
170. face `trackerLogin` (`users.hl` `exchangeCode`, `POST <ident>/api/exchange` — nothing else:
171. ident does not hand over a display name yet, ident#23 done / ident#11 on hold), which makes
172. or finds this app's own user record (`usersTable`, `storage/mpackdb/users.db`) and sets
173. `session.user = { id }`. `/login/callback` is the same exchange as a plain redirect (the
174. login button's return URL) for a client that has no socket yet. "Log out" in the header
175. calls `trackerLogOut`; ident's own login is untouched. This app's own session cookie is
176. `trackersid` (cookies ignore ports, an own name keeps it apart from ident's `identsid` on
177. the same dev host, hybriel#10/#17).
178. * **The empty homepage** (until #13 — see "What it does (step 13)") (`/`, `components/home.hl`): no content at all, signed in or out —
179. the header alone carries the login state. Nothing else.
180. * **One mpackdb table**: `storage/mpackdb/users.db` (`identity` → this app's user id — no
181. display name, no other per-user data yet). Step 2 (tracker.worldapi.org#2, open) brings the
182. old tracker's data across.
184. ## What it does (step 4)
186. `/shows/:slug` (`components/show.hl`, slug = the migrated `urlSegment`), read-only data from
187. `shows.hl`, per-user watch state from `watches.hl` (both reused as-is by `/unwatched` and
188. `/schedule`, tickets #5/#6 — "reuse, don't copy"):
190. * **Header**: poster on the left (`/posters/<name>`, `project.hl` `posterRoute` — served from
191. `storage/mpackdb/posters/`; the real poster files were never in the Mongo backup, see
192. `DECISIONS.md` "tracker concept" — a placeholder SVG shows until a later TMDB step brings
193. real ones back), title, genre pills (blue, link to `/genre/<slug>`) and the plot summary to
194. its right, the cast below as comma-separated red links to `/person/<slug>`.
195. * **Seasons**, latest first, the latest expanded and every other one collapsed to start; each
196. row: a round check icon (a disc with a check, drawn in SVG/CSS, no icon font), the season
197. title/number, its episode count. Its episodes (number + title, and their own check icon)
198. show once expanded (click the row to toggle).
199. * **The watch check** (signed-in only — signed out, no check icons show at all): clicking an
200. episode's check toggles that one watch; clicking a season's check toggles the season's own
201. watch AND every one of its episodes at once (`watches.hl` `setSeasonWatched`). Every click is
202. a server round trip (`emit server showToggleEpisode` / `showToggleSeason`; collapse/expand: see
203. ticket #17 below) that rebuilds the row list server-side and reassigns it whole — a season has
204. at most a few dozen episodes, cheap. A **Hybriel gap found here** (filed as hybriel#115,
205. reported after this ticket's first attempt): a plain per-instance member function that calls
206. an hl:mpackdb-backed import comes back `null` when called from inside an `on server` face,
207. even though the very same call works as a member's own top-level initializer — the fix is to
208. declare such helpers `static` (the same shape calendar.worldapi.org's `soonOf` already uses).
209. `components/show.hl`'s own helpers (`buildRows`, `genreRowsOf`, …) are `static` for exactly
210. this reason.
211. * **Data fix**: `tools/relink-episode-seasons.hl`, a one-off idempotent tool that links every
212. episode whose `season` was still null in the migrated data (36,194 of them, ticket #2) to its
213. season by matching `(show, seasonNumber)` — never touches an episode whose season is already
214. set, never runs `update()` on the live table (hybriel#113; it rebuilds a fresh episodes table
215. and swaps the files over, the same rule `tools/migrate.hl` follows). Not run against the live
216. data yet — the architect runs it (stop the container first, `storage/mpackdb` is not shared
217. across processes, hybriel#21).
219. ## What it does (step 5)
221. `/unwatched` (`components/unwatched.hl`): every episode of a show the signed-in user follows
222. (`follows.hl`, new) that is already released (`release` ≤ today) and not yet watched
223. (`watches.hl` `isWatched`), newest release first. Creator: "/unwatched lists all unwatched
224. episodes of shows followed in release date desc order"; "check icon is a disc with a check in
225. it" — the SAME icon/class (`watches.hl` `watchClassOf`, moved there from `components/show.hl` so
226. both pages share one definition instead of each declaring its own) and the same
227. `setWatched`/`isWatched` watch logic as the show page (step 4), not copied.
229. * Each row: check icon, the show's title (linking to `/shows/:slug`), "Ep `<number>` ·
230. `<title>`", the release date.
231. * A row only ever shows an unwatched episode, so a click always marks it watched (never toggles
232. back, unlike the show page's checks) — the row leaves the list. Signed out: no rows, a
233. "Sign in to see the shows you follow." message instead.
234. * `follows.hl` (new): read-only per-user follow data (`followedShowIds`), kept apart from
235. `shows.hl`/`watches.hl` the same way, for `/schedule` (ticket #6) to reuse.
237. ## What it does (step 6)
239. `/schedule` (`components/schedule.hl`): every episode of a show the signed-in user follows
240. (`follows.hl`) that is NOT yet released (`release` > today), soonest first. Creator: "/schedule
241. has a similar list without check icons for the upcoming episodes of shows followed" — same row
242. layout as `/unwatched` (show title linking to `/shows/:slug`, "Ep `<number>` · `<title>`", the
243. release date), minus the check icon and minus any click handling: the page is read-only, so it
244. needs no `on server` face at all. Signed out: no rows, the same "Sign in to see the shows you
245. follow." message as `/unwatched`.
247. ## What it does (step 7)
249. `/my/shows` (`components/myshows.hl`): every show the signed-in user follows, ordered by the
250. follow's `at` (epoch ms), newest first. Creator: "trackers /my/shows that just ists the shows in
251. desc order i followed them, small poster, title, last episode like s08e35". Read-only, no `on
252. server` face (like `/schedule`).
254. * Each row: a small poster (2.5rem wide, the same `/posters/<name>` route + placeholder as the show
255. page; a show with no poster name at all gets `/posters/none`, i.e. the route's placeholder), the
256. title linking to `/shows/<slug>`, and the **last watched episode** as `SxxEyy` (zero-padded to
257. two digits) = the HIGHEST season/episode number the user has an episode watch for (architect's
258. reading, in the ticket) — not the most recently watched one; season watches alone don't count.
259. Nothing watched → no episode element at all.
260. * Signed out: no rows, the same "Sign in to see the shows you follow." as `/unwatched`.
261. * Cost: touches only the user's follows + episode watches (one `episodeById` fetch per watch),
262. never all episodes of a followed show — 128 rows in ~0.2 s on a copy of the real data (STATUS.md).
263. * Reuse: `follows.hl` `followsOfUser` (new — the whole follow record, `followedShowIds` now
264. builds on it), `shows.hl` `episodeById` (new, one line), `showById`, `posterName`, `watches.hl`
265. `watchesOfUser`.
267. ## What it does (step 8)
269. * Routes (`project.hl`): `/my/unwatched`, `/my/schedule`; the old `/unwatched`, `/schedule` are
270. function routes answering **301** to them (`movedTo`, no query carried — the pages take none).
271. * `shows.hl` (shared by all lists): `episodeCode(s, e)` → `S01E01` (`/my/shows`, `/my/unwatched`,
272. `/my/schedule` rows: `S01E02 · <title>`), `titleWithYear(show)` → `Doctor Who (2005)` (`year`,
273. else the first 4 chars of `release`, else the bare title — 197 shows have no `year`), `pad2`,
274. `posterUrlOf(show)`. The show page's own `h1` stays without the year (it is not a list).
275. * Show page: season count `1 episode` / `N episodes`.
276. * **Speed** (the creator's real data: 128 follows, 11,692 watches, 831 unwatched rows):
277. * `/unwatched` was ~15 s: `isWatched` scanned all 11,692 watches per episode. Now
278. `watches.hl` `watchedSet(rows)` builds `'<type>:<id>' → true` ONCE; `/my/unwatched` and the
279. show page's `buildRows` look up that map. Now ~0.6 s server render (the rest is fetching
280. every episode of 128 shows; the insertion sort is ~50 ms).
281. * `/my/shows` "25 s full load while data takes 0.2 s": (1) **hl:web serves one request at a
282. time** — measured: a `/my/shows` request made during a 15 s `/unwatched` render waited 13.7 s;
283. every poster request of every visitor queues the same way. Fixing `/unwatched` removes that.
284. (2) 128 rows = 128 different `/posters/<oldId>.jpg` URLs, all answering the same placeholder
285. (no poster files exist yet) — 128 requests per load. `posterUrlOf` now gives rows without a
286. poster FILE the one URL `/posters/none` → 1 request, cached.
287. * How it was measured: see STATUS.md ticket #8 ("Real-data check").
289. ## What it does (step 9)
291. `tmdbsync.hl` (`syncShow`), run by the app every day and by `tools/sync-tmdb.hl`:
293. * **Which shows**: every show ANY user follows (`follows.hl` `allFollowedShowIds`, 126 on the real data). **Since #18 only the
294. "full walk"** (first run, gap > 14 days, failed change list) — the normal daily run takes the changed titles (ticket #18 below).
295. * **Per show** (1 + ⌈seasons/20⌉ requests + the poster if new): `GET /tv/<tmdbId>?append_to_response=external_ids`
296. (status, counts, overview, `poster_path`, the season list, since #12 the external ids), then `GET /tv/<tmdbId>?append_to_response=season/1,…` (≤ 20
297. seasons with their episodes per request). Bearer `TMDB_READ_TOKEN`.
298. * **Matching, never duplicating, never deleting**: episode by `tmdbId`, else (161 migrated episodes have
299. none) by S/E number if that one has no tmdbId yet — then it gets the tmdbId; season by (show,
300. seasonNumber) (migrated seasons have no tmdbId). Existing episodes/seasons: title, summary, release
301. updated; an empty TMDB value never overwrites. New ones: self-assigned 16-hex ids (NOT mpackdb's own —
302. hybriel#113), `oldId = 'tmdb-episode:<id>'` / `'tmdb-season:<showTmdb>:<n>'` (the tables' `!oldId`
303. index), linked into `season.episodes` / `show.seasons`. Show: `status`, `seasonsCount`,
304. `episodesCount`, `tmdbSummary`, `tmdbPoster`, `tmdbSync` (ms). Season 0 (specials) only for a show that
305. already has one; a TMDB season with no episodes is not created.
306. * **Posters**: TMDB `poster_path`, size w342 → `storage/mpackdb/posters/<oldId>.<ext>` (the folder the
307. existing `/posters/:name` route already served; inside `storage/` → in the nightly backup), `show.image`
308. set to the ext. Downloaded again only when the file is missing or TMDB's `poster_path` changed.
309. * **Daily run inside the app** (`project.hl` `syncTick`): an hl:time `every(1)` clock; at
310. `TRACKER_SYNC_HOUR` (UTC), once per day, it queues all followed shows and then syncs ONE show per tick.
311. hl:web serves one request at a time and `fetch()` blocks the whole process, so a step blocks requests
312. for its duration (real data: ~0.7 s average, Saturday Night Live 53 seasons ~1.7 s; logged as
313. `tmdb sync: slow step …` above 1.5 s) — requests queued meanwhile are served between two ticks; a tick
314. never runs inside a page render. After a show with n requests the next waits n × 260 ms (TMDB ≤ 40
315. requests / 10 s). Log: `tmdb sync: daily at 4:00 UTC`, `tmdb sync: start, N followed shows`,
316. `tmdb sync done: shows=… newSeasons=… newEpisodes=… updatedEpisodes=… updatedSeasons=… posters=…
317. requests=… errors=… seconds=…` (`docker logs tracker.worldapi.org | grep tmdb`). Tables are persisted
318. every 10 shows and at the end (~36 ms).
319. * **The tool** (same sync, sequential, paced; prints one line per show + the totals). **App stopped or a
320. COPY only** — hl:mpackdb is one process per storage (hybriel#21):
321. ```bash
322. set -a; . ./.env; set +a # TMDB_READ_TOKEN, never print it
323. TRACKER_STORAGE=$PWD/.scratch/realdata/storage/mpackdb ./bin/hybriel tools/sync-tmdb.hl [--limit N]
324. ```
325. * Real data (copy, 2026-09-30): first run 126 shows → +20 seasons, +800 episodes, 2651 episodes
326. updated, 125 posters (6.4 MB), 380 requests, 0 errors, 186 s; a second run: all 0, 255 requests, 90 s.
327. Details: STATUS.md ticket #9.
329. ## What it does (step 10)
331. The installable app, the same way calendar.worldapi.org does it (its ticket #3) — hl:web's own manifest
332. and service worker from settings in `project.hl`, no JavaScript of ours:
334. * `appIcons` (192 + 512 PNG, each `any` and `maskable`), `appTouchIcon` (180 PNG), `appFavicon`
335. (`/icons/favicon.svg`), `appThemeColor` = token `darker` (the header, rgb(15, 20, 25)),
336. `appBackgroundColor` = token `dark` (rgb(25, 30, 35)); name = `appTitle` "tracker". hl:web serves
337. `/__hl/manifest.webmanifest` (start_url/scope `/`, display standalone) and links it, the
338. apple-touch-icon and theme-color from every head. `/favicon.ico` is a real icon (16/32/48) for
339. browsers that ask for it themselves. Each icon has its own `file` route in `project.hl`.
340. * **Icons** (`icons/`): `icon.svg` is the source (512, hand-written: a TV with antennas and a check on
341. the screen, `#4ec9b0` on rgb(25,30,35); everything inside the maskable safe zone, a circle of radius
342. 204, so one image serves `any` and `maskable`); `favicon.svg` is the same drawing cropped tight on a
343. rounded tile. Rendered on Loreana:
344. ```bash
345. rsvg-convert -w 192 -h 192 icons/icon.svg -o icons/icon-192.png
346. rsvg-convert -w 512 -h 512 icons/icon.svg -o icons/icon-512.png
347. rsvg-convert -w 180 -h 180 icons/icon.svg -o icons/apple-touch-icon.png
348. for s in 16 32 48; do rsvg-convert -w $s -h $s icons/favicon.svg -o /tmp/fav-$s.png; done
349. magick /tmp/fav-16.png /tmp/fav-32.png /tmp/fav-48.png icons/favicon.ico
350. ```
351. * **Offline**: `offline = [ Home ]` — the worker precaches the shell (runtime, modules, CSS, manifest,
352. icons, favicon) and the document of `/`. Navigations are network-first; without a network `/` comes
353. from the cache and every other page gets hl:web's own "Unavailable offline" page (503) — data pages
354. need the network, no offline data. The shell (`components/main.hl`) shows "You are offline. Your shows
355. and lists need the network." while `navigator.onLine` is false: hl:web gives a page no connection
356. state and no mount hook, so an invisible `netProbe` runs an endless 1 s CSS animation (`styles.hl`
357. `@keyframes tracker-net-tick`) and its `animationiteration` handler reads `navigator.onLine` (a write
358. only when it changes). A server that is down while the device is online shows no note.
360. ## What it does (step 12)
362. * **Link icons** (`components/show.hl`, `shows.hl` `externalLinksOf`, `styles.hl` `showLinks`/`a.extlink`): under the
363. title (phone: between the title row and the full-width poster; desktop: under the title, right of the poster) one
364. small monochrome badge per id the show HAS — `TMDB` https://www.themoviedb.org/tv/<tmdbId> (a movie: `/movie/<id>`),
365. `IMDb` https://www.imdb.com/title/<imdbId>/, `TVDB` https://thetvdb.com/dereferrer/series/<tvdbId>, `TVmaze`
366. https://www.tvmaze.com/shows/<tvmzId>. A movie (`type = 'movie'`) shows only TMDB + IMDb. Each `target="_blank"
367. rel="noopener"`. CSS only (muted grey badge, dark text, accent on hover), no copied logos. URL forms checked
368. 2026-10-01: all four 301 to the site's slug page (TheTVDB's old `?tab=series&id=` too; `dereferrer` is its id form).
369. * **Sync** (`tmdbsync.hl` `externalIdsFor`): the details request carries `append_to_response=external_ids` (no extra
370. request); a MISSING `imdbId` (`tt` + digits) / `tvdbId` (number > 0) is filled from it. A show still without
371. `tvmzId` → `GET <TVMAZE_BASE_URL>/lookup/shows?imdb=tt…`, on 404 `?thetvdb=<id>` (hl:fetch follows TVmaze's 301 to
372. `/shows/<id>`); 550 ms pause per lookup (TVmaze 20 / 10 s) — both runners wait `pauseMsAfter(r)`. **An id the show
373. has is never overwritten.** Totals gain `ids=` (ids filled) and `tvmazeRequests=`.
374. * **Movies** (7814 of 9453 migrated rows are `type = 'movie'`, 1 followed: Star Trek: First Contact): before #12 the
375. sync asked `/tv/<tmdbId>` for them — a DIFFERENT title on TMDB. Now a movie gets one `GET
376. /movie/<tmdbId>?append_to_response=external_ids`: missing imdbId + its poster; no seasons/status/overview.
377. * **Counting ids** (a COPY only): `TRACKER_STORAGE=$PWD/.scratch/realdata/storage/mpackdb ./bin/hybriel
378. tools/count-external-ids.hl` → per type, all / followed: shows, tmdbId, imdbId, tvdbId, tvmzId.
380. ## What it does (step 15)
382. * **TVmaze merge** (`tmdbsync.hl` `mergeTvmaze`, after the TMDB step of `syncShow`, both runners): a series with a
383. `tvmzId` → `GET <TVMAZE_BASE_URL>/shows/<tvmzId>/episodes` (1 request, 550 ms pause like the lookups). Matched by
384. season/episode number: a missing episode is ADDED (`tmdbId = null`, `tvmzId`, `oldId 'tvmaze-episode:<id>'`; its
385. season too if missing: `oldId 'tvmaze-season:<tvmzId>:<n>'`), an existing one gets an EMPTY title / air date
386. filled. **A TMDB value is never overwritten.** Placeholder titles (`Episode 4`, `Épisode 4`, `Folge 4`, `TBA`,
387. `isPlaceholderTitle`) count as empty — and TMDB's placeholder no longer overwrites a real title (else they flip
388. daily). When TMDB lists the episode later, the TMDB step adopts the row by number (it has no tmdbId), TMDB wins.
389. Skipped: TVmaze specials (no number), season 0 unless the show has one, summaries (TVmaze's are HTML).
390. * **Numbering check** (`numberingAgrees`): ≥ 80 % of the episodes both know must air within a day OR share the title,
391. and a show with episodes must share at least one S/E with TVmaze — else TVmaze is ignored for that show
392. (`tvmazeSkipped`). Real data: The Daily Show (TVmaze seasons by year: would have added 1093 episodes), Star Trek:
393. Prodigy / Blood & Treasure (TMDB merges double episodes → shifted), Alex Rider / Intergalactic / The Rising (other
394. premiere dates, placeholder titles) are skipped.
395. * Totals/tool line gain `tvmazeEpisodes= tvmazeSeasons= tvmazeFilled= tvmazeSkipped=`; `tvmazeRequests` now also
396. counts the episode lists (≈ 125/day on real data, ~70 s more per daily run).
398. ## What it does (ticket #14: search)
400. Creator: "in the actionbar a search symbol that first checks our database and then has a fetch from web button that
401. actually searches tmdb then for things we dont have yet". `components/search.hl` (page), `search.hl` (index, TMDB, import).
403. * **Header**: a magnifier (`#searchlink`, SVG) before the ident selector → `/search` (client-side navigation).
404. * **Address `/search/<text>`, not `/search?q=`**: hl:web gives a page its route params but no query string. A full load gets
405. the segment already decoded by the server (an encoded `/` would split it → 404), a client-side navigation the raw one —
406. `queryOfParam` decodes what is left, safely (malformed `%` → literal). While typing the address follows the text
407. (`history.replaceState`; `/ ? #` become spaces). `/search?q=x` just opens the empty search page.
408. * **Our database, as you type** (face `searchDb`, no session): `search.hl` builds an **in-memory index once at boot**
409. (`project.hl` → `ensureIndex`, ~0.6 s on the real data, log line `search index: 9453 titles, 15157 people, 4180 keys,
410. … ms`): every title/name → words (lower case, apostrophes dropped, punctuation = word break, Latin accents folded; a
411. title's year is one more word); each word's first 2 and 3 letters are bucket keys. A query reads ONE bucket (its longest
412. word's first 3 letters, 2 for a 2-letter word) and keeps entries where every query word starts one of the entry's words.
413. Order: whole name, name starts with the text, first word does, rest; then shorter name; then series before movie, newer
414. first. ≤ 30 titles ("The first 30 of N titles — type more…") + ≤ 10 people. Under 2 letters: "Type at least 2 letters.".
415. People link to `/person/<slug>` (no person page yet — same as the show page's cast).
416. * **Fetch from web** (face `searchWeb`, signed out too): ONE `GET <TMDB>/search/multi?query=…&include_adult=false&page=1`;
417. people dropped, titles we have (same TMDB id AND kind: `series:<id>` / `movie:<id>`) left out. Poster thumbs straight
418. from TMDB's image host (`w92`).
419. * **Add** (face `searchImport`, session): signed out → the sign-in modal, nothing sent. Signed in → `importTitle`: one
420. `GET /tv/<id>` (or `/movie/<id>`) for the new record (title, overview, first air/release date → year, status, genres
421. matched to OUR genres by name, language, homepage, tagline; own 16-hex id, `oldId = tmdb-tv-<id>` / `tmdb-movie-<id>`,
422. `urlSegment` like the migrated ones: `Pluribus`, taken → `-2`…), then **`tmdbsync.hl` `syncShow`** — seasons, episodes,
423. poster, external ids exactly like the daily sync — `persistSync`, index updated, the public lists too (`catalog.hl`
424. `refreshCatalogShow`, since the #13/#14 merge); then the show page opens. A title we
425. already have answers its slug. A failure (TMDB 404 …) is shown on the page ("Could not add it: …"), the lists stay.
426. Log: `search import: tv 225171 "Pluribus" → /shows/Pluribus (newSeasons=1 newEpisodes=9 posters=1 ids=3 requests=4 errors=0)`.
427. * An imported title is NOT followed automatically, so the daily sync (followed shows only) does not update it later.
428. * Hybriel traps (hybriel#121 / #122): no list is ever appended to (replaced whole); no member reads both the route param
429. `q` and `session`; the gate checks the lists after a failed import (a session face that stays on the page).
431. ## What it does (ticket #16: person pages)
433. Creator: "when one clicks on an artist that has no dataset yet the old version fetched that artist and its entire bibilogrqphy
434. so its profile was complete". `components/person.hl` (page), `people.hl` (data + the fetch).
436. * **`/person/<slug>`** (slug = the person's `urlSegment`; `people.hl` builds a slug → id map once, ~15k people): photo (TMDB
437. w185 → `storage/mpackdb/profiles/<oldId>.<ext>`, route `/profiles/:name` like `/posters/:name`; no file → no photo), name,
438. "Born 4 March 1986 in …", "Died …", the bio (four lines, a click shows all), then **Filmography**: one row per title (cast and
439. crew of the same title merged: "Hero / Director"), poster (our file, else TMDB's w92 thumb like search's web rows, else the
440. placeholder), title, year, Movie/Series, role; newest first by the title's release (else year), undated last; each row →
441. `/shows/<slug>`. Only PUBLIC titles are listed (merge, mission 056: shows.hl `isPublicTitle` = `adult == false`, unknown hidden like the lists) whose credit's TMDB flag is not adult either. Unknown slug →
442. "Not found".
443. * **Complete** = `persons.filmographyAt` (ms) set → the page is just read, TMDB never asked again. Not complete (every migrated
444. person; 153 carry the old tracker's credits, shown meanwhile and kept) and a `tmdbId` → **the fill**:
445. * the page shows "Loading filmography…"; an invisible `personProbe` (endless 0.1 s CSS animation, the same trick as the
446. shell's offline tick — hl:web has no mount hook) fires `animationiteration` → the client handler calls the face
447. `personFill` again and again until `done` ("Loading filmography… 40 of 87 titles"), then replaces the rows and facts —
448. no reload, works after a client-side navigation too.
449. * step 1: `GET <TMDB>/person/<tmdbId>?append_to_response=combined_credits` (ONE request) + the photo; facts updated (empty
450. TMDB values never overwrite: biography, birthday, deathday, place_of_birth, imdb_id, adult). Every next step: 40 titles.
451. A title we have (search.hl `titleIdByTmdb`, TMDB id + kind) is linked; one we don't gets a MINIMAL show record (title,
452. overview, year/release, type, tmdbId, `adult`, language, genres by TMDB id, `cast` = this person, no seasons/poster;
453. `oldId tmdb-<tv|movie>-<id>`, a free slug — search.hl's `slugOf`), indexed for the search (`indexShow`) and put into the
454. public lists (`catalog.hl refreshCatalogShow`). Last step: `person.shows` = the credits `{ show, character, posterPath,
455. adult }` (+ migrated credits TMDB no longer lists), `filmographyAt`, `persist()`. Log:
456. `person fill: Jared Harris (tmdb 15440) credits=97 added=80 requests=2 steps=4 tmdbMs=287 ms=624`.
457. * hl:web serves one request at a time and `fetch()` blocks: the TMDB step blocks other requests for its duration
458. (~0.25–0.65 s real), each 40-title step ~0.15 s; other requests are served between steps (real data: a 758-credit
459. person, 20 steps, 3.8 s, other requests waited ≤ 0.61 s).
460. * the job lives in memory (`people.hl jobs`); a restart mid-fill loses it — the next visit starts again, titles stored
461. already are found by TMDB id (no duplicates). A TMDB failure shows "Could not load the filmography: …"; the next visit
462. retries. No sign-in needed (no session face on this page — hybriel#121/#122 can't bite).
464. ## What it does (ticket #26: titles from a filmography are complete)
466. Creator: "when i go to an artist and it loads the bibliography those loaded movies and series miss most information and it stays
467. that way when i click on them". `details.hl` (the completion), `components/show.hl` (loading state), `project.hl` (`repairTick`).
469. * **What the filmography stored** (people.hl `addMinimalTitle`, #16): title, year/release, overview, type, tmdbId, `adult`, genres
470. (by TMDB id), language, `cast` = only that person; NO poster file, seasons/episodes, imdb/tvdb/tvmaze ids, homepage, tagline,
471. status, `tmdbSync`. A search import (search.hl `importTitle`) has all of those via `syncShow` — but no cast either. Real data
472. 2026-10-01: 321 such titles (6 people filled), every open showed exactly what the filmography stored.
473. * **Incomplete** (details.hl `isIncomplete`): `minimal = true` (set by the fill from now on) or `imported` without `tmdbSync`
474. (the 321 existing ones, and an import whose sync failed), no `detailsAt`, no `detailsCheck`. Adult titles are never completed.
475. * **Complete** (`completeStep`, called until `done`): step 1 = ONE `GET /movie|tv/<id>?append_to_response=external_ids,
476. credits|aggregate_credits` (fields + cast; a movie also its ids + poster → done). A series then goes on IN STEPS with that
477. answer (tmdbsync.hl `syncShowPart`, the daily sync's code without a second details request): one step per part of its seasons
478. (≤ 300 episodes, ≤ 20 seasons = one `season/…` request; `TRACKER_DETAILS_PART_EPISODES`), a last step = ids, TVmaze lookup +
479. episode merge, poster. Why: The Late Show (22 seasons, 4,252 episodes) as ONE step blocked the server 10.7 s (hl:web serves one
480. request at a time) — in parts every other request is served between two steps. The job state lives in memory (`jobs`); a
481. restart begins the title again, stored seasons/episodes are matched by tmdbId. Stored too: overview (summary only if ours is
482. empty), homepage, tagline, status, genres (TMDB ids → ours), release/year/language when missing, and the **cast** (+ crew since
483. #28 — the whole list, see "What it does (ticket #28)"; #26 kept TMDB's first 20) (billing order; a series: all seasons' cast, roles " / "-joined), adult people left out; a person we have (by tmdbId) is linked,
484. a new one is added (people.hl `castPersonId`: name, tmdbId, free slug, `oldId tmdb-person-<id>`, this credit — their page fills
485. the rest on the first visit; in the search at once). Then `detailsAt` (ms). TMDB 404 → `detailsCheck = 'TMDB 404'`; TMDB says
486. adult → flag stored, `detailsCheck = 'adult'`, nothing downloaded. TVmaze/poster failures are logged (`detailsError`), the title
487. still counts complete (the daily sync refreshes followed titles).
488. * **On open** (`components/show.hl`): an incomplete, non-adult title renders what it has + skeletons (poster box, a cast line, 3
489. season rows for a series; the #17 shimmer). An invisible `showProbe` (CSS animation tick, like the person page — no mount hook)
490. calls the face `showComplete(slug)` (no session → no page re-mount) again and again — one step per call — until `done`; the
491. last answer (title, summary, poster, genres, cast, links, collapsed, rows) is assigned on the client, no reload. Persisted at the
492. end. Log: `details on open: "<title>" steps=… requests=… tvmazeRequests=… cast=… newSeasons=… newEpisodes=… posters=… ids=…
493. ms=…`. A second viewer drives the same job; a reload after it: complete at once.
494. * **The repair job** (`project.hl repairTick`, the app's clock — after the adult backfill is done, never during a daily run): at
495. start every incomplete non-adult title (`detailsRepairQueue`), ONE `completeStep` per tick (a series: the same title until it is
496. done), then `pauseMsAfter(step)` (260 ms per TMDB + 550 ms per
497. TVmaze request). Resumes after a restart (the queue = what is still incomplete); passing failures (no answer, 429, 5xx) go to the
498. back (≤ 2 more tries per start). Persisted every 10 titles. Log: `details repair: start, N incomplete titles`, every 100:
499. `details repair: 100/401 titles, complete=… failed=… adult=… newSeasons=… newEpisodes=… posters=… cast=… requests=… tvmazeRequests=…
500. retries=… seconds=…`, each failure, `details repair done: …`, slow steps (> 1.5 s). Titles a fill adds while the app runs are
501. completed when opened or at the next start.
502. * Real data (copy 2026-10-01, real TMDB): open → skeleton 0.18–0.26 s, complete 0.64–0.71 s (movie) / 1.26–1.36 s (series);
503. The Late Show 11.4 s in 23 steps (other requests ≤ 1.3 s). Repair: 321 titles in 507 s, 0 failed, +638 seasons, +16,528
504. episodes, 299 posters; /movies + a show page meanwhile median 0.13 s, p99 1.0 s, max 2.4 s. STATUS.md ticket #26.
506. ## What it does (ticket #28: full cast and crew)
508. Creator: "you show quite few people, usually there are more especially at picard, and directors and so". `details.hl` (what is
509. stored), `components/show.hl` + `styles.hl` (the page), `project.hl` (`creditsTick`).
511. * **Before**: the page showed `show.cast` — for 1,820 of 10,011 titles the old tracker's migrated list (Star Trek: Picard 4 names, no
512. characters shown), nothing for the rest (Superman 1978: 0); #26 added TMDB's first 20 for filmography titles only; no crew anywhere.
513. * **Stored** (details.hl `withCredits`, from a details answer with `credits` (movie) / `aggregate_credits` (series = every season)
514. appended): `cast` [{ person, character, actor, episodes, guest }] — TMDB's order (a series: by episode count; roles " / "-joined),
515. at most `TRACKER_CAST_MAX` (300; Grey's Anatomy lists 3,243, SNL 2,771), adult people left out, entries we had and TMDB does not
516. list kept after them; `guest` = fewer episodes than `mainMinOf` (half the series' episodes, ≤ 10, ≥ 1). `crew` [{ person, name,
517. job, episodes }] — job creator (series `created_by`) / director (Director) / writer (Writer) / screenplay / story / composer
518. (Original Music Composer, Composer, Music); one entry per person and job; a series' by that job's episodes, a movie's in TMDB's
519. order; ≤ 30 per job. `creditsAt` (ms). Every person linked by TMDB id; a new one gets a minimal record (people.hl `castPersonId`,
520. #26: name, slug, search entry, this credit; their page fills on the first visit).
521. * **Who stores them**: the completion of an incomplete title (#26 `completeStep`, same request), the daily sync and
522. `tools/sync-tmdb.hl` (tmdbsync.hl now asks `/tv/<id>?append_to_response=external_ids,aggregate_credits` / `/movie/<id>?…,credits`
523. — no extra request — and hands the answer back as `res.details` → `applyCredits`), the search import (`importTitle` asks ONCE with
524. ids + credits and passes that answer to `syncShowWith` — one request less than before; the face `searchImport` stores the
525. credits, log `search import credits: /shows/<slug> cast=… crew=… people=…`), and the **credits job** (`project.hl creditsTick`):
526. at start (after the adult backfill and the details repair are done, never during a daily run) every public title (`adult ==
527. false`) with a tmdbId, no `creditsAt`/`creditsCheck`, not incomplete (`creditsQueue`); ONE request per step
528. (`/movie|tv/<id>?append_to_response=credits|aggregate_credits`), then 260 ms. Resumes after a restart (the queue = what has no
529. credits); 404 / adult → `creditsCheck`; passing failures to the back (≤ 2 more tries). Persisted every 25 titles. Log: `credits job:
530. start, N titles without credits`, every 500 `credits job: 500/N titles, done=… failed=… cast=… crew=… people=… requests=…
531. retries=… slowestStepMs=… seconds=…`, failures, slow steps (> 1.5 s), `credits job done: …`. Adult titles: never. Persists only
532. shows + people (`persistCredits`, not the 16 MB episode indexes). MEMORY: before each title it reads its RSS (/proc/self/status);
533. above `TRACKER_CREDITS_MAX_MB` (16000) it logs `credits job: paused, RSS … MB > … — N titles left for the next start` and stops
534. for this process (the native runtime never frees what a fetch/JSON decode took — see STATUS #28).
535. * **`showBySlug`** (shows.hl) is a slug → id map now (one scan at the first call, a miss = one scan, remembered): the old per-call
536. scan decoded every record — 82 ms per call, twice per show page, and it grew with the stored credits (and leaked memory).
537. * **The page**: crew first (under the plot; a small label per job, the names comma-separated), then "Cast": `Name (Character)`, main
538. cast, then a "Guest stars" label and the guests. The first 20 cast names show, the rest after "Show all (N)" (→ "Show less"); per
539. crew job the first 6, the rest after "Show all crew (N)". The toggles are CLIENT-ONLY (`castClass`/`crewClass` member flip, CSS
540. hides `.more` while `collapsed`; no face). The character + comma are the link's `::after` (`data-role`) — with a wrapper element the
541. SSR's line breaks put a space before every comma. A completed #26 title gets cast + crew in its last `showComplete` answer.
542. * Real data (copy 2026-10-01, real TMDB): Picard 4 → 202 cast (17 main, then guests) + 45 crew; Superman (1978) 0 → 90 cast, Donner /
543. Mankiewicz / Puzo / John Williams; the credits job did 5,867 titles in ~51 min (0 failed; +105k people in the 2nd run), pages
544. meanwhile median 28 ms / p99 0.44 s / max 1.1 s; show page 0.1–0.2 s full load (Picard HTML 166 kB); "Show all" 2–18 ms. Costs:
545. people 15k → 160k, search index at boot 0.66 → 4.3 s, RSS after boot 1.6 → 4.9 GB. STATUS.md ticket #28.
547. ## What it does (mission 062: #26 + #28 merged into main, short ids for new people, lazy guest stars, lean watch clicks)
549. * **Merge**: branch t26 (#26 `25a50bc`, #28 `10bb3f9`) merged into main (#27 short ids, #29). Kept both sides everywhere.
550. * **Short ids for every new record** (shortids.hl `claimShortId`): shows were already covered (search import, filmography fill).
551. Persons are now covered too, in people.hl `castPersonId`. That is the only place the app creates people: #26 completion,
552. credits job, daily sync + `tools/sync-tmdb.hl`, search import, and a guest opened for the first time. A tool in `tools/`
553. finds `data/old-short-ids.json` one folder up.
554. * **Guest stars are no person records** (memory): a series' guest (`guest = true`, see #28) is stored on the title only.
555. The entry is `{ person: null (or the id when we already have them), tmdbId, actor, character, profile (TMDB profile_path),
556. episodes, guest }`. The page links it to `/person/tmdb/<tmdbId>?show=<show id>`, a function route (project.hl `guestRoute`).
557. The first open creates the person (people.hl `guestPersonId`: name + role from that title's entry, a short id, persisted),
558. logs `guest opened: <name> (tmdb <id>) → new person <id> from show <id>` and answers 302 → `/person/<slug>`. That page then
559. fills from TMDB as usual. If the person exists, the route answers 302 at once. Without `?show=`, the name comes from TMDB's
560. `/person/<id>`. An unknown id or a non-number answers 404. Guests are not in the search until opened. The main cast
561. (≥ `mainMinOf` episodes; every movie cast) and the crew still get records.
562. * **Watch / follow clicks** (components/show.hl): a face taking `session` makes hl:web re-mount the page and re-send every member
563. derived from the session (SESSION SYNC). Before, that sent the whole show record (`showRow`, with 300 cast + crew entries) and
564. the cast/crew rows. Now:
565. * `showRow` is a small object (`pageShowOf`: title, summary, poster, short id, genres, links, follow/watch state).
566. * Cast + crew derive from the slug (`castOfSlug`, `crewOfSlug`). An adult title (followers only) shows none; it never gets
567. credits.
568. * `watches.hl watchesOfUser` is kept per user until that user's next `setWatched`. It was a scan of all 11.7k watches,
569. 3× per click.
570. * The episode/season faces send their own `rows` only when the page opened/closed a season (`collapsedChanged`). Otherwise
571. the session sync's rows are the answer, so they are not sent twice.
572. * Real data: STATUS.md mission 062.
573. ## What it does (ticket #18: the daily sync by change lists, the show record, summaries)
575. Creator: "it checks every show in a request to the api? cause they have an endpoint that gives a delta on bulk" and "some shows changed
576. descriptions and titles and that did not update". `deltasync.hl` (the plan), `project.hl` `syncTick` (the run), `tmdbsync.hl`
577. (`syncShowDelta`, the TVmaze summary), `details.hl` (`withDetailsFields`, `applySynced`), `shows.hl` `summaryOf`.
579. * **The run** (04:00 UTC, as before; one step per clock tick, requests served between steps): `GET /tv/changes` and
580. `GET /movie/changes?start_date=<last run day>&end_date=<today>&page=n` (100 ids per page, all pages, the Bearer token) and
581. `GET <TVmaze>/updates/shows?since=day|week|month` — one request per step, 260 / 550 ms between them. Intersected with OUR titles via
582. the search index (TMDB id by kind, TVmaze id; no table scan) → the queue, followed titles first. Left out: no tmdbId, incomplete
583. (#26's repair does them), adult titles nobody follows; TVmaze's list brings only FOLLOWED series (TVmaze is merged for those only).
584. * **Per title**: a FOLLOWED one gets the full step as before (`syncShow`: all seasons, TVmaze merge, poster, ids); one NOBODY follows
585. the LIGHT step (`syncShowDelta`): details + ids + poster + the record, and only the seasons that can have changed — the ones we
586. don't have, the ones where TMDB counts more episodes than we do, the newest one — no TVmaze. (Real data: Today 15,716 episodes /
587. EastEnders 7,349 change daily; their full step blocked the server 20.8 s each and grew the process by ~5 GB.) A movie: 1 request.
588. * **The show record** (`applySynced` → `withDetailsFields`, the same details answer, no extra request): title (series `name`, movie
589. `title`; the slug stays), TMDB overview → `tmdbSummary`, homepage, tagline, status, genres, adult; type/release/year/language only
590. when empty; cast + crew (#28). A renamed title is re-indexed for the search at once (`searchTitleRenamed`). Empty TMDB values never
591. overwrite. TVmaze: the series request is `GET /shows/<id>?embed=episodes` now (show + episodes, one request, was `/episodes`) — its
592. summary, HTML stripped (`stripHtml`), → `tvmazeSummary`.
593. * **Summaries**: `summary` = the creator's own text, written by nothing (sync, import, person fill, completion). The page shows
594. `summary` > `tmdbSummary` > `tvmazeSummary` (`shows.hl summaryOf`). ONE TIME at start (`moveCopiedSummaries`, marker
595. `storage/mpackdb/summaries-moved.txt`): a `summary` identical to `tmdbSummary` (the migration's copy) is MOVED to `migratedSummary`
596. (mission 070: kept, shown nowhere — nothing is deleted). Live copy
597. 2026-10-01/02: **9,647 moved, 24 kept** — all 24 are followed shows whose tmdbSummary the #9 sync changed since (older TMDB texts,
598. probably no rewrites; list in STATUS #18).
599. * **The last run**: `storage/mpackdb/sync-state.txt` = the UTC day the last FINISHED run started. FULL WALK (followed shows, as
600. before) when it is missing (the first run after the deploy), more than 14 days old (TMDB's lists reach 14 days), or a change list
601. fails (then the day is not advanced). Log: `tmdb sync: start, changes since <day> (n days)` / `tmdb sync: start, N followed shows,
602. full walk (<why>)`, `tmdb sync: changes since …: tmdb tv=… movie=… tvmaze(day)=… → N of our titles (M followed), changeRequests=…`,
603. `tmdb sync done: mode=delta|full changeRequests=… shows=… … requests=… tvmazeRequests=… errors=… seconds=…`.
604. * **The tool**: `tools/sync-tmdb.hl --delta [--since YYYY-MM-DD] [--plan]` = the app's run from the command line (app stopped / a
605. copy); `--plan` only the change lists + the queue; the day is stored only by a plain `--delta` run. Without `--delta`: the full walk.
606. * Real data (copy 2026-10-01, real TMDB/TVmaze, one day): TMDB tv 3,088 + movie 9,559 changed ids (31 + 96 pages), TVmaze 312 →
607. **336 of our titles (28 followed)**; the run: 128 change requests + 477 title requests + 29 TVmaze, **511–536 s**, 0 errors, slowest
608. step 2.5 s, pages meanwhile median 45 ms / p99 0.7 s / max 2.2 s. The old full walk on the same copy: 128 shows, 259 + 128 TVmaze
609. requests, 312 s. First day only: +79 seasons / +4,475 episodes (unfollowed series never synced since the migration), 24 titles
610. renamed (TMDB's en-US name, e.g. "Star Wars: Andor" → "Andor", "Kaun Banega Crorepati(कौन बनेगा करोड़पति)" → "Kaun Banega
611. Crorepati"), 79 TMDB texts, 173 taglines. MEMORY: ~10 MB per synced title are never given back (the runtime, see #28) — a delta
612. run +3.5 GB RSS, the old full walk +2.8 GB. Details: STATUS.md ticket #18.
614. ## What it does (mission 047: checks after client-side navigation, mobile first, show page)
616. * **Invisible checks fixed** by re-vendoring Hybriel master ff51cf46 (see "Vendored Hybriel"): the old
617. build created SVG built in the BROWSER (client-side navigation, expand/collapse, a check click) in the
618. XHTML namespace — nothing drawn. Server-rendered pages were fine.
619. * **Mobile first** (`styles.hl`): every rule outside the one `@media (min-width: 40rem)` block is the
620. phone's (390 px); the block only adds the wide layout. The header is one row on a phone (brand, ident
621. selector, Log out; signed out — mission 056 — the selector + "Log in", " with ident" only from 40rem, nowrap); `/my/unwatched` + `/my/schedule` rows are two lines on a phone (show + date, the
622. episode below), one line from 40rem.
623. * **Show page** (`components/show.hl`): phone = title (+ Follow) first, then the poster at the full
624. content width, then genres/plot/cast (`showMeta` is `display: contents` there, the title row
625. `order: -1`); desktop unchanged (poster 10rem left). Season header: a caret at the right edge (since
626. ticket #17: DOWN = closed, `season-row.expanded` turns it UP). Episode rows are not indented (episode check under the
627. season check). Artists orange (`#ce9178`, token `orange`). Unwatched check = the inverted solid icon:
628. outline circle + check mark, since ticket #29 in MUTED grey (accent only under the pointer — in accent a page of
629. them read as "all ticked"); watched = filled accent disc, dark check.
630. * **Follow toggle** (signed in): "Follow" = the inverted button (accent border), "Following" = filled;
631. face `showToggleFollow` → `follows.hl` `setFollowed` (new record, `at` = now → top of `/my/shows`;
632. unfollow deletes the user's records for that show). **Signed out** the checks and Follow are visible;
633. a click opens the modal "You need to sign in to follow shows and mark episodes." (Close + "Sign in with
634. ident" = the ident login link) and sends nothing. The modal is `components/modal.hl`, copied verbatim
635. from components.hybriel.worldapi.org `components/modal/modal.hl` (becbd59, 2026-09-30) — its own
636. `Style` lands in `/__hl/app.css`.
638. ## What it does (step 13)
640. * **`/`** (`components/home.hl`): signed in, four icon tiles (inline SVG, accent) → `/my/unwatched`, `/my/schedule`,
641. `/my/shows`, `/my/movies` (2×2 on a phone, 4 in a row from 40rem). For everyone: the text about the site (signed out
642. plus a "Log in with ident" hint), then **Movies** (→ `/movies`) = the 10 latest RELEASED movies (release ≤ today) and
643. **Shows** (→ `/shows`) = the 10 series whose newest released episode is newest. Tiles: poster (`posterUrlOf`:
644. the file or `/posters/none`), title, year; a link to `/shows/<slug>` (movies too — same page, same Follow button).
645. Phone: each row scrolls sideways inside itself (`tile-row`, the page never does); from 40rem a grid of 5.
646. * **`/movies`, `/shows`** (`components/movies.hl`, `components/allshows.hl`, both compose `components/tilelist.hl`):
647. every movie by `release` desc (future-dated ones first, undated last — "all … sorted by release date desc"); every
648. series by its newest released episode (series with none released after them, by their own `release`). 24 per page,
649. grid of 3 (phone) / 6. **The page is a path segment: `/movies/page/2`**, page 1 = `/movies`; out of range → the last
650. page, not a number → page 1. NOT `?page=2` (the mission asked for it): a page component cannot read the query
651. (hybriel#11, query half open) and hl:web's Back (`popstate`) drops the query — every page change would be a full
652. reload and Back would land on page 1. With the path, paging is a client-side navigation and Back works.
653. * **Pagination** = components.hybriel's `components/pagination/pagination.hl`, **copied verbatim** to
654. `components/pagination.hl` (4b4dabe) — re-copy, don't edit. It has no data out (its README, hybriel#87), so
655. `tilelist.hl` wraps it in `pager-box { on click }`: a click on one of its buttons bubbles there, the button's text
656. ("Previous", "Next", a number; "…" nothing) gives the page → `navigate()`. Hidden when there is only one page.
657. `styles.hl` makes its buttons a bit tighter on a phone so "Previous 1 … 99 100 101 … 326 Next" stays one row.
658. * **`/my/movies`** (`components/mymovies.hl`): the movies the user follows, newest follow first (same rows as
659. `/my/shows`: small poster, title with year). `/my/shows` now leaves movies out. Shared sort: `follows.hl`
660. `insertByFollowDesc`.
661. * **Speed — `catalog.hl`**: built ONCE at start (one scan of shows + episodes: real data ~1 s, server answers
662. ~1.5 s after launch) into two sorted id lists in memory (`static index`; hand-written merge sort). A page only
663. slices 24 ids and fetches those shows: real data, server time ≈ 5 ms per page. Kept fresh without a rebuild:
664. `project.hl` `syncTick` calls `refreshCatalogShow(id)` after each synced show (its seasons/episodes only, ~2.5 ms;
665. 126 shows 0.3 s); a NEW DAY moves episodes that came out today (each series keeps its upcoming release dates) into
666. place on the first read. `tools/sync-tmdb.hl` runs with the app stopped → the next start rebuilds.
667. * **Content warning** (open, not decided here): the migrated movie table holds many adult titles (the old tracker
668. imported TMDB lists); they show on the public `/` and `/movies`. No `adult` flag is stored.
670. ## What it does (ticket #17: season caret, skeleton rows, faster opening)
672. * Caret DOWN while a season is closed, UP while open (`styles.hl` `'season-row.expanded season-caret'` rotate 180°).
673. * **Opening** a season (`components/show.hl` `on toggleCollapse`): at once, client-side, the caret turns up and
674. min(episodes, 8) **skeleton rows** (`skeleton-row`: disc, number, title bar in the episode row's shape and height; CSS
675. shimmer `@keyframes tracker-shimmer`) appear under it; then the face `showSeasonEpisodes(seasonId)` answers the
676. season's episodes `{ id, episodeNumber, title }` and `filledRows` replaces the skeletons. **Closing** is client-only.
677. Opened and closed again before the answer → the answer is dropped. An error closes the season again.
678. * Why it was slow: the old face `showRows` took `session`, and hl:web answers every session face by RE-MOUNTING the whole
679. page on the server and shipping every session-derived member as `sync` (`plugins/web/WebFramework.hl` `inbound`) —
680. slug scan, watches scan, all rows twice. `showSeasonEpisodes` takes NO session (public data): no mount, no sync.
681. * The watch state of the new rows comes from the member `watchedEpisodes` (`watchedEpisodesOf`: per season with a
682. watched episode `{ all, except }` — `all` = mostly watched, `except` = the other ids), rendered with the page and
683. re-synced after every watch face. Only this show's, never the user's watch list.
684. * An ADULT title's page (followers only) still opens seasons through the session face `showRows(slug, collapsed)`:
685. `showSeasonEpisodes` refuses seasons of adult titles (no user known). Real data has no adult title with seasons.
686. * Real data (Daily Show S30 142 ep / LWT S12 30 ep, 390 + 1280): episodes visible 380–450 / 235–390 ms (outliers 1.1 s)
687. before → 230–290 / 55–100 ms after; skeleton after 80–100 / 25–45 ms; close 420–500 / 260–390 → 135–175 / 26–45 ms. STATUS.md ticket #17.
689. ## What it does (mission 054: adult titles, the backfill)
691. * `shows.adult`: `true` / `false` from TMDB, `null` = not known yet (all 9,453 migrated rows). `shows.hl isPublicTitle` =
692. `adult == false`. Used by `catalog.hl` (the homepage rows, `/movies`, `/shows`: built from public titles only;
693. `refreshCatalogShow` adds or removes one) and `search.hl` (`titleOk[id]`; `dbSearch` skips the others — they are not
694. counted either). "Fetch from web" asks TMDB with `include_adult=false` and keeps only results with `adult: false`.
695. * Not filtered: `/my/shows`, `/my/movies`, `/my/unwatched`, `/my/schedule` (the user's own follows).
696. * The show page (mission 056, `components/show.hl adultHiddenFor`/`visibleShow`): a title with `adult == true` is "Not
697. found" for everyone who does not follow it (signed out too; no member carries its row); a follower sees it, and an
698. Unfollow on the page hides it at once. Unknown (`null`) pages open (they are in no list). The faces `showRows` /
699. `showToggle*` refuse a hidden title. Needs hybriel ≥ 73267707 for the instant hide (#122); `collapsed` derives from the
700. slug, never from the session-derived `showRow` (else it snaps back after every session face).
701. * Every TMDB answer sets the flag: the daily sync (`syncShow`/`syncMovie`), the search import (`importTitle`), the backfill.
702. * **The backfill** (`project.hl backfillTick` → `tmdbsync.hl backfillTitle`): at start the app collects every title with
703. `adult` unknown, a `tmdbId` and no `adultCheck`; then ONE title per step on the app's clock (0.1 s tick, shared with the
704. daily sync, which takes precedence — the backfill waits while a daily run is going on):
705. `GET /movie|tv/<tmdbId>?append_to_response=external_ids` → `adult`, a missing `imdbId`/`tvdbId`, the poster (w342 into
706. `posters/<oldId>.<ext>` when the file is missing — NOT for an adult title, mission 056; no TVmaze, no seasons). Then 260 ms pause per API request (≤ 40 /
707. 10 s; poster downloads go to TMDB's image host and are not counted). The lists and the search take the title at once.
708. TMDB 404 → `adultCheck = 'TMDB 404'` (stays hidden, never asked again); no answer / 429 / 5xx → back of the queue, at
709. most 2 more tries per start. Persisted every 25 titles (mpackdb's `update` is durable anyway — the gate's resume check
710. stops it after 6 titles, before any persist). A restart builds the queue from what is still unknown = it resumes.
711. * Log: `adult backfill: start, N titles without an adult flag`, every 500: `adult backfill: 500/9453 titles, adult=…
712. notAdult=… unknown=… posters=… ids=… requests=… errors=… retries=… seconds=…`, each failure, then `adult backfill done: …`;
713. with nothing to do: `adult backfill: nothing to do`.
714. * Real data (copy of 2026-10-01, real TMDB): 9,453 titles in 99 min — 3,450 adult, 5,866 not, 137 unknown (TMDB 404),
715. 8,740 posters (459 MB in total); pages during the run avg 0.05–0.16 s, max 4.9 s (a slow TMDB answer blocks). STATUS.md mission 054.
717. ## What it does (mission 058: tickets #22–#25)
719. * **#22 air dates** (`components/show.hl`): every episode row ends with its air date (`episodes.release`, `YYYY-MM-DD`
720. like /my/unwatched; none → empty) — in `buildRows` (server) and in `showSeasonEpisodes` → `filledRows` (an opened season).
721. * **#23 season check = all episodes watched** (`seasonAllWatched`): derived from the episode watches on every render, not
722. from the season's own watch record (a season without episodes still uses that record). Unchecking one episode turns the
723. season off with the face's answer (`showToggleEpisode` rebuilds the rows), checking it again turns it on. A season click
724. watches every episode when not all are watched (a partly watched season → all), else unwatches all (`setSeasonWatched`
725. still writes the season record too — nothing reads it for the check any more).
726. * **#24 movie watched check** (show page, `isMovie`): the episodes' check icon `#moviewatch` + "Watched"/"Not watched"
727. under the title (phone: above the poster). Face `showToggleMovie(slug, session)` → watch `{ targetType: 'movie', target:
728. <the title's shows.db id> }` (new target type; `tools/verify.hl` knows it). Signed out → the sign-in modal ("You need to
729. sign in to follow movies and mark them watched."). `/my/movies`: "N movies · M watched" and a solid check per watched row.
730. * **#25 `/genres/<genre>`** (+ `/genres/<genre>/page/<n>`, `components/genre.hl`, composes `tilelist.hl`): the genre by
731. `urlSegment` (`shows.hl genreBySlug`; case-insensitive fallback), its public movies AND series together, newest first,
732. 24 per page. Lists in `catalog.hl` (`index.genres`, built in the same start scan; key = a movie's release / a series'
733. newest released episode, else its release — so the date is comparable across both kinds; `refreshGenres` on every
734. `refreshCatalogShow`, `rollDay` re-keys the series). Adult/unknown hidden like /movies. Unknown genre → "Not found".
735. The show page's genre pills link `/genres/<x>`; the old `/genre/<x>` answers 301.
736. * Real data: TMDB's TV genres differ from its movie genres ("Action & Adventure", "Sci-Fi & Fantasy" vs "Action",
737. "Science Fiction") — `/genres/Action` is movies only (643), `/genres/Action-Adventure` series only (266),
738. `/genres/Drama` both (2,665). Not merged (open question for the creator).
740. ## What it does (mission 060: short ids, ticket #27)
742. Creator: "i wanted to give all our movies, shows, series and persons own short ids like a9s9a … those ids show below the
743. poster." `shortids.hl` (all of it), `project.hl` (route, backfill clock), the two pages.
745. * **Field** `shortId` on `shows.db` (movies + series) and `persons.db`: 5 characters `[a-z0-9]`. **Unique across both tables**
746. through ONE in-memory map (`shortids.hl owners`, short id → `{ kind, id }`), built at start (one scan of each table,
747. ~0.2 s real data); every id handed out goes through it. Not an mpackdb index: those are per table and cannot span two.
748. * **Old ids**: the old export had `id` on 702 of 9,453 shows (`Show.jsonl`, base62 counter `100jh`…`1012y`, all lower case).
749. `data/old-short-ids.json` (old Mongo `_id` = our `oldId` → id; made ONCE by `tools/old-short-ids.hl` from the export on
750. Loreana; committed, deployed with the code) gives those shows their old id back. All 702 are RESERVED: a new id is never
751. one of them, even where the show is gone. Persons never had one. Real data: 702 of 702 kept (Foundation `100sp`, Raised by
752. Wolves `100jh`).
753. * **New ids**: 5 random characters (hl:crypto, bytes ≥ 252 skipped so all 36 are equally likely), retried while taken or old.
754. New records get one AT CREATION (`claimShortId`): search import (`search.hl importTitle`) and the person fill's minimal
755. titles (`people.hl addMinimalTitle`) — the only places the app creates shows; persons are created only by `tools/migrate.hl`.
756. The daily sync creates seasons/episodes only.
757. * **Backfill** (records without one — at first all 25,167): in the background, its own `every(0.1)` clock in `project.hl`
758. (`shortIdTick` → `shortIdStep`): per tick ONE batch of as many records as fit into 40 ms (fetch, set `shortId`, `update`);
759. persisted every 20 batches and at the end. The queue is built at start from what is still missing → resumes after a
760. restart. A record whose id another one already holds (never seen; logged "… gets a new one") is re-queued. Log:
761. `short ids: N records, X with a short id, Y without, 702 old ids, … ms`, `short ids: backfill start …`, every 5000 `short
762. ids: N assigned, M left, s`, `short ids done: assigned=… oldKept=… batches=… slowestBatchMs=… slowestPersistMs=… seconds=…`,
763. or `short ids: nothing to do`. Chosen over "all at start": 25k records in one go would block the server ≥ 16 s (a tool on the copy: 1,564 records/s; inside
764. the app a record costs more, ~3 ms — not found out why).
765. Real data (copy 2026-10-01): **~300 s**, 1,859 batches, slowest batch 62 ms, slowest persist 31 ms (STATUS.md mission 060).
766. * **Shown** (`components/show.hl`, `components/person.hl`, `styles.hl shortIdLabel`): `#shortid` under the poster (phone: under
767. the full-width poster) and under the person's photo — no photo: under the name and facts; small (.8rem), muted, monospace,
768. centred under the image, one click selects it. None while the backfill has not reached the record yet.
769. * **`/<shortId>`** (`project.hl shortIdRoute`, the LAST route — every named one-segment route wins): 301 → `/shows/<slug>` or
770. `/person/<slug>`; upper case is read as lower; unknown / not 5 chars → 404 (plain text). An adult title's page itself then
771. says "Not found" to non-followers as before.
772. * Count on a COPY (or app stopped): `TRACKER_STORAGE=… TRACKER_OLD_SHORT_IDS=$PWD/data/old-short-ids.json ./bin/hybriel
773. tools/count-short-ids.hl` → per kind with/without/malformed, old kept/lost, duplicates across both.
775. ## What it does (mission 068: typed headings, series / shows split, tickets #20 + #21)
777. Everything (kinds, the backfill, addresses, the `/shows/<slug>` function route, colours): **[docs/kinds.md](docs/kinds.md)**.
778. Code: `shows.hl` (`kindOfTitle`, `titleKind`, `titlePath`, `typeWordOf`, `typeClassOf`), `catalog.hl` (two TV lists),
779. `tmdbsync.hl` (`kindTitle`, type stored by the sync / adult backfill), `search.hl` (import), `project.hl` (`showsRoute`, routes,
780. `kindTick`), components (headings, `allseries.hl`, `myshows.hl` for `/my/:tv`, home), `styles.hl` (`type-*`, `h1.typed`).
781. ## What it does (ticket #19: franchises and timelines)
783. Details: [docs/franchises.md](docs/franchises.md). New tables `franchises.db`, `timelines.db`, `collectionchecks.db`
784. (`franchises.hl`). `/franchises` (list), `/franchises/<slug>` (index: each timeline sortable Timeline | Release — a series by
785. its LAST episode), `/timelines/<slug>`; on every show/movie page one widget per timeline: franchise above, `‹ Prequel | name |
786. Sequel ›`. The creator (`TRACKER_EDITORS`, az5b2) edits at `/franchises/edit` and `/timelines/<slug>/edit`. A background seed
787. turns TMDB movie collections (≥ 2 of our movies) into timelines (one movie per step, ≤ 40 TMDB req / 10 s, resumes).
789. ## Test
791. THREE gates, all must pass (deploy.sh runs all three in a row on the default ports 8700–8710):
792. ```bash
793. mkdir -p /tmp/shots # TRACKER_GATE_SHOTS must exist before browser.mjs runs
794. node tests/browser.mjs # the main gate (below) — mission 070: 325 passed, 0 failed
795. node tests/kinds.mjs # #20/#21: kinds, the kind backfill, /series /shows /movies, typed headings — 32 passed, 0 failed
796. node tests/franchises.mjs # #19: franchises/timelines, seed, widget, editor; mission 070: typed headings + /movies|/series links — 50 passed
797. # a worker on other ports: TRACKER_GATE_SHOTS=/tmp/x TRACKER_GATE_PORT=8750 TRACKER_GATE_IDENT_PORT=8751 TRACKER_GATE_CHROME=8752-8756 TRACKER_GATE_TMDB_PORT=8757 node tests/<gate>.mjs
798. ```
800. ```bash
801. node tests/browser.mjs # THE GATE: this app's own server + its OWN ident (a copy of
802. # ident's code without .env, codes to a mail sink — no live
803. # ident, no real mail; tests/identkit.mjs, same as calendar's
804. # gate) + a real headless Chrome: signed out (header shows the
805. # selector + "Log in with ident"; since #13 the homepage has
806. # content, see below) -> sign in -> header shows "Log out" -> sign
807. # out -> signed out again -> a reload stays signed out. Also
808. # checks the header has no bottom border and filled buttons
809. # (incl. the ident selector's) have none, while inverted ones
810. # ("Log out") keep theirs.
811. # tracker.worldapi.org#4: a fixture show (tests/seed-show.hl,
812. # written into the gate's own storage before the server starts)
813. # proves /shows/:slug — header renders, no checks signed out;
814. # signed in, click an episode check (turns solid), click a
815. # season check (itself AND its episodes turn solid, proven
816. # after a reload — server-side, not just client state).
817. # tracker.worldapi.org#5: the same fixture (now followed by the
818. # test user) proves /unwatched — signed out, no rows; signed in,
819. # only the two already-released episodes show, newest first, the
820. # far-future one excluded; click a check, the row disappears
821. # (proven server-side after a reload).
822. # tracker.worldapi.org#6: the same fixture proves /schedule —
823. # signed out, no rows; signed in, only the far-future episode
824. # shows (the two already-released ones excluded), no check icons
825. # at all, unaffected by any watch toggle on the other pages.
826. # tracker.worldapi.org#7: the fixture now has a 2nd followed
827. # show (followed later, no poster name, never watched) and
828. # proves /my/shows — signed out, the sign-in message; signed
829. # in, newest follow first, title links, loaded posters (the
830. # placeholder), S01E02 after /unwatched's click, then S02E01
831. # (highest, not most recent) after the show-page checks; the
832. # unwatched show shows no episode.
833. # tracker.worldapi.org#8: the lists moved to /my/unwatched and
834. # /my/schedule (old addresses: 301, and the browser lands on
835. # the new one), rows say S01E02 / S02E01, titles carry the
836. # year ("Gate Test Show (2024)", "Unwatched Gate Show (2025)"
837. # — from `release`, that show has no `year`), the show page
838. # says "1 episode" / "2 episodes", /my/shows gives the show
839. # with a poster FILE (the gate writes posters/sh1.png) its
840. # own URL and the other one /posters/none.
841. # tracker.worldapi.org#9: a FAKE TMDB (tests/faketmdb.mjs, a
842. # node http server inside the gate, fixed JSON + a png; the
843. # app/tool get TMDB_BASE_URL/TMDB_IMAGE_URL + a gate token —
844. # never the real TMDB). App stopped → tools/sync-tmdb.hl: +2
845. # seasons, +4 episodes, 3 fixture episodes matched by S/E and
846. # updated, 2 posters, no season 0, Bearer token on every call;
847. # a 2nd run changes nothing. Restarted: /my/schedule and
848. # /my/unwatched show the new episodes, /my/shows + the show
849. # page the downloaded posters (/posters/sh2.png byte-equal),
850. # S1 keeps its watched episodes, no duplicates. Then the
851. # app's OWN daily run (TRACKER_SYNC_HOUR = the current UTC
852. # hour, fake TMDB slowed to 700 ms/request, one new episode):
853. # it runs, finds the episode, and pages asked meanwhile are
854. # answered < 2.5 s (between shows) though the run takes ≥ 3 s.
855. # tracker.worldapi.org#10: the PWA — head links (manifest,
856. # apple-touch-icon, favicon, theme-color = token darker), the
857. # manifest (type, name, start_url, scope, standalone, colours),
858. # every icon a real PNG of its size, touch icon 180, favicon
859. # svg + ico; Chrome: no installability error, manifest parsed
860. # without errors; the worker /__hl/sw.js registered, scope /,
861. # controls the page (a separate 390 px phone tab). OFFLINE:
862. # CDP offline → the note appears; the server STOPPED too (CDP's
863. # offline does not reach the worker's own fetches) → reload of
864. # / shows header + note from the cache; /my/shows gives
865. # "Unavailable offline"; server back → no note.
866. # mission 047: a CLIENT-SIDE navigation /my/shows → the show
867. # (window marker proves no reload): every check/caret svg in the
868. # SVG namespace and drawn — again after expand, an episode click,
869. # collapse (FAILED on the old vendored build: XHTML namespace,
870. # circle width 0 — /tmp/w047/old-build-gate.log); caret down/
871. # right and flips; episode check under the season check; artists
872. # rgb(206,145,120); unwatched icon = accent outline + accent
873. # check; Follow: "Following" filled → unfollow ("Follow",
874. # outlined) → gone from /my/shows, survives reload → follow →
875. # top of /my/shows; signed out: checks + Follow visible, a click
876. # (episode, season, Follow) opens the sign-in modal, Close
877. # closes it, a reload shows nothing was written. 390 px + 1280
878. # px: header one row, no sideways scroll, show-page order
879. # (phone: title, full-width poster, genres, plot, cast).
880. # tracker.worldapi.org#12: link icons only for existing ids
881. # (show 1: TMDB+IMDb, show 2: TMDB+TVDB, a fixture MOVIE: TMDB
882. # /movie/), exact hrefs, target _blank + rel noopener, a click
883. # opens a NEW tab (CDP target, canAccessOpener false) and the
884. # page stays; also after client-side navigation; phone/desktop
885. # position. Sync: external_ids come with the details request,
886. # the movie only via /movie/, a FAKE TVmaze (same fake server,
887. # /tvmaze/…, 301 like the real one): imdb hit, imdb 404 → thetvdb
888. # hit; afterwards the stored ids are unchanged (tt0000001, 4002
889. # — TMDB says otherwise) and the missing ones filled; 2nd run:
890. # ids=0, no TVmaze request.
891. # tracker.worldapi.org#13: fixture + 32 never-followed movies
892. # (Pager Movie 01…30 = 2001-01-01…30, one 2099, one undated) and
893. # a series with an episode 2021-01-01. Home signed out: no
894. # icons, the text, headings → /movies /shows, the 10 latest
895. # RELEASED movies (30…21, not the 2099 one), the series by
896. # newest released episode; signed in: 4 icons → /my/…, the
897. # Movies icon → /my/movies (client-side). /movies via the
898. # heading: page 1 = 2099 + 30…08, "1 2 Next"; Next →
899. # /movies/page/2 (client-side, 9 left, undated last), "1" →
900. # /movies, Back → page 2, direct load, page 99 → 2, abc → 1.
901. # /shows: series only, order, no pagination for one page; a
902. # tile → show page. Follow a MOVIE on its page → /my/movies
903. # has it, /my/shows does not. After the sync tool + restart
904. # /shows reorders (S03E01); after the IN-APP run (TMDB adds a
905. # released episode to show 2) /shows and the home row put
906. # show 2 first without a restart. Offline / from the cache
907. # still has the text and both rows.
908. # Session-sync (hybriel 64527baa): after each face-driven change
909. # (/my/unwatched click, expand, episode click, Follow) every row
910. # appears exactly once.
911. # tracker.worldapi.org#15: the fake TVmaze also serves
912. # /tvmaze/shows/<id>/episodes. First runs: lists agree with
913. # TMDB → nothing taken. Then TVmaze knows more: S3E4's title +
914. # date filled (TMDB "Episode 4"), S3E5 + a new S4 added, TMDB's
915. # S3E1 title kept, show 2 numbered differently → skipped;
916. # idempotent; /my/schedule + show page show them; TMDB catches
917. # up → adopts the rows by number (no duplicates, TMDB wins).
918. # tracker.worldapi.org#14: the header magnifier → /search
919. # (client-side); typing: 1 letter → "at least 2", "gate" →
920. # 3 titles in order with year/type/link, address /search/gate;
921. # people ("test act"); case/accents ("GATE tést"); nothing →
922. # note + Fetch; reload of the address = same results; bad %
923. # escapes → 200; "/" → space in the address. Fetch from web
924. # (fake TMDB search/multi): ours (tv 1001, movie 1003) and the
925. # person left out, w92 thumbs; signed out Add → modal, nothing
926. # sent. Signed in: Add tv 1006 (details 404) → error, every
927. # list unchanged (hybriel#121/#122); Add tv 1004 → its page
928. # with season/episodes/poster/genre/4 id links, exact TMDB
929. # calls; then in our results and gone from the web list; Add
930. # movie 1005 → /movie/ only. Screenshots search-{390,1280},
931. # search-web-{390,1280}, search-signed-out-modal-1280.
932. # Merge #13 + #14: an imported title is on /movies and /shows
933. # at once (search.hl → catalog.hl refreshCatalogShow); #13's
934. # 32 extra fixture titles are 'Pager …' so 'gate' finds only #14's.
935. # Mission 054 (adult titles): tests/seed-adult.hl adds, app
936. # stopped, 1 adult movie alice follows + 14 titles with an
937. # unknown flag (fake TMDB: 1 adult, 1 404). Backfill off:
938. # none on /, /movies (both pages), /shows, the search; the
939. # adult one on alice's /my/movies + its page; Fetch from web
940. # drops TMDB's adult result. Backfill on (fake TMDB 300 ms per
941. # answer): stopped after 6 titles, restarted → only the rest is
942. # asked (resume), pages < 1.2 s meanwhile, then the lists and
943. # the search have the non-adult ones at once, the poster is
944. # served, a 3rd start has nothing to do. Screenshot
945. # adult-home-1280.png. startServer() runs TRACKER_BACKFILL=0
946. # unless a block switches it on.
947. # tracker.worldapi.org#16: person pages (fake TMDB /3/person/<id>
948. # + combined_credits, w185 photo → fake.personRequests): a
949. # COMPLETE person at once (facts, rows, TMDB not asked); the
950. # cast link on the show page → /person/Test-Actor CLIENT-SIDE:
951. # "Loading filmography…" (TMDB slowed to 900 ms), fills without
952. # reload: facts + photo, 86 rows (87 titles − adult), order,
953. # roles merged, posters (file / TMDB w92 / placeholder), ONE
954. # person request + photo, log steps=4 added=85; bio four lines
955. # → click → all; row → show page → Back; an added title's page;
956. # second visit: complete, nothing asked/added; Second Actor:
957. # linked, added=0, our adult record hidden though TMDB's credit
958. # is not; search finds the added title once; 390/1280 no
959. # sideways scroll; screenshots person{,-complete}-{390,1280}.
960. # Mission 056: signed-out header ONE row at 320/390/1280
961. # ("Log in" on a phone, "Log in with ident" wide; shots
962. # header-signed-out-{320,390,1280}.png); the backfill does not
963. # download the adult title's poster; /shows/adult-gate-movie is
964. # "Not found" signed out (no title in the HTML), alice sees it,
965. # her Unfollow on the page hides it at once (+ reload,
966. # /my/movies); an episode check on a normal show page toggles
967. # and back. Every stopServer() leaves the tab on about:blank first
968. # (else websocket reconnect errors fail the console check).
969. # #17: caret up (open) / down (closed) and flips; opening with the
970. # face held back 1.5 s (`window.__delayFace`, an init script that
971. # delays that websocket frame): 2 skeleton rows under season 1,
972. # episode-row shape/height, shimmer animation, caret already up,
973. # no episode yet (390 + 1280, shots show-skeleton-*); after the
974. # answer: skeleton gone, every row once (hybriel#121), watch state
975. # right (show-open-*, show-closed-*); closed while loading stays
976. # closed; only showSeasonEpisodes is sent (wsSent), closing sends
977. # nothing.
978. # mission 058: #22 air dates (server + opened season), #23 season
979. # check off/on live + after reload + partly watched → all, #24 the
980. # movie check signed out (modal) / in (click, reload, uncheck) +
981. # /my/movies count, #25 /genres/Action (2 pages, series + movies,
982. # adult hidden, pill → /genres/Drama, /genre/ 301, imports at once).
983. # Fixture: genre Action on the Pager titles + adult "Hidden Action Film".
984. # mission 060 (short ids): the gate's own old-id file (sh1 → 100jh,
985. # 'gone-old' → 100ji reserved); the first start's background
986. # backfill (oldKept=1); a search import has its id at once;
987. # #shortid under the poster/photo (person without photo: under
988. # the facts), monospace, muted, smaller, 390 + 1280 (shots
989. # shortid-{show,movie,person,person-nophoto}-{390,1280}); the
990. # fill's new titles have one (the run's backfill was done
991. # before); /<id> → 301 (show, movie, person, upper case),
992. # unknown / reserved / 3 chars → 404, the browser lands on the
993. # page; tools/count-short-ids.hl (app stopped): all have one,
994. # 0 duplicates; RESUME: tests/seed-shortids.hl adds 60 persons
995. # without + 2 sharing aaaa1, a start with a 1 ms budget is
996. # stopped part way, the next does only the rest, one of the two
997. # keeps aaaa1, the other gets a new one, a 3rd start: nothing.
998. # #26: Person Film One (opened from Test Actor's filmography,
999. # client-side, face held back 1.5 s): skeleton (poster, cast
1000. # line), TMDB not asked yet → complete without reload: poster,
1001. # IMDb, cast (Test Actor linked, a NEW person, adult person left
1002. # out); ONE details request; reload complete; the new person's
1003. # page. Person Series: 3 skeleton season rows → in 4 steps (gate:
1004. # TRACKER_DETAILS_PART_EPISODES=1, a season per step) 2 seasons
1005. # with TMDB + TVmaze episodes, 4 links, cast from aggregate_
1006. # credits; open/close; the search finds the new person. Repair job (fake TMDB
1007. # 60 ms, TRACKER_REPAIR_FAST): 82 titles, stopped after 6,
1008. # restart resumes, 404s marked, adult never asked, /movies < 1.2 s
1009. # meanwhile, a 3rd start has nothing to do. Shots
1010. # title-skeleton-1280, series-skeleton-1280.
1011. # #28: the sync tool stores the credits from its details request
1012. # (show 1: cast 23 / crew 7 / +25 people; the movie 3 / 13 / +15;
1013. # the 2nd run adds nobody). Movie page: TMDB cast order with the
1014. # characters, crew by job (a director listed twice: once; 9
1015. # writers → 6 + "Show all crew (13)", client-side, no face).
1016. # Series page: 23 of all seasons by episodes (Test Actor linked,
1017. # not added twice, adult left out), the later-season actor,
1018. # collapsed to 20 + "Show all (23)" (no comma after the 20th) →
1019. # all + "Guest stars" (no face sent), "Show less"; crew Created
1020. # by / Directed by (by episodes) / Written by / Story / Music; a
1021. # new person's page; the completed #26 title's crew; the search
1022. # import's cast + crew. Credits job (fake TMDB 60 ms,
1023. # TRACKER_CREDITS_FAST): TRACKER_CREDITS_MAX_MB=1 pauses it before
1024. # the first title; stopped after 6, resumes, done, /movies
1025. # < 1.2 s meanwhile, a done title's cast + crew, a 3rd start has
1026. # nothing to do. Shots credits-movie-1280, credits-series-
1027. # {collapsed,all}-390, credits-series-1280.
1028. # mission 062 (merge of #26/#28 + 3 fixes): count-short-ids after
1029. # the sync tool and after the credits job (app stopped): every
1030. # person has a short id, 0 duplicates; the #26 new cast member,
1031. # the import's Imported Lead / Import Creator have one at once;
1032. # the next start: "short ids: nothing to do". Guests: Guest Star
1033. # One/Two link /person/tmdb/<id>?show=<id>, no person page before;
1034. # a click creates Guest Star One (short id, filled from TMDB:
1035. # Gate Test Show / Stranger), shots guest-person-{390,1280}; the
1036. # link again → 302, no 2nd person; without ?show= the name from
1037. # TMDB /person/3141; /person/tmdb/999999 + /abc → 404. A watch
1038. # click's ack + session sync: rows, NOT castRows/crewRows/names,
1039. # < 20 kB; cast + crew still there after two clicks; after a
1040. # client-side navigation a watch click answers only ok (rows by
1041. # the session sync), flips + back, every row once.
1042. # tracker.worldapi.org#18: the first start clears the seed
1043. # movie's copied summary (marker file); the first in-app run is
1044. # the full walk and stores its day. At the end: fake change
1045. # lists (2 ids per page) + TVmaze updates, day = yesterday →
1046. # the in-app run asks 2+2 pages + TVmaze, syncs ONLY 1001,
1047. # the movie (followed) and Person Series (light: details +
1048. # season/2 only), never 1002/1004; renamed titles on their old
1049. # slugs, the creator's summary kept, the movie TMDB's new text,
1050. # Person Series' new episode, search by the new name; records
1051. # via tests/peek-shows.hl (tmdbSummary, tagline, tvmazeSummary
1052. # HTML-stripped, untouched tmdbSync); a 503 change list → full
1053. # walk, day kept; 20 days → full walk; TVmaze text shown when
1054. # there is no other. Shots delta-{show,movie}-1280.
1055. # 323 checks (#13 + #14 + #15 + 1 merge check + 17 mission 054 + #16: 19 + 5 mission 056 + 6 #17 + 21 mission 060 + 17 #26 + 18 #28 + 7 mission 063 + 10 mission 062 + 12 #18). Own servers :8700 (this app) / :8701 (ident copy)
1056. # / :8710 (fake TMDB); own storage .scratch/gate-store; Chrome
1057. # on 8702-8709. Other ports (workers get 8720-8739):
1058. # TRACKER_GATE_PORT=8720 TRACKER_GATE_IDENT_PORT=8721 TRACKER_GATE_CHROME=8722-8726 TRACKER_GATE_TMDB_PORT=8727 node tests/browser.mjs
1059. # TRACKER_GATE_SHOTS=/tmp/x also writes {home,show,my-shows,
1060. # my-unwatched,my-schedule,movies,movies-page-2,shows-list,
1061. # my-movies,genre-action,genre-action-page-2,movie}-{390,1280}.png,
1062. # movie-watched-{390,1280}.png, movie-signed-out-modal-1280.png, signed-out-modal-
1063. # 1280.png, synced-*.png (after the sync, 1280×900; synced-show-390
1064. # + synced-movie-390 at 390) + pwa-phone-
1065. # online/offline.png (390 px, dpr 2), show-{skeleton,open,closed}-
1066. # {390,1280}.png (#17) — LOOK at them
1067. node tests/kinds.mjs # mission 068 (#20, #21): its own gate — own storage .scratch/kinds-store, own ident, fake TMDB
1068. # with TV types (tests/seed-show.hl + tests/seed-kinds.hl): kinds from stored data, the daily
1069. # sync storing the type, the kind backfill (stop + resume, pacing, 404, nothing to do), the
1070. # two lists, 301s, every heading's words + computed colours, the function-rendered show page
1071. # (season, Follow), home rows/tiles, /my/series + /my/shows, search labels + a talk-show
1072. # import → /shows/<slug>, person labels, 390/1280 no sideways scroll + the long name wrap.
1073. # TRACKER_GATE_SHOTS=/tmp/w068/shots TRACKER_GATE_PORT=8761 TRACKER_GATE_IDENT_PORT=8762 TRACKER_GATE_CHROME=8763-8765 TRACKER_GATE_TMDB_PORT=8766 node tests/kinds.mjs
1074. # → 32 passed. Shots {series-page,show-page,movie-page,genre,person,series-list,shows-list,
1075. # movies-list,home,my-series,my-shows,search}-{390,1280}.png — LOOK at them.
1076. # The shots dir must exist for tests/browser.mjs (mkdir -p first).
1077. ps -eo pid,args | grep [h]l-browser-tier # must print nothing afterwards
1078. ```
1080. ## Deploy (Byrodin)
1082. Target: `/CONTAINERS/projects/tracker.worldapi.org` on Byrodin, container
1083. `tracker.worldapi.org` (`docker-compose.yml`: debian:12-slim, host network,
1084. `HL_HOST=127.0.0.1`, `TRACKER_PORT=45008`, `TRACKER_WATCH=0`, the folder mounted at
1085. `/home/tracker`, `./bin/hybriel project.hl`), public https://tracker.worldapi.org/ via
1086. nginx (TLS ends there; no baseUrl/tls in the app, like ident/notes).
1087. * **First deploy: done by the architect** (folder, nginx vhost with WebSocket Upgrade
1088. headers, cert, DNS, **and registering this app in the live ident** — `/apps` → name +
1089. origin `https://tracker.worldapi.org` → the API key + secret go into `.env` next to
1090. `docker-compose.yml`, `TRACKER_KEY=… TRACKER_SECRET=…`).
1091. * **Later: `./deploy.sh`** on Loreana, in this folder: runs the gate (refuses on a failure;
1092. `--skip-tests` skips it LOUDLY), backs up `storage/`/`.sessions/`/`.env` (what exists) to
1093. Loreana's `/media/SLOW1TB2/deploy-backups/<app>/` (newest 5 kept; an empty/failed backup stops
1094. the deploy), then rsyncs the code to
1095. `[email protected]:/CONTAINERS/projects/tracker.worldapi.org` (never `storage/`,
1096. `.sessions/`, `.env`, `.scratch/`, `server.*`, logs — the preview is checked for them; no
1097. `--delete`), `docker compose up -d && docker compose restart` over `ssh -F /dev/null`,
1098. then waits for https://tracker.worldapi.org/ to answer 200.
1099. * **Deploy of mission 062** (#26 + #28 first time live): after the start the details repair runs ~14 min (557 titles), then
1100. the credits job ~50 min (5,867 titles). Watch `docker stats` and the log (`details repair …`, `credits job …`). Real-data
1101. copy: people 15k → 102k. RSS after boot 4.3 → ~8 GB, swinging 1.2–8.8 GB during the jobs. One step was ~5 s, so pages waited
1102. that long once. `TRACKER_CREDITS_MAX_MB` (16000) pauses the job.
1103. * **Deploy of mission 070** (#18 + #20/#21 + #19 + Hybriel 8efba065 first time live; measured on a copy of live `storage/` of
1104. 2026-10-02 with real TMDB): start ~1 min (search index; ONE TIME the summaries move, 9,647 → `migratedSummary`, ~16 s); adult
1105. backfill / repair / credits have nothing to do; **kind backfill ~16–21 min** (1,789 TV titles, 1,522 requests), then the
1106. **collection seed ~32–37 min** (4,774 movies, 4,957 requests, 110 timelines) — ~50 min in all. No `sync-state.txt` yet → the
1107. first 04:00 UTC run is the full walk; from the next day the delta run (~6–10 min). Log: `kind backfill done`, `collections done`.
1108. **Memory**: RSS after boot 2–4 GB; during the jobs up to 11.8 GB (swinging back to 1–3 GB). After the jobs the process keeps
1109. ~25 MB per page load (3 → 19 GB in 600 loads) — **restart the container once after `collections done`** (`docker restart
1110. tracker.worldapi.org`); a fresh process keeps ~3.5 MB per load, after one daily delta run ~8 MB → a daily restart after the
1111. 04:00 run is advisable (not built). The old binary kept ~21 MB per load from the start. `/my/unwatched` takes ~5 s (old binary
1112. 1.2 s): Hybriel's GC by bytes (`HL_GC_BYTES`, README "Config"). Watch `docker stats`. STATUS.md mission 070.
1113. * `./deploy.sh --dry-run` = the gate + `rsync -n` + the commands it would run (no restart, no
1114. URL check). `--target DIR|HOST:DIR` and `--url URL` point it elsewhere (tested against a
1115. local directory, see STATUS.md).
1117. ## Data
1119. `storage/mpackdb/`: `users.db` (`identity` → this app's own user id, `users.hl`) plus, since
1120. step 2, the old tracker's data — `genres.db` (27), `persons.db` (15157; #16 adds `biography`, `tmdbProfile`, `adult`, `filmographyAt` and TMDB credits in `shows`), `shows.db` (9453),
1121. `seasons.db` (6430), `episodes.db` (231584), `follows.db` (128), `watches.db` (11692), all with
1122. fresh mpackdb ids and every reference re-pointed (`tools/migrate.hl`); the one old user is this
1123. app's user for ident short id `az5b2`. Since step 9 the TMDB sync adds seasons/episodes and
1124. `storage/mpackdb/posters/<oldId>.<ext>` (w342, ~50 KB each), and sets `tmdbSync`/`tmdbPoster` on
1125. shows. #18: `shows.tvmazeSummary` (TVmaze's text, HTML stripped), `summary` = the creator's own text only;
1126. `sync-state.txt` (the day of the last finished daily run), `summaries-moved.txt` (the one-time move ran: counts). Mission 070 (nothing migrated is destroyed): `shows.migratedSummary` (the copied summary), `migratedTitle` / `migratedGenres` / `migratedHomepage` / `migratedTagline` (a MIGRATED record's value before TMDB's first replaced it; set once), seasons/episodes `migratedTitle` / `migratedSummary` (same rule) — kept, shown nowhere. Mission 054: `shows.adult` (true/false/null = unknown → hidden from lists + search) and `shows.adultCheck` (why the
1127. backfill gave up on a title, e.g. `TMDB 404`). Mission 060: `shows.shortId` and `persons.shortId` (5 × `[a-z0-9]`, unique across
1128. both; `data/old-short-ids.json` holds the old tracker's 702). #26: `shows.minimal` (added by a filmography), `detailsAt` (ms, complete),
1129. `detailsCheck` (`TMDB 404` / `adult`: never asked again), `detailsError` (a TVmaze/poster failure while completing); people added
1130. from a cast: `oldId tmdb-person-<id>`, `added` (ms). #28: `shows.cast` [{ person, tmdbId, actor, character, profile, episodes,
1131. guest }] — mission 062: a guest's `person` is null until someone opens them (`/person/tmdb/<id>`); `shows.crew`, `creditsAt`,
1132. `creditsCheck`. Known in the migrated data (not caused by the sync): 12 duplicate episode tmdbIds and 1
1133. episode whose `show`/`season` disagrees among the followed shows. Counts and the referential-integrity proof:
1134. `STATUS.md` "ticket #2". Real data 2026-10-01: 7,814 movies + 1,639 series (`shows.type`), 232,384 episodes (935
1135. series have a released one).
1137. ## Files
1139. | File | |
1140. |---|---|
1141. | `CONCEPT.md` | the creator's concept — do not edit |
1142. | `project.hl` | manifest: the installable app (`appIcons`, `appTouchIcon`, `appFavicon`, theme colours from the tokens, `offline = [ Home ]`, #10), the daily TMDB run (`syncTick`, #9), routes (`/login/callback`, `/login/failed`, `/login.js`, `/posters/:name`, `/movies` + `/movies/page/:page`, `/shows` + `/shows/page/:page` (#13), mission 068: `/movies/:slug`, `/series` + `/series/page/:page` + `/series/:slug`, `/shows/:slug` = function `showsRoute` (show rendered, series/movie → 301), `/my/:tv` (series | shows), `/my/movies` (#13), `/genres/:genre` + `/genres/:genre/page/:page` + `/genre/:genre` → 301 (#25), `/my/unwatched`, `/my/schedule`, `/unwatched` + `/schedule` → 301, `/search/:q?` (#14), `/person/:slug` + `/profiles/:name` (#16), `/` component Home, `/:sid` → 301 (mission 060, last)), the app's own session cookie; #14: builds the search index at boot (`ensureIndex`); mission 054: the adult backfill (`backfillTick`) on the clock it shares with the daily run; mission 060: the short-id map at boot (`ensureShortIds`) and its backfill (`shortIdTick`, own clock); mission 068: the kind backfill (`kindTick`, shared clock, after the adult backfill) |
1143. | `users.hl` | the ident exchange + the `usersTable`, copied from calendar.worldapi.org's `users.hl` |
1144. | `login.js` | bridge between `<ident-selector>`'s `ident-login` event and the shell, copied verbatim from calendar.worldapi.org |
1145. | `components/home.hl` | `/` (#13): signed in the icon tiles (5 since mission 068); the text; the latest 10 movies, series and shows (catalog.hl) |
1146. | `components/movies.hl`, `components/allseries.hl`, `components/allshows.hl` | `/movies`, `/series`, `/shows` (+ `/page/:page`): every movie / series / show, 24 per page (#13, #21) |
1147. | `components/tilelist.hl` | the poster grid + pagination both lists compose; turns a pagination click into `navigate()` (#13) |
1148. | `components/pagination.hl` | the pagination, copied verbatim from components.hybriel.worldapi.org (`components/pagination/pagination.hl`, 4b4dabe) — re-copy, don't edit |
1149. | `components/mymovies.hl` | `/my/movies`: followed movies, newest follow first (#13); watched count + check per watched movie (#24) |
1150. | `components/genre.hl` | `/genres/<genre>` (+ `/page/<n>`): a genre's movies and series, newest first, 24 per page (#25, mission 058) |
1151. | `catalog.hl` | the public lists: built once at start, refreshed per synced show and per new day (#13); only `adult == false` titles (mission 054); per-genre lists (#25) |
1152. | `components/main.hl` | the shell (header: brand, `<ident-selector>`, log in / log out; the offline note + its `netProbe` tick, #10) |
1153. | `components/loginfailed.hl` | `/login/failed`: why a login didn't work |
1154. | `components/show.hl` | `/series/:slug`, `/movies/:slug`, `/shows/:slug` (via `showsRoute`): the title page, heading "<Type> \| <title>" (#20) (header, Follow toggle, seasons with caret, episodes, watch check; signed out the sign-in modal) |
1155. | `components/modal.hl` | the modal dialog, copied verbatim from components.hybriel.worldapi.org (`components/modal/modal.hl`, becbd59) — re-copy, don't edit |
1156. | `components/unwatched.hl` | `/my/unwatched`: unwatched, already-released episodes of followed shows, newest first |
1157. | `components/schedule.hl` | `/my/schedule`: not-yet-released episodes of followed shows, soonest first, no checks |
1158. | `components/myshows.hl` | `/my/series` and `/my/shows` (`/my/:tv`, #21): followed series / shows (#13: movies → `/my/movies`), newest follow first — small poster, title, last watched `SxxEyy` |
1159. | `shows.hl` | read-only data access (shows/seasons/episodes/genres/persons; #28: `showBySlug` via a slug → id map; #18: `summaryOf`; #21: the kind of a title `titleKind`/`kindOfTitle` and its page `titlePath`), plus the list labels `episodeCode`/`titleWithYear`/`posterUrlOf` (#8), reused by the `/my/` pages; `externalLinksOf` (#12: the link icons); `isPublicTitle` (mission 054: `adult == false`) |
1160. | `follows.hl` | per-user follow data (`followsOfUser`, `followedShowIds`), reused by `/unwatched`/`/schedule`/`/my/shows`; `allFollowedShowIds` (#9: every show anyone follows, for the sync); `isFollowing`/`setFollowed` (mission 047: the show page's Follow toggle) |
1161. | `deltasync.hl` | #18: the daily run's plan — the change lists (`planStart`/`planStep`/`planIds`/`planLine`), the last run's day (`sync-state.txt`), `isFollowedInRun`; the one-time `moveCopiedSummaries` (mission 070: to `migratedSummary`) |
1162. | `tests/peek-shows.hl` | #18: the gate reads a title's description fields (`TRACKER_PEEK`; mission 070: + `migratedTitle`/`migratedSummary`/`migratedGenres` count, and a title's episodes with `TRACKER_PEEK_EPISODES`), or clears summary + tmdbSummary of one (`TRACKER_CLEAR_TEXTS`) — app stopped |
1163. | `tools/count-migrated.hl` | mission 070: how many shows/seasons/episodes keep a migrated value the sync replaced (`migrated*`) + example titles — ONLY on a copy or the stopped app |
1164. | `tools/ref-params.py`, `tools/lambda-audit.py` | mission 070 (hybriel#48): `python3 tools/ref-params.py . [--apply]` gives every read-only lambda parameter `&`; `python3 tools/lambda-audit.py .` lists the lambdas that change a parameter (a copy since 8efba065) |
1165. | `tests/realdata-070-{run,sampler,load200,bench}.sh` | mission 070, NOT the gate: the app (or the old binary's tree) on a COPY under `.scratch/w070/real-<name>/` with all jobs + a delta run, RSS every 10 s, 200 signed-in loads, a fresh-process bench (STATUS.md mission 070 "How to repeat") |
1166. | `tests/realdata-070.mjs` | mission 070, NOT the gate: on a COPY of live `storage/` after the jobs (STATUS.md mission 070), signed in, 390/1280: every page kind (lists, `/franchises`, a franchise + timeline, a movie with the widget, a series with cast, a show, `/my/*`, search) — full load, heading, sideways scroll, screenshot, console problems |
1167. | `tests/realdata-018.mjs`, `tests/realdata-018-poll.mjs` | #18, NOT the gate: screenshots/times of refreshed titles on a COPY; page latency while the daily run works (STATUS.md #18) |
1168. | `tmdbsync.hl` | the TMDB sync of one show (`syncShow`: seasons, episodes, poster; #18: `syncShowDelta` (the light step), `tvmazeSummary`, `stripHtml`; #26: `syncShowWith`/`syncShowPart` = with the details answer given, in parts; #12: external ids + TVmaze lookup, movies via `/movie/`; #15: TVmaze episode merge `mergeTvmaze`) + run totals — used by `project.hl`'s daily run and `tools/sync-tmdb.hl` (#9); mission 054: `adultOf`, the adult backfill (`adultBackfillQueue`, `backfillTitle`) |
1169. | `watches.hl` | per-user watch state (episode/season/movie (#24) checks, the shared check-icon class `watchClassOf`, `watchedSet` = the user's watches as a map, built once per page; mission 062: `watchesOfUser` kept per user until their next write), reused by `/unwatched`/`/schedule`/`/my/shows` |
1170. | `icons/` | #10: `icon.svg` (source) → `icon-192.png`, `icon-512.png`, `apple-touch-icon.png`; `favicon.svg`, `favicon.ico` (commands in "What it does (step 10)") |
1171. | `styles.hl` | all CSS (imports the tokens from `shared/tokens.hl`; accent green `#4ec9b0`) |
1172. | `shared/tokens.hl` | the WorldAPI tokens, vendored verbatim from `ident.worldapi.org/shared/tokens.hl` |
1173. | `tests/browser.mjs` | the gate (above) |
1174. | `tests/kinds.mjs`, `tests/seed-kinds.hl` | mission 068: the gate of #20/#21 and its TV fixtures (Talk/Reality/News genres, alice's follows) |
1175. | `tests/realdata-068.mjs` | mission 068, NOT the gate: on a COPY of live `storage/` after the kind backfill: spot checks by name (search label + link), every page kind at 390/1280 signed in (time, heading, sideways), screenshots |
1176. | `docs/kinds.md` | mission 068: kinds, the kind backfill, addresses, the `/shows/<slug>` function route, heading colours |
1177. | `tests/seed-adult.hl` | mission 054: the gate's adult/unknown-flag titles + alice's follow of the adult one, written with the app stopped |
1178. | `tests/seed-show.hl` | writes the gate's fixture show (two seasons, three episodes) and, since ticket #5, a follow of it for the test user (#7: plus a 2nd, later-followed, never-watched show; #12: partial external ids + a MOVIE followed by another user), before the server starts |
1179. | `tests/identkit.mjs` | starts a throwaway ident copy for the gate, copied from calendar.worldapi.org's `tests/` |
1180. | `tests/faketmdb.mjs` | the gate's fake TMDB (#9): details, `append_to_response=season/N`, w342 png; records every request; #12: `external_ids`, `/3/movie/<id>`, a fake TVmaze under `/tvmaze` (`tvmazeRequests`); #15: `/tvmaze/shows/<id>/episodes` from `tvmaze.episodes`; #26/#28: `credits` / `aggregate_credits`, a series' `created_by`; #18: `/3/tv|movie/changes` (paged, `changesFail`), `/tvmaze/updates/shows`, `/tvmaze/shows/<id>?embed=episodes` + `summaries` |
1181. | `tests/realdata-check.mjs` | NOT the gate: the real-data browser check on a COPY of live `storage/` (STATUS.md, mission 047 / #13): times of every page incl. `/`, `/movies`, `/shows`, `/my/movies`, pagination clicks + Back, client-side navigation, screenshots to `$REAL_SHOTS`; env REAL_PORT / REAL_CHROME / REAL_OUT / REAL_SHOW (#15) |
1182. | `search.hl` | #14: the search — in-memory index (`ensureIndex`, `dbSearch`), `queryOfParam`, TMDB `webSearch`, `importTitle` (new record + `syncShow`); #18: `titleIdByTvmaze`, `searchTitleRenamed` (stale entries skipped) |
1183. | `components/search.hl` | #14: `/search` + `/search/<text>` — the field, our titles/people as you type, "Fetch from web", "Add" (signed out: the sign-in modal) |
1184. | `people.hl` | #16: person by slug, facts, filmography rows, the step-by-step TMDB fill (`personFillStep`); #26: minimal titles get `minimal = true`, `castPersonId` (a cast member → our person, new ones added, with a short id — mission 062); mission 062: `guestPersonId` (a guest star → a person on the first open) |
1185. | `components/person.hl` | #16: `/person/<slug>` — photo, facts, bio, filmography; "Loading filmography…" + the fill without reload |
1186. | `tests/realdata-search.mjs` | #14, NOT the gate: the search on a COPY of live `storage/` with the real TMDB (STATUS.md "ticket #14"): keystroke times, web search, ONE import, screenshots |
1187. | `tests/realdata-person.mjs` | #16, NOT the gate: person pages on a COPY of live `storage/` with the real TMDB (STATUS.md "ticket #16"): cast click → filled times, other requests' wait, second visit, screenshots |
1188. | `tests/realdata-056.mjs` | mission 056, NOT the gate: the merged app on a COPY of live `storage/` after the adult backfill (STATUS.md mission 056): every public page type signed out + /my/* signed in at 390/1280 (times, header one row, screenshots), all list/search/person links → `slugs.txt`, an adult page signed out/in |
1189. | `tools/count-external-ids.hl` | #12: how many shows have each external id, per type, all / followed — ONLY on a copy |
1190. | `tools/count-duplicate-ids.hl` | duplicate `@id`s per table — ONLY on a copy (hybriel#113 check, mission 047) |
1191. | `tests/realdata-058.mjs` | mission 058, NOT the gate: on a COPY of live `storage/` (STATUS.md mission 058), signed in, 390/1280: full loads of a show, a movie, `/genres/<g>` + page 2, `/my/movies`; air dates, season check == its episodes, genre kinds, one movie-check click + reload (and back) |
1192. | `shortids.hl` | mission 060 (#27): the short ids — old ones from `data/old-short-ids.json`, the map over shows + persons (`ensureShortIds`), `claimShortId` for new records, the backfill batch `shortIdStep`, `shortIdPath` for `/<shortId>` |
1193. | `data/old-short-ids.json` | mission 060: the old tracker's 702 short ids (`oldId` → id), made by `tools/old-short-ids.hl`; deployed with the code |
1194. | `tools/old-short-ids.hl` | mission 060, one-off: the old export's `Show.jsonl` → `data/old-short-ids.json` (reads only the export) |
1195. | `tools/count-short-ids.hl` | mission 060: short ids counted (with/without, old kept/lost, duplicates across shows + persons) — ONLY on a copy or app stopped |
1196. | `tests/seed-shortids.hl` | mission 060: the gate's resume/duplicate fixture (60 persons without a short id, 2 sharing `aaaa1`), written with the app stopped |
1197. | `tests/realdata-063.mjs` | ticket #29, NOT the gate: on a COPY of live `storage/` (STATUS.md ticket #29), signed in, 390/1280: every check's class + computed look on `/my/unwatched`, a show page, `/my/movies`; screenshots; one `/my/unwatched` click + reload |
1198. | `tests/realdata-060.mjs`, `tests/realdata-060-backfill.sh` | mission 060, NOT the gate: on a COPY of live `storage/` (STATUS.md mission 060): page times + short id under the image + `/<id>` redirects at 390/1280; the backfill run timed with curl |
1199. | `tests/realdata-057.mjs` | ticket #17, NOT the gate: on a COPY of live `storage/` (STATUS.md #17), signed in, 390/1280: opening/closing seasons of big shows timed in the page (skeleton shown, rows there, skeleton gone), skeleton count, faces sent; then client-filled watch state == server-built |
1200. | `details.hl` | #26: a title's full details — `isIncomplete`/`needsDetails`, `completeStep` (details + cast, then a series' parts via `syncShowPart`), the repair queue; #28: cast + crew from a details answer (`castOf`, `crewOf`, `withCredits`, `applyCredits` for the import), the credits job's `creditsQueue`/`creditsStep`; mission 062: guests stored on the title, no person record (`mergedCast`); #18: `withDetailsFields` (the record's fields, shared with the completion), `applySynced` (the daily sync: fields + credits) |
1201. | `tests/realdata-062.mjs` | mission 062, NOT the gate: on a COPY of live `storage/` (STATUS.md mission 062), signed in, 390/1280: a series + a movie (full load, HTML size, cast/crew/guest counts, screenshots), a guest star click → its new person page (time, screenshot), then 6 watch + 2 follow clicks on Picard and The Simpsons (ms + bytes received each) |
1202. | `tests/realdata-028.mjs` | #28, NOT the gate: on a COPY of live `storage/` (STATUS.md #28): given titles at 390/1280 — full load, HTML size, cast/crew as shown, "Show all" time (faces sent: 0), screenshots collapsed/all |
1203. | `tests/realdata-026.mjs` | #26, NOT the gate: on a COPY of live `storage/` with the real TMDB (STATUS.md #26): person page → an incomplete movie/series → skeleton / complete / poster times, /movies latency meanwhile, reload, screenshots; `REAL_TITLES` opens given slugs directly |
1204. | `tools/check-public-slugs.hl` | mission 056: the slugs `tests/realdata-056.mjs` collected → adult true/false/unknown counts, a user's followed adult titles, poster files of adult vs other titles — ONLY on a copy |
1205. | `franchises.hl` | #19: franchises, timelines (tables, `byShow`), the pages' data, the widget (`widgetsOf`), the editor operations (`isEditorSession`), the TMDB collection seed (`seedStep`) — docs/franchises.md |
1206. | `components/franchises.hl`, `components/franchise.hl` | #19: `/franchises`; `/franchises/<slug>` + `/timelines/<slug>` (one component, sort toggle client-side) |
1207. | `components/franchisewidget.hl` | #19: the `‹ Prequel | Timeline | Sequel ›` widget, composed by `components/show.hl` |
1208. | `components/franchiseedit.hl`, `components/timelineedit.hl` | #19: the creator's editor (`/franchises/edit`, `/timelines/<slug>/edit`) |
1209. | `tests/franchises.mjs`, `tests/seed-franchises.hl` | #19: the franchise gate + its 7 collection movies |
1210. | `tests/realdata-066.mjs` | #19, NOT the gate: on a COPY after the seed: `/franchises`, matching timelines (REAL_MATCH), a movie's widget, the creator's editor; times + screenshots |
1211. | `tests/cdp.mjs`, `tests/ports.mjs` | the CDP browser driver, copied from `ident.worldapi.org/tests/` |
1212. | `docker-compose.yml`, `Dockerfile`, `deploy.sh` | Byrodin container; the deploy from Loreana (section "Deploy") |
1213. | `tools/migrate.hl` | one-off: old MongoDB export → `storage/mpackdb/` (step 2, idempotent, see `STATUS.md`) |
1214. | `tools/verify.hl` | one-off: proves the migrated data against a COPY of `storage/mpackdb/` (counts, zero dangling references, one show end to end) |
1215. | `tools/sync-tmdb.hl` | the TMDB sync from the command line (#9) — app stopped or a copy; `--limit N`; #18: `--delta [--since DAY] [--plan]`; prints counts |
1216. | `tools/relink-episode-seasons.hl` | one-off: links episodes with no `season` (step 2 leftovers) by `(show, seasonNumber)` — not run against the live data yet, see "What it does (step 4)" |
1218. ## Vendored Hybriel
1220. `bin/hybriel` + `plugins/` = **hybriel master 8efba065** (mission 070, 2026-10-02; includes #126 f685f240 — the GC also collects
1221. by bytes, `HL_GC_BYTES` — and #48 4371b7aa — a lambda PARAMETER copies its argument, `&p` = a reference; also #112 #117 #119
1222. #120, nothing to adopt), `bin/hybriel` sha256 `50361e954e317a819aa58adfc2e3553f0b4b7adc82abd1fad79385d33b3d5b72`. No local
1223. patch. **#48 in this app**: every lambda parameter a body only READS is declared `&` (319 of them, by
1224. `tools/ref-params.py`; `tools/lambda-audit.py` lists the ones that change a parameter; the copy per call made `/my/schedule` 5× and `/my/unwatched` 2.5× slower). The 13 lambdas that
1225. change a passed record/list (`withCredits`, `withDetailsFields`, `addSum`, `firstStep`, `storeOrder`, `startFill`, `fillStep`,
1226. `mergeTvmaze`, `syncMovie`, `addTotals` — mostly through a local `let x = p`) keep the copy: every caller uses the RETURNED
1227. value (audit: STATUS mission 070). A NEW lambda: `&` for a record/list/map it only reads.
1228. The previous build (8590df63, sha256 `ca830ef8…`) is in `.scratch/pre-070/`. Built read-only from an archive
1229. (never in the hybriel checkout; the same with `hybriel-047` → `hybriel-056` for the older build):
1230. ```bash
1231. git -C /media/STORAGE/projects/hybriel archive master | tar x -C .scratch/hybriel-047
1232. cd .scratch/hybriel-047/native && /media/STORAGE/projects/hybriel/native/zig-toolchain/zig build -Dtarget=x86_64-linux-gnu.2.39 -Doptimize=ReleaseFast # → zig-out/bin/hybriel (hybriel README + ReleaseFast)
1233. cp zig-out/bin/hybriel ../../../bin/hybriel
1234. # plugins: copy every plugin dir the app already has (core crypto data fetch fs http http1 mpackdb proc smtp
1235. # time web) from .scratch/hybriel-047/plugins/ over plugins/ — `diff -rq plugins .scratch/hybriel-047/plugins`
1236. # must then list only plugins the app does not use
1237. ```
1238. Older builds: `.scratch/pre-056/` (ff51cf46, sha256 `55a724a0…`) and `.scratch/pre-047/`
1239. (837fe120 via ident, sha256 `9e5e95b3…`), bin/ + plugins/ each. After a re-vendor run the gate.