tracker

60.5 KB

1. # tracker.worldapi.org
3. The new tracker: a Hybriel app that will track the TV shows (and later movies) the creator
4. follows and watches. **`CONCEPT.md` (the creator's) is the source of truth** — read it first;
5. nothing is built that it does not describe. Built step by step, one ticket per step
6. (tracker.worldapi.org#1, #2, …).
8. **Built so far — step 1 (ticket #1)**: the shell. The header carries the ident login (an
9. identity selector, like calendar/gitoria) and sign-out; the homepage itself stays empty. No
10. design carried over from the old app.
12. **Step 2 (ticket #2)**: the old tracker's MongoDB export is now in `storage/mpackdb/` (new
13. mpackdb ids, every reference re-pointed) — see "Data" and `STATUS.md`. Nothing is shown in the
14. UI yet; the homepage still renders empty. That is a later step, from the creator.
16. **Step 4 (ticket #4)**: the show page, `/shows/:slug` (`components/show.hl`) — poster, title,
17. genre pills, plot, cast (as described in "What it does (step 4)" below), every season with its
18. episodes, and the watch check (episode and season, season bulk-toggles its episodes).
20. **Step 5 (ticket #5)**: `/unwatched` (`components/unwatched.hl`) — every already-released,
21. unwatched episode of a show the signed-in user follows, newest release first (see "What it does
22. (step 5)" below).
24. **Step 6 (ticket #6)**: `/schedule` (`components/schedule.hl`) — every not-yet-released episode
25. of a show the signed-in user follows, soonest first, no check icons (see "What it does (step 6)"
26. below).
28. **Step 7 (ticket #7)**: `/my/shows` (`components/myshows.hl`) — every show the signed-in user
29. follows, newest follow first: small poster, title, last watched episode (`S08E35`) (see "What it
30. does (step 7)" below). The homepage itself is still empty.
32. **Step 8 (ticket #8)**: the personal lists all live under `/my/`: `/my/shows`, `/my/unwatched`,
33. `/my/schedule` (old `/unwatched`, `/schedule` answer 301). Episodes everywhere as `S01E01`, show
34. titles in lists with the year (`Doctor Who (2005)`), "1 episode" singular, and every `/my/` page
35. under 1 s full load on the real data (see "What it does (step 8)" below).
37. **Step 9 (ticket #9)**: the TMDB sync — every show someone follows gets its new seasons/episodes
38. (existing ones updated: title, summary, release) and its poster from TMDB, once a day inside the app
39. (04:00 UTC) and on demand with `tools/sync-tmdb.hl` (see "What it does (step 9)" below).
41. **Step 10 (ticket #10)**: an installable app (PWA) with its own icon — a TV with a check mark, tracker green
42. on dark; also the favicon. The shell (`/`) opens without network and says it is offline (see "What it does
43. (step 10)" below).
45. **Step 12 (ticket #12)**: the show page's header has small link icons to the show on TMDB, IMDb, TheTVDB and
46. TVmaze (a movie: TMDB + IMDb), only for the ids the show has, each in a new tab; the TMDB sync fills missing
47. ids (TMDB `external_ids`, TVmaze lookup) and never overwrites one (see "What it does (step 12)" below).
49. **Step 13 (ticket #13)**: the homepage has content — signed in, four big icons (Unwatched, Schedule, Shows,
50. Movies → the `/my/` pages); for everyone a short text about the site, the 10 latest released movies and the 10
51. series with the newest released episode (headings link to `/movies` and `/shows`: everything, 24 per page). Plus
52. `/my/movies` (the movies you follow); `/my/shows` is series only now (see "What it does (step 13)" below).
54. **Step 15 (ticket #15)**: the daily sync also asks TVmaze for every series and adds what TVmaze has but TMDB not
55. yet — new episodes/seasons, missing titles and air dates; TMDB values are never overwritten (see "What it does
56. (step 15)" below).
58. **Step 14 (ticket #14)**: the search — a magnifier in the header opens `/search`; typing searches OUR shows, movies and
59. people at once (in-memory index), "Fetch from web" lists TMDB titles we don't have yet, "Add" imports one like the daily
60. sync and opens its page (signed out: the sign-in modal) (see "What it does (ticket #14)" below).
62. **Mission 054**: adult titles never appear in the public lists (homepage rows, `/movies`, `/shows`) or the search (ours and
63. "Fetch from web") — only titles TMDB says are NOT adult are listed, unknown counts as hidden. A followed title stays on the
64. user's `/my/` pages and its page. A background job in the app (the "adult backfill") asks TMDB once per title for the
65. adult flag, the poster and the external ids (see "What it does (mission 054)" below).
67. Written in **Hybriel** on **hl:web**, same stack and conventions as `ident.worldapi.org`
68. (vendored plugins/binary copied from there, see "Vendored Hybriel").
70. ## Run (dev, Loreana)
72. ```bash
73. cd /media/STORAGE/projects/tracker.worldapi.org
74. TRACKER_PORT=8700 TRACKER_SYNC=0 setsid nohup ./bin/hybriel project.hl > server.log 2>&1 < /dev/null & echo $! > server.pid
75. # TRACKER_SYNC=0: the runtime loads .env (real TMDB token) by itself — without it a dev server syncs from TMDB at 04:00 UTC
76. # stop: kill $(cat server.pid)
77. ```
79. * Config (env; the real environment outranks nothing here — there is no `.env` reader in
80. project.hl, unlike ident's SMTP settings — set these in the shell or `docker-compose.yml`):
82. | Variable | Default | |
83. |---|---|---|
84. | `TRACKER_PORT` | 45008 | |
85. | `TRACKER_URL` | `http://127.0.0.1:<port>` | this app's own origin — the ident login button's `return=` is built from it |
86. | `IDENT_URL` | `https://ident.worldapi.org` | |
87. | `TRACKER_KEY` / `TRACKER_SECRET` | `''` | this app's API key + secret, from ident's `/apps` (register once, origin = `TRACKER_URL`) — **not set yet on Byrodin**: the first deploy (architect) registers the app in the live ident and puts these in `.env` next to `docker-compose.yml`, exactly like ident's own `.env` holds its SMTP settings |
88. | `TRACKER_WATCH` | on | `0` = no dev watcher (the container) |
89. | `TRACKER_STORAGE` | `./storage/mpackdb` | the `usersTable` directory |
90. | `TRACKER_SESSIONS` | `./.sessions` (hl:web's own default) | this app's own session store |
91. | `HL_HOST` (or `HOST`) | 0.0.0.0 | interface to bind; `127.0.0.1` on Byrodin |
92. | `TMDB_READ_TOKEN` | — | TMDB v4 read token (in `.env`, never print it). Unset → the daily sync is off. **The runtime loads the `.env` beside `project.hl` by itself** (the real environment wins): every dev server on Loreana has the real token — start dev servers with `TRACKER_SYNC=0` |
93. | `TRACKER_SYNC` | on | `0` = no daily TMDB run inside the app |
94. | `TRACKER_SYNC_HOUR` | 4 | UTC hour of the daily run (hl:time has no time zones; the container is UTC) |
95. | `TMDB_BASE_URL` / `TMDB_IMAGE_URL` | `https://api.themoviedb.org/3` / `https://image.tmdb.org/t/p` | the gate points both at its fake TMDB |
96. | `TVMAZE_BASE_URL` | `https://api.tvmaze.com` | #12: the TVmaze id lookup; the gate points it at its fake (`<fake>/tvmaze`) |
97. | `TRACKER_BACKFILL` | on | mission 054: `0` = no adult-flag backfill (off too without `TMDB_READ_TOKEN`). **A dev server with the real token runs it** — it only writes the storage it was given, but asks real TMDB |
98. | `TRACKER_BACKFILL_LOG_EVERY` | 500 | a progress line every N titles |
100. Without `TRACKER_KEY`/`TRACKER_SECRET`, `/` still renders (signed out, the selector and "Log
101. in with ident" show in the header); a login attempt answers `'login is not set up on this
102. server (TRACKER_KEY / TRACKER_SECRET missing)'` instead of exchanging a code — so the app is
103. never a dead 500 while waiting for that one-time setup.
105. ## What it does (step 1)
107. * **Login, copied unchanged from calendar.worldapi.org** (rejected once for a centered
108. sign-in and no header selector — architect, 2026-09-27): ident only, no own passwords
109. (ident's login button flow, ident.worldapi.org README "How apps use ident"), the identity
110. selector (`<ident-selector>`, ident's `/selector.js`) sits in the header
111. (`components/main.hl`, `userBox`), a plain "Log in with ident" link beside it when signed
112. out. Choosing an identity in the selector fires `ident-login` (one-time code); `login.js`
113. (an external-component bridge, allowed per the creator: "login.js is correct, as thats for
114. externals in general") hands the code to a hidden input, whose `change` calls the server
115. face `trackerLogin` (`users.hl` `exchangeCode`, `POST <ident>/api/exchange` — nothing else:
116. ident does not hand over a display name yet, ident#23 done / ident#11 on hold), which makes
117. or finds this app's own user record (`usersTable`, `storage/mpackdb/users.db`) and sets
118. `session.user = { id }`. `/login/callback` is the same exchange as a plain redirect (the
119. login button's return URL) for a client that has no socket yet. "Log out" in the header
120. calls `trackerLogOut`; ident's own login is untouched. This app's own session cookie is
121. `trackersid` (cookies ignore ports, an own name keeps it apart from ident's `identsid` on
122. the same dev host, hybriel#10/#17).
123. * **The empty homepage** (until #13 — see "What it does (step 13)") (`/`, `components/home.hl`): no content at all, signed in or out —
124. the header alone carries the login state. Nothing else.
125. * **One mpackdb table**: `storage/mpackdb/users.db` (`identity` → this app's user id — no
126. display name, no other per-user data yet). Step 2 (tracker.worldapi.org#2, open) brings the
127. old tracker's data across.
129. ## What it does (step 4)
131. `/shows/:slug` (`components/show.hl`, slug = the migrated `urlSegment`), read-only data from
132. `shows.hl`, per-user watch state from `watches.hl` (both reused as-is by `/unwatched` and
133. `/schedule`, tickets #5/#6 — "reuse, don't copy"):
135. * **Header**: poster on the left (`/posters/<name>`, `project.hl` `posterRoute` — served from
136. `storage/mpackdb/posters/`; the real poster files were never in the Mongo backup, see
137. `DECISIONS.md` "tracker concept" — a placeholder SVG shows until a later TMDB step brings
138. real ones back), title, genre pills (blue, link to `/genre/<slug>`) and the plot summary to
139. its right, the cast below as comma-separated red links to `/person/<slug>`.
140. * **Seasons**, latest first, the latest expanded and every other one collapsed to start; each
141. row: a round check icon (a disc with a check, drawn in SVG/CSS, no icon font), the season
142. title/number, its episode count. Its episodes (number + title, and their own check icon)
143. show once expanded (click the row to toggle).
144. * **The watch check** (signed-in only — signed out, no check icons show at all): clicking an
145. episode's check toggles that one watch; clicking a season's check toggles the season's own
146. watch AND every one of its episodes at once (`watches.hl` `setSeasonWatched`). Every click is
147. a server round trip (`emit server showToggleEpisode` / `showToggleSeason` / `showRows` for
148. collapse/expand) that rebuilds the row list server-side and reassigns it whole — a season has
149. at most a few dozen episodes, cheap. A **Hybriel gap found here** (filed as hybriel#115,
150. reported after this ticket's first attempt): a plain per-instance member function that calls
151. an hl:mpackdb-backed import comes back `null` when called from inside an `on server` face,
152. even though the very same call works as a member's own top-level initializer — the fix is to
153. declare such helpers `static` (the same shape calendar.worldapi.org's `soonOf` already uses).
154. `components/show.hl`'s own helpers (`buildRows`, `genreRowsOf`, …) are `static` for exactly
155. this reason.
156. * **Data fix**: `tools/relink-episode-seasons.hl`, a one-off idempotent tool that links every
157. episode whose `season` was still null in the migrated data (36,194 of them, ticket #2) to its
158. season by matching `(show, seasonNumber)` — never touches an episode whose season is already
159. set, never runs `update()` on the live table (hybriel#113; it rebuilds a fresh episodes table
160. and swaps the files over, the same rule `tools/migrate.hl` follows). Not run against the live
161. data yet — the architect runs it (stop the container first, `storage/mpackdb` is not shared
162. across processes, hybriel#21).
164. ## What it does (step 5)
166. `/unwatched` (`components/unwatched.hl`): every episode of a show the signed-in user follows
167. (`follows.hl`, new) that is already released (`release` ≤ today) and not yet watched
168. (`watches.hl` `isWatched`), newest release first. Creator: "/unwatched lists all unwatched
169. episodes of shows followed in release date desc order"; "check icon is a disc with a check in
170. it" — the SAME icon/class (`watches.hl` `watchClassOf`, moved there from `components/show.hl` so
171. both pages share one definition instead of each declaring its own) and the same
172. `setWatched`/`isWatched` watch logic as the show page (step 4), not copied.
174. * Each row: check icon, the show's title (linking to `/shows/:slug`), "Ep `<number>` ·
175. `<title>`", the release date.
176. * A row only ever shows an unwatched episode, so a click always marks it watched (never toggles
177. back, unlike the show page's checks) — the row leaves the list. Signed out: no rows, a
178. "Sign in to see the shows you follow." message instead.
179. * `follows.hl` (new): read-only per-user follow data (`followedShowIds`), kept apart from
180. `shows.hl`/`watches.hl` the same way, for `/schedule` (ticket #6) to reuse.
182. ## What it does (step 6)
184. `/schedule` (`components/schedule.hl`): every episode of a show the signed-in user follows
185. (`follows.hl`) that is NOT yet released (`release` > today), soonest first. Creator: "/schedule
186. has a similar list without check icons for the upcoming episodes of shows followed" — same row
187. layout as `/unwatched` (show title linking to `/shows/:slug`, "Ep `<number>` · `<title>`", the
188. release date), minus the check icon and minus any click handling: the page is read-only, so it
189. needs no `on server` face at all. Signed out: no rows, the same "Sign in to see the shows you
190. follow." message as `/unwatched`.
192. ## What it does (step 7)
194. `/my/shows` (`components/myshows.hl`): every show the signed-in user follows, ordered by the
195. follow's `at` (epoch ms), newest first. Creator: "trackers /my/shows that just ists the shows in
196. desc order i followed them, small poster, title, last episode like s08e35". Read-only, no `on
197. server` face (like `/schedule`).
199. * Each row: a small poster (2.5rem wide, the same `/posters/<name>` route + placeholder as the show
200. page; a show with no poster name at all gets `/posters/none`, i.e. the route's placeholder), the
201. title linking to `/shows/<slug>`, and the **last watched episode** as `SxxEyy` (zero-padded to
202. two digits) = the HIGHEST season/episode number the user has an episode watch for (architect's
203. reading, in the ticket) — not the most recently watched one; season watches alone don't count.
204. Nothing watched → no episode element at all.
205. * Signed out: no rows, the same "Sign in to see the shows you follow." as `/unwatched`.
206. * Cost: touches only the user's follows + episode watches (one `episodeById` fetch per watch),
207. never all episodes of a followed show — 128 rows in ~0.2 s on a copy of the real data (STATUS.md).
208. * Reuse: `follows.hl` `followsOfUser` (new — the whole follow record, `followedShowIds` now
209. builds on it), `shows.hl` `episodeById` (new, one line), `showById`, `posterName`, `watches.hl`
210. `watchesOfUser`.
212. ## What it does (step 8)
214. * Routes (`project.hl`): `/my/unwatched`, `/my/schedule`; the old `/unwatched`, `/schedule` are
215. function routes answering **301** to them (`movedTo`, no query carried — the pages take none).
216. * `shows.hl` (shared by all lists): `episodeCode(s, e)` → `S01E01` (`/my/shows`, `/my/unwatched`,
217. `/my/schedule` rows: `S01E02 · <title>`), `titleWithYear(show)` → `Doctor Who (2005)` (`year`,
218. else the first 4 chars of `release`, else the bare title — 197 shows have no `year`), `pad2`,
219. `posterUrlOf(show)`. The show page's own `h1` stays without the year (it is not a list).
220. * Show page: season count `1 episode` / `N episodes`.
221. * **Speed** (the creator's real data: 128 follows, 11,692 watches, 831 unwatched rows):
222. * `/unwatched` was ~15 s: `isWatched` scanned all 11,692 watches per episode. Now
223. `watches.hl` `watchedSet(rows)` builds `'<type>:<id>' → true` ONCE; `/my/unwatched` and the
224. show page's `buildRows` look up that map. Now ~0.6 s server render (the rest is fetching
225. every episode of 128 shows; the insertion sort is ~50 ms).
226. * `/my/shows` "25 s full load while data takes 0.2 s": (1) **hl:web serves one request at a
227. time** — measured: a `/my/shows` request made during a 15 s `/unwatched` render waited 13.7 s;
228. every poster request of every visitor queues the same way. Fixing `/unwatched` removes that.
229. (2) 128 rows = 128 different `/posters/<oldId>.jpg` URLs, all answering the same placeholder
230. (no poster files exist yet) — 128 requests per load. `posterUrlOf` now gives rows without a
231. poster FILE the one URL `/posters/none` → 1 request, cached.
232. * How it was measured: see STATUS.md ticket #8 ("Real-data check").
234. ## What it does (step 9)
236. `tmdbsync.hl` (`syncShow`), run by the app every day and by `tools/sync-tmdb.hl`:
238. * **Which shows**: every show ANY user follows (`follows.hl` `allFollowedShowIds`, 126 on the real data).
239. * **Per show** (1 + ⌈seasons/20⌉ requests + the poster if new): `GET /tv/<tmdbId>?append_to_response=external_ids`
240. (status, counts, overview, `poster_path`, the season list, since #12 the external ids), then `GET /tv/<tmdbId>?append_to_response=season/1,…` (≤ 20
241. seasons with their episodes per request). Bearer `TMDB_READ_TOKEN`.
242. * **Matching, never duplicating, never deleting**: episode by `tmdbId`, else (161 migrated episodes have
243. none) by S/E number if that one has no tmdbId yet — then it gets the tmdbId; season by (show,
244. seasonNumber) (migrated seasons have no tmdbId). Existing episodes/seasons: title, summary, release
245. updated; an empty TMDB value never overwrites. New ones: self-assigned 16-hex ids (NOT mpackdb's own —
246. hybriel#113), `oldId = 'tmdb-episode:<id>'` / `'tmdb-season:<showTmdb>:<n>'` (the tables' `!oldId`
247. index), linked into `season.episodes` / `show.seasons`. Show: `status`, `seasonsCount`,
248. `episodesCount`, `tmdbSummary`, `tmdbPoster`, `tmdbSync` (ms). Season 0 (specials) only for a show that
249. already has one; a TMDB season with no episodes is not created.
250. * **Posters**: TMDB `poster_path`, size w342 → `storage/mpackdb/posters/<oldId>.<ext>` (the folder the
251. existing `/posters/:name` route already served; inside `storage/` → in the nightly backup), `show.image`
252. set to the ext. Downloaded again only when the file is missing or TMDB's `poster_path` changed.
253. * **Daily run inside the app** (`project.hl` `syncTick`): an hl:time `every(1)` clock; at
254. `TRACKER_SYNC_HOUR` (UTC), once per day, it queues all followed shows and then syncs ONE show per tick.
255. hl:web serves one request at a time and `fetch()` blocks the whole process, so a step blocks requests
256. for its duration (real data: ~0.7 s average, Saturday Night Live 53 seasons ~1.7 s; logged as
257. `tmdb sync: slow step …` above 1.5 s) — requests queued meanwhile are served between two ticks; a tick
258. never runs inside a page render. After a show with n requests the next waits n × 260 ms (TMDB ≤ 40
259. requests / 10 s). Log: `tmdb sync: daily at 4:00 UTC`, `tmdb sync: start, N followed shows`,
260. `tmdb sync done: shows=… newSeasons=… newEpisodes=… updatedEpisodes=… updatedSeasons=… posters=…
261. requests=… errors=… seconds=…` (`docker logs tracker.worldapi.org | grep tmdb`). Tables are persisted
262. every 10 shows and at the end (~36 ms).
263. * **The tool** (same sync, sequential, paced; prints one line per show + the totals). **App stopped or a
264. COPY only** — hl:mpackdb is one process per storage (hybriel#21):
265. ```bash
266. set -a; . ./.env; set +a # TMDB_READ_TOKEN, never print it
267. TRACKER_STORAGE=$PWD/.scratch/realdata/storage/mpackdb ./bin/hybriel tools/sync-tmdb.hl [--limit N]
268. ```
269. * Real data (copy, 2026-09-30): first run 126 shows → +20 seasons, +800 episodes, 2651 episodes
270. updated, 125 posters (6.4 MB), 380 requests, 0 errors, 186 s; a second run: all 0, 255 requests, 90 s.
271. Details: STATUS.md ticket #9.
273. ## What it does (step 10)
275. The installable app, the same way calendar.worldapi.org does it (its ticket #3) — hl:web's own manifest
276. and service worker from settings in `project.hl`, no JavaScript of ours:
278. * `appIcons` (192 + 512 PNG, each `any` and `maskable`), `appTouchIcon` (180 PNG), `appFavicon`
279. (`/icons/favicon.svg`), `appThemeColor` = token `darker` (the header, rgb(15, 20, 25)),
280. `appBackgroundColor` = token `dark` (rgb(25, 30, 35)); name = `appTitle` "tracker". hl:web serves
281. `/__hl/manifest.webmanifest` (start_url/scope `/`, display standalone) and links it, the
282. apple-touch-icon and theme-color from every head. `/favicon.ico` is a real icon (16/32/48) for
283. browsers that ask for it themselves. Each icon has its own `file` route in `project.hl`.
284. * **Icons** (`icons/`): `icon.svg` is the source (512, hand-written: a TV with antennas and a check on
285. the screen, `#4ec9b0` on rgb(25,30,35); everything inside the maskable safe zone, a circle of radius
286. 204, so one image serves `any` and `maskable`); `favicon.svg` is the same drawing cropped tight on a
287. rounded tile. Rendered on Loreana:
288. ```bash
289. rsvg-convert -w 192 -h 192 icons/icon.svg -o icons/icon-192.png
290. rsvg-convert -w 512 -h 512 icons/icon.svg -o icons/icon-512.png
291. rsvg-convert -w 180 -h 180 icons/icon.svg -o icons/apple-touch-icon.png
292. for s in 16 32 48; do rsvg-convert -w $s -h $s icons/favicon.svg -o /tmp/fav-$s.png; done
293. magick /tmp/fav-16.png /tmp/fav-32.png /tmp/fav-48.png icons/favicon.ico
294. ```
295. * **Offline**: `offline = [ Home ]` — the worker precaches the shell (runtime, modules, CSS, manifest,
296. icons, favicon) and the document of `/`. Navigations are network-first; without a network `/` comes
297. from the cache and every other page gets hl:web's own "Unavailable offline" page (503) — data pages
298. need the network, no offline data. The shell (`components/main.hl`) shows "You are offline. Your shows
299. and lists need the network." while `navigator.onLine` is false: hl:web gives a page no connection
300. state and no mount hook, so an invisible `netProbe` runs an endless 1 s CSS animation (`styles.hl`
301. `@keyframes tracker-net-tick`) and its `animationiteration` handler reads `navigator.onLine` (a write
302. only when it changes). A server that is down while the device is online shows no note.
304. ## What it does (step 12)
306. * **Link icons** (`components/show.hl`, `shows.hl` `externalLinksOf`, `styles.hl` `showLinks`/`a.extlink`): under the
307. title (phone: between the title row and the full-width poster; desktop: under the title, right of the poster) one
308. small monochrome badge per id the show HAS — `TMDB` https://www.themoviedb.org/tv/<tmdbId> (a movie: `/movie/<id>`),
309. `IMDb` https://www.imdb.com/title/<imdbId>/, `TVDB` https://thetvdb.com/dereferrer/series/<tvdbId>, `TVmaze`
310. https://www.tvmaze.com/shows/<tvmzId>. A movie (`type = 'movie'`) shows only TMDB + IMDb. Each `target="_blank"
311. rel="noopener"`. CSS only (muted grey badge, dark text, accent on hover), no copied logos. URL forms checked
312. 2026-10-01: all four 301 to the site's slug page (TheTVDB's old `?tab=series&id=` too; `dereferrer` is its id form).
313. * **Sync** (`tmdbsync.hl` `externalIdsFor`): the details request carries `append_to_response=external_ids` (no extra
314. request); a MISSING `imdbId` (`tt` + digits) / `tvdbId` (number > 0) is filled from it. A show still without
315. `tvmzId` → `GET <TVMAZE_BASE_URL>/lookup/shows?imdb=tt…`, on 404 `?thetvdb=<id>` (hl:fetch follows TVmaze's 301 to
316. `/shows/<id>`); 550 ms pause per lookup (TVmaze 20 / 10 s) — both runners wait `pauseMsAfter(r)`. **An id the show
317. has is never overwritten.** Totals gain `ids=` (ids filled) and `tvmazeRequests=`.
318. * **Movies** (7814 of 9453 migrated rows are `type = 'movie'`, 1 followed: Star Trek: First Contact): before #12 the
319. sync asked `/tv/<tmdbId>` for them — a DIFFERENT title on TMDB. Now a movie gets one `GET
320. /movie/<tmdbId>?append_to_response=external_ids`: missing imdbId + its poster; no seasons/status/overview.
321. * **Counting ids** (a COPY only): `TRACKER_STORAGE=$PWD/.scratch/realdata/storage/mpackdb ./bin/hybriel
322. tools/count-external-ids.hl` → per type, all / followed: shows, tmdbId, imdbId, tvdbId, tvmzId.
324. ## What it does (step 15)
326. * **TVmaze merge** (`tmdbsync.hl` `mergeTvmaze`, after the TMDB step of `syncShow`, both runners): a series with a
327. `tvmzId` → `GET <TVMAZE_BASE_URL>/shows/<tvmzId>/episodes` (1 request, 550 ms pause like the lookups). Matched by
328. season/episode number: a missing episode is ADDED (`tmdbId = null`, `tvmzId`, `oldId 'tvmaze-episode:<id>'`; its
329. season too if missing: `oldId 'tvmaze-season:<tvmzId>:<n>'`), an existing one gets an EMPTY title / air date
330. filled. **A TMDB value is never overwritten.** Placeholder titles (`Episode 4`, `Épisode 4`, `Folge 4`, `TBA`,
331. `isPlaceholderTitle`) count as empty — and TMDB's placeholder no longer overwrites a real title (else they flip
332. daily). When TMDB lists the episode later, the TMDB step adopts the row by number (it has no tmdbId), TMDB wins.
333. Skipped: TVmaze specials (no number), season 0 unless the show has one, summaries (TVmaze's are HTML).
334. * **Numbering check** (`numberingAgrees`): ≥ 80 % of the episodes both know must air within a day OR share the title,
335. and a show with episodes must share at least one S/E with TVmaze — else TVmaze is ignored for that show
336. (`tvmazeSkipped`). Real data: The Daily Show (TVmaze seasons by year: would have added 1093 episodes), Star Trek:
337. Prodigy / Blood & Treasure (TMDB merges double episodes → shifted), Alex Rider / Intergalactic / The Rising (other
338. premiere dates, placeholder titles) are skipped.
339. * Totals/tool line gain `tvmazeEpisodes= tvmazeSeasons= tvmazeFilled= tvmazeSkipped=`; `tvmazeRequests` now also
340. counts the episode lists (≈ 125/day on real data, ~70 s more per daily run).
342. ## What it does (ticket #14: search)
344. Creator: "in the actionbar a search symbol that first checks our database and then has a fetch from web button that
345. actually searches tmdb then for things we dont have yet". `components/search.hl` (page), `search.hl` (index, TMDB, import).
347. * **Header**: a magnifier (`#searchlink`, SVG) before the ident selector → `/search` (client-side navigation).
348. * **Address `/search/<text>`, not `/search?q=`**: hl:web gives a page its route params but no query string. A full load gets
349. the segment already decoded by the server (an encoded `/` would split it → 404), a client-side navigation the raw one —
350. `queryOfParam` decodes what is left, safely (malformed `%` → literal). While typing the address follows the text
351. (`history.replaceState`; `/ ? #` become spaces). `/search?q=x` just opens the empty search page.
352. * **Our database, as you type** (face `searchDb`, no session): `search.hl` builds an **in-memory index once at boot**
353. (`project.hl` → `ensureIndex`, ~0.6 s on the real data, log line `search index: 9453 titles, 15157 people, 4180 keys,
354. … ms`): every title/name → words (lower case, apostrophes dropped, punctuation = word break, Latin accents folded; a
355. title's year is one more word); each word's first 2 and 3 letters are bucket keys. A query reads ONE bucket (its longest
356. word's first 3 letters, 2 for a 2-letter word) and keeps entries where every query word starts one of the entry's words.
357. Order: whole name, name starts with the text, first word does, rest; then shorter name; then series before movie, newer
358. first. ≤ 30 titles ("The first 30 of N titles — type more…") + ≤ 10 people. Under 2 letters: "Type at least 2 letters.".
359. People link to `/person/<slug>` (no person page yet — same as the show page's cast).
360. * **Fetch from web** (face `searchWeb`, signed out too): ONE `GET <TMDB>/search/multi?query=…&include_adult=false&page=1`;
361. people dropped, titles we have (same TMDB id AND kind: `series:<id>` / `movie:<id>`) left out. Poster thumbs straight
362. from TMDB's image host (`w92`).
363. * **Add** (face `searchImport`, session): signed out → the sign-in modal, nothing sent. Signed in → `importTitle`: one
364. `GET /tv/<id>` (or `/movie/<id>`) for the new record (title, overview, first air/release date → year, status, genres
365. matched to OUR genres by name, language, homepage, tagline; own 16-hex id, `oldId = tmdb-tv-<id>` / `tmdb-movie-<id>`,
366. `urlSegment` like the migrated ones: `Pluribus`, taken → `-2`…), then **`tmdbsync.hl` `syncShow`** — seasons, episodes,
367. poster, external ids exactly like the daily sync — `persistSync`, index updated, the public lists too (`catalog.hl`
368. `refreshCatalogShow`, since the #13/#14 merge); then the show page opens. A title we
369. already have answers its slug. A failure (TMDB 404 …) is shown on the page ("Could not add it: …"), the lists stay.
370. Log: `search import: tv 225171 "Pluribus" → /shows/Pluribus (newSeasons=1 newEpisodes=9 posters=1 ids=3 requests=4 errors=0)`.
371. * An imported title is NOT followed automatically, so the daily sync (followed shows only) does not update it later.
372. * Hybriel traps (hybriel#121 / #122): no list is ever appended to (replaced whole); no member reads both the route param
373. `q` and `session`; the gate checks the lists after a failed import (a session face that stays on the page).
375. ## What it does (mission 047: checks after client-side navigation, mobile first, show page)
377. * **Invisible checks fixed** by re-vendoring Hybriel master ff51cf46 (see "Vendored Hybriel"): the old
378. build created SVG built in the BROWSER (client-side navigation, expand/collapse, a check click) in the
379. XHTML namespace — nothing drawn. Server-rendered pages were fine.
380. * **Mobile first** (`styles.hl`): every rule outside the one `@media (min-width: 40rem)` block is the
381. phone's (390 px); the block only adds the wide layout. The header is one row on a phone (brand, ident
382. selector, Log out); `/my/unwatched` + `/my/schedule` rows are two lines on a phone (show + date, the
383. episode below), one line from 40rem.
384. * **Show page** (`components/show.hl`): phone = title (+ Follow) first, then the poster at the full
385. content width, then genres/plot/cast (`showMeta` is `display: contents` there, the title row
386. `order: -1`); desktop unchanged (poster 10rem left). Season header: a caret at the right edge (down =
387. open, `season-row.collapsed` turns it right). Episode rows are not indented (episode check under the
388. season check). Artists orange (`#ce9178`, token `orange`). Unwatched check = the inverted solid icon:
389. accent outline circle + accent check mark; watched = filled accent disc, dark check.
390. * **Follow toggle** (signed in): "Follow" = the inverted button (accent border), "Following" = filled;
391. face `showToggleFollow` → `follows.hl` `setFollowed` (new record, `at` = now → top of `/my/shows`;
392. unfollow deletes the user's records for that show). **Signed out** the checks and Follow are visible;
393. a click opens the modal "You need to sign in to follow shows and mark episodes." (Close + "Sign in with
394. ident" = the ident login link) and sends nothing. The modal is `components/modal.hl`, copied verbatim
395. from components.hybriel.worldapi.org `components/modal/modal.hl` (becbd59, 2026-09-30) — its own
396. `Style` lands in `/__hl/app.css`.
398. ## What it does (step 13)
400. * **`/`** (`components/home.hl`): signed in, four icon tiles (inline SVG, accent) → `/my/unwatched`, `/my/schedule`,
401. `/my/shows`, `/my/movies` (2×2 on a phone, 4 in a row from 40rem). For everyone: the text about the site (signed out
402. plus a "Log in with ident" hint), then **Movies** (→ `/movies`) = the 10 latest RELEASED movies (release ≤ today) and
403. **Shows** (→ `/shows`) = the 10 series whose newest released episode is newest. Tiles: poster (`posterUrlOf`:
404. the file or `/posters/none`), title, year; a link to `/shows/<slug>` (movies too — same page, same Follow button).
405. Phone: each row scrolls sideways inside itself (`tile-row`, the page never does); from 40rem a grid of 5.
406. * **`/movies`, `/shows`** (`components/movies.hl`, `components/allshows.hl`, both compose `components/tilelist.hl`):
407. every movie by `release` desc (future-dated ones first, undated last — "all … sorted by release date desc"); every
408. series by its newest released episode (series with none released after them, by their own `release`). 24 per page,
409. grid of 3 (phone) / 6. **The page is a path segment: `/movies/page/2`**, page 1 = `/movies`; out of range → the last
410. page, not a number → page 1. NOT `?page=2` (the mission asked for it): a page component cannot read the query
411. (hybriel#11, query half open) and hl:web's Back (`popstate`) drops the query — every page change would be a full
412. reload and Back would land on page 1. With the path, paging is a client-side navigation and Back works.
413. * **Pagination** = components.hybriel's `components/pagination/pagination.hl`, **copied verbatim** to
414. `components/pagination.hl` (4b4dabe) — re-copy, don't edit. It has no data out (its README, hybriel#87), so
415. `tilelist.hl` wraps it in `pager-box { on click }`: a click on one of its buttons bubbles there, the button's text
416. ("Previous", "Next", a number; "…" nothing) gives the page → `navigate()`. Hidden when there is only one page.
417. `styles.hl` makes its buttons a bit tighter on a phone so "Previous 1 … 99 100 101 … 326 Next" stays one row.
418. * **`/my/movies`** (`components/mymovies.hl`): the movies the user follows, newest follow first (same rows as
419. `/my/shows`: small poster, title with year). `/my/shows` now leaves movies out. Shared sort: `follows.hl`
420. `insertByFollowDesc`.
421. * **Speed — `catalog.hl`**: built ONCE at start (one scan of shows + episodes: real data ~1 s, server answers
422. ~1.5 s after launch) into two sorted id lists in memory (`static index`; hand-written merge sort). A page only
423. slices 24 ids and fetches those shows: real data, server time ≈ 5 ms per page. Kept fresh without a rebuild:
424. `project.hl` `syncTick` calls `refreshCatalogShow(id)` after each synced show (its seasons/episodes only, ~2.5 ms;
425. 126 shows 0.3 s); a NEW DAY moves episodes that came out today (each series keeps its upcoming release dates) into
426. place on the first read. `tools/sync-tmdb.hl` runs with the app stopped → the next start rebuilds.
427. * **Content warning** (open, not decided here): the migrated movie table holds many adult titles (the old tracker
428. imported TMDB lists); they show on the public `/` and `/movies`. No `adult` flag is stored.
430. ## What it does (mission 054: adult titles, the backfill)
432. * `shows.adult`: `true` / `false` from TMDB, `null` = not known yet (all 9,453 migrated rows). `shows.hl isPublicTitle` =
433. `adult == false`. Used by `catalog.hl` (the homepage rows, `/movies`, `/shows`: built from public titles only;
434. `refreshCatalogShow` adds or removes one) and `search.hl` (`titleOk[id]`; `dbSearch` skips the others — they are not
435. counted either). "Fetch from web" asks TMDB with `include_adult=false` and keeps only results with `adult: false`.
436. * Not filtered: `/my/shows`, `/my/movies`, `/my/unwatched`, `/my/schedule` (the user's own follows) and the show page
437. itself (`/shows/<slug>` opens for anyone with the link). Hiding the page from non-followers was tried and dropped: a
438. `found` that depends on the session breaks the page after any session face (hybriel, see STATUS mission 054).
439. * Every TMDB answer sets the flag: the daily sync (`syncShow`/`syncMovie`), the search import (`importTitle`), the backfill.
440. * **The backfill** (`project.hl backfillTick` → `tmdbsync.hl backfillTitle`): at start the app collects every title with
441. `adult` unknown, a `tmdbId` and no `adultCheck`; then ONE title per step on the app's clock (0.1 s tick, shared with the
442. daily sync, which takes precedence — the backfill waits while a daily run is going on):
443. `GET /movie|tv/<tmdbId>?append_to_response=external_ids` → `adult`, a missing `imdbId`/`tvdbId`, the poster (w342 into
444. `posters/<oldId>.<ext>` when the file is missing; no TVmaze, no seasons). Then 260 ms pause per API request (≤ 40 /
445. 10 s; poster downloads go to TMDB's image host and are not counted). The lists and the search take the title at once.
446. TMDB 404 → `adultCheck = 'TMDB 404'` (stays hidden, never asked again); no answer / 429 / 5xx → back of the queue, at
447. most 2 more tries per start. Persisted every 25 titles (mpackdb's `update` is durable anyway — the gate's resume check
448. stops it after 6 titles, before any persist). A restart builds the queue from what is still unknown = it resumes.
449. * Log: `adult backfill: start, N titles without an adult flag`, every 500: `adult backfill: 500/9453 titles, adult=…
450. notAdult=… unknown=… posters=… ids=… requests=… errors=… retries=… seconds=…`, each failure, then `adult backfill done: …`;
451. with nothing to do: `adult backfill: nothing to do`.
452. * Real data (copy of 2026-10-01, real TMDB): 9,453 titles in 99 min — 3,450 adult, 5,866 not, 137 unknown (TMDB 404),
453. 8,740 posters (459 MB in total); pages during the run avg 0.05–0.16 s, max 4.9 s (a slow TMDB answer blocks). STATUS.md mission 054.
455. ## Test
457. ```bash
458. node tests/browser.mjs # THE GATE: this app's own server + its OWN ident (a copy of
459. # ident's code without .env, codes to a mail sink — no live
460. # ident, no real mail; tests/identkit.mjs, same as calendar's
461. # gate) + a real headless Chrome: signed out (header shows the
462. # selector + "Log in with ident"; since #13 the homepage has
463. # content, see below) -> sign in -> header shows "Log out" -> sign
464. # out -> signed out again -> a reload stays signed out. Also
465. # checks the header has no bottom border and filled buttons
466. # (incl. the ident selector's) have none, while inverted ones
467. # ("Log out") keep theirs.
468. # tracker.worldapi.org#4: a fixture show (tests/seed-show.hl,
469. # written into the gate's own storage before the server starts)
470. # proves /shows/:slug — header renders, no checks signed out;
471. # signed in, click an episode check (turns solid), click a
472. # season check (itself AND its episodes turn solid, proven
473. # after a reload — server-side, not just client state).
474. # tracker.worldapi.org#5: the same fixture (now followed by the
475. # test user) proves /unwatched — signed out, no rows; signed in,
476. # only the two already-released episodes show, newest first, the
477. # far-future one excluded; click a check, the row disappears
478. # (proven server-side after a reload).
479. # tracker.worldapi.org#6: the same fixture proves /schedule —
480. # signed out, no rows; signed in, only the far-future episode
481. # shows (the two already-released ones excluded), no check icons
482. # at all, unaffected by any watch toggle on the other pages.
483. # tracker.worldapi.org#7: the fixture now has a 2nd followed
484. # show (followed later, no poster name, never watched) and
485. # proves /my/shows — signed out, the sign-in message; signed
486. # in, newest follow first, title links, loaded posters (the
487. # placeholder), S01E02 after /unwatched's click, then S02E01
488. # (highest, not most recent) after the show-page checks; the
489. # unwatched show shows no episode.
490. # tracker.worldapi.org#8: the lists moved to /my/unwatched and
491. # /my/schedule (old addresses: 301, and the browser lands on
492. # the new one), rows say S01E02 / S02E01, titles carry the
493. # year ("Gate Test Show (2024)", "Unwatched Gate Show (2025)"
494. # — from `release`, that show has no `year`), the show page
495. # says "1 episode" / "2 episodes", /my/shows gives the show
496. # with a poster FILE (the gate writes posters/sh1.png) its
497. # own URL and the other one /posters/none.
498. # tracker.worldapi.org#9: a FAKE TMDB (tests/faketmdb.mjs, a
499. # node http server inside the gate, fixed JSON + a png; the
500. # app/tool get TMDB_BASE_URL/TMDB_IMAGE_URL + a gate token —
501. # never the real TMDB). App stopped → tools/sync-tmdb.hl: +2
502. # seasons, +4 episodes, 3 fixture episodes matched by S/E and
503. # updated, 2 posters, no season 0, Bearer token on every call;
504. # a 2nd run changes nothing. Restarted: /my/schedule and
505. # /my/unwatched show the new episodes, /my/shows + the show
506. # page the downloaded posters (/posters/sh2.png byte-equal),
507. # S1 keeps its watched episodes, no duplicates. Then the
508. # app's OWN daily run (TRACKER_SYNC_HOUR = the current UTC
509. # hour, fake TMDB slowed to 700 ms/request, one new episode):
510. # it runs, finds the episode, and pages asked meanwhile are
511. # answered < 2.5 s (between shows) though the run takes ≥ 3 s.
512. # tracker.worldapi.org#10: the PWA — head links (manifest,
513. # apple-touch-icon, favicon, theme-color = token darker), the
514. # manifest (type, name, start_url, scope, standalone, colours),
515. # every icon a real PNG of its size, touch icon 180, favicon
516. # svg + ico; Chrome: no installability error, manifest parsed
517. # without errors; the worker /__hl/sw.js registered, scope /,
518. # controls the page (a separate 390 px phone tab). OFFLINE:
519. # CDP offline → the note appears; the server STOPPED too (CDP's
520. # offline does not reach the worker's own fetches) → reload of
521. # / shows header + note from the cache; /my/shows gives
522. # "Unavailable offline"; server back → no note.
523. # mission 047: a CLIENT-SIDE navigation /my/shows → the show
524. # (window marker proves no reload): every check/caret svg in the
525. # SVG namespace and drawn — again after expand, an episode click,
526. # collapse (FAILED on the old vendored build: XHTML namespace,
527. # circle width 0 — /tmp/w047/old-build-gate.log); caret down/
528. # right and flips; episode check under the season check; artists
529. # rgb(206,145,120); unwatched icon = accent outline + accent
530. # check; Follow: "Following" filled → unfollow ("Follow",
531. # outlined) → gone from /my/shows, survives reload → follow →
532. # top of /my/shows; signed out: checks + Follow visible, a click
533. # (episode, season, Follow) opens the sign-in modal, Close
534. # closes it, a reload shows nothing was written. 390 px + 1280
535. # px: header one row, no sideways scroll, show-page order
536. # (phone: title, full-width poster, genres, plot, cast).
537. # tracker.worldapi.org#12: link icons only for existing ids
538. # (show 1: TMDB+IMDb, show 2: TMDB+TVDB, a fixture MOVIE: TMDB
539. # /movie/), exact hrefs, target _blank + rel noopener, a click
540. # opens a NEW tab (CDP target, canAccessOpener false) and the
541. # page stays; also after client-side navigation; phone/desktop
542. # position. Sync: external_ids come with the details request,
543. # the movie only via /movie/, a FAKE TVmaze (same fake server,
544. # /tvmaze/…, 301 like the real one): imdb hit, imdb 404 → thetvdb
545. # hit; afterwards the stored ids are unchanged (tt0000001, 4002
546. # — TMDB says otherwise) and the missing ones filled; 2nd run:
547. # ids=0, no TVmaze request.
548. # tracker.worldapi.org#13: fixture + 32 never-followed movies
549. # (Pager Movie 01…30 = 2001-01-01…30, one 2099, one undated) and
550. # a series with an episode 2021-01-01. Home signed out: no
551. # icons, the text, headings → /movies /shows, the 10 latest
552. # RELEASED movies (30…21, not the 2099 one), the series by
553. # newest released episode; signed in: 4 icons → /my/…, the
554. # Movies icon → /my/movies (client-side). /movies via the
555. # heading: page 1 = 2099 + 30…08, "1 2 Next"; Next →
556. # /movies/page/2 (client-side, 9 left, undated last), "1" →
557. # /movies, Back → page 2, direct load, page 99 → 2, abc → 1.
558. # /shows: series only, order, no pagination for one page; a
559. # tile → show page. Follow a MOVIE on its page → /my/movies
560. # has it, /my/shows does not. After the sync tool + restart
561. # /shows reorders (S03E01); after the IN-APP run (TMDB adds a
562. # released episode to show 2) /shows and the home row put
563. # show 2 first without a restart. Offline / from the cache
564. # still has the text and both rows.
565. # Session-sync (hybriel 64527baa): after each face-driven change
566. # (/my/unwatched click, expand, episode click, Follow) every row
567. # appears exactly once.
568. # tracker.worldapi.org#15: the fake TVmaze also serves
569. # /tvmaze/shows/<id>/episodes. First runs: lists agree with
570. # TMDB → nothing taken. Then TVmaze knows more: S3E4's title +
571. # date filled (TMDB "Episode 4"), S3E5 + a new S4 added, TMDB's
572. # S3E1 title kept, show 2 numbered differently → skipped;
573. # idempotent; /my/schedule + show page show them; TMDB catches
574. # up → adopts the rows by number (no duplicates, TMDB wins).
575. # tracker.worldapi.org#14: the header magnifier → /search
576. # (client-side); typing: 1 letter → "at least 2", "gate" →
577. # 3 titles in order with year/type/link, address /search/gate;
578. # people ("test act"); case/accents ("GATE tést"); nothing →
579. # note + Fetch; reload of the address = same results; bad %
580. # escapes → 200; "/" → space in the address. Fetch from web
581. # (fake TMDB search/multi): ours (tv 1001, movie 1003) and the
582. # person left out, w92 thumbs; signed out Add → modal, nothing
583. # sent. Signed in: Add tv 1006 (details 404) → error, every
584. # list unchanged (hybriel#121/#122); Add tv 1004 → its page
585. # with season/episodes/poster/genre/4 id links, exact TMDB
586. # calls; then in our results and gone from the web list; Add
587. # movie 1005 → /movie/ only. Screenshots search-{390,1280},
588. # search-web-{390,1280}, search-signed-out-modal-1280.
589. # Merge #13 + #14: an imported title is on /movies and /shows
590. # at once (search.hl → catalog.hl refreshCatalogShow); #13's
591. # 32 extra fixture titles are 'Pager …' so 'gate' finds only #14's.
592. # Mission 054 (adult titles): tests/seed-adult.hl adds, app
593. # stopped, 1 adult movie alice follows + 14 titles with an
594. # unknown flag (fake TMDB: 1 adult, 1 404). Backfill off:
595. # none on /, /movies (both pages), /shows, the search; the
596. # adult one on alice's /my/movies + its page; Fetch from web
597. # drops TMDB's adult result. Backfill on (fake TMDB 300 ms per
598. # answer): stopped after 6 titles, restarted → only the rest is
599. # asked (resume), pages < 1.2 s meanwhile, then the lists and
600. # the search have the non-adult ones at once, the poster is
601. # served, a 3rd start has nothing to do. Screenshot
602. # adult-home-1280.png. startServer() runs TRACKER_BACKFILL=0
603. # unless a block switches it on.
604. # 184 checks (#13 + #14 + #15 + 1 merge check + 17 mission 054). Own servers :8700 (this app) / :8701 (ident copy)
605. # / :8710 (fake TMDB); own storage .scratch/gate-store; Chrome
606. # on 8702-8709. Other ports (workers get 8720-8739):
607. # TRACKER_GATE_PORT=8720 TRACKER_GATE_IDENT_PORT=8721 TRACKER_GATE_CHROME=8722-8726 TRACKER_GATE_TMDB_PORT=8727 node tests/browser.mjs
608. # TRACKER_GATE_SHOTS=/tmp/x also writes {home,show,my-shows,
609. # my-unwatched,my-schedule,movies,movies-page-2,shows-list,
610. # my-movies}-{390,1280}.png, signed-out-modal-
611. # 1280.png, synced-*.png (after the sync, 1280×900; synced-show-390
612. # + synced-movie-390 at 390) + pwa-phone-
613. # online/offline.png (390 px, dpr 2) — LOOK at them
614. ps -eo pid,args | grep [h]l-browser-tier # must print nothing afterwards
615. ```
617. ## Deploy (Byrodin)
619. Target: `/CONTAINERS/projects/tracker.worldapi.org` on Byrodin, container
620. `tracker.worldapi.org` (`docker-compose.yml`: debian:12-slim, host network,
621. `HL_HOST=127.0.0.1`, `TRACKER_PORT=45008`, `TRACKER_WATCH=0`, the folder mounted at
622. `/home/tracker`, `./bin/hybriel project.hl`), public https://tracker.worldapi.org/ via
623. nginx (TLS ends there; no baseUrl/tls in the app, like ident/notes).
624. * **First deploy: done by the architect** (folder, nginx vhost with WebSocket Upgrade
625. headers, cert, DNS, **and registering this app in the live ident** — `/apps` → name +
626. origin `https://tracker.worldapi.org` → the API key + secret go into `.env` next to
627. `docker-compose.yml`, `TRACKER_KEY=… TRACKER_SECRET=…`).
628. * **Later: `./deploy.sh`** on Loreana, in this folder: runs the gate (refuses on a failure;
629. `--skip-tests` skips it LOUDLY), backs up `storage/`/`.sessions/`/`.env` (what exists) to
630. Loreana's `/media/SLOW1TB2/deploy-backups/<app>/` (newest 5 kept; an empty/failed backup stops
631. the deploy), then rsyncs the code to
632. `[email protected]:/CONTAINERS/projects/tracker.worldapi.org` (never `storage/`,
633. `.sessions/`, `.env`, `.scratch/`, `server.*`, logs — the preview is checked for them; no
634. `--delete`), `docker compose up -d && docker compose restart` over `ssh -F /dev/null`,
635. then waits for https://tracker.worldapi.org/ to answer 200.
636. * `./deploy.sh --dry-run` = the gate + `rsync -n` + the commands it would run (no restart, no
637. URL check). `--target DIR|HOST:DIR` and `--url URL` point it elsewhere (tested against a
638. local directory, see STATUS.md).
640. ## Data
642. `storage/mpackdb/`: `users.db` (`identity` → this app's own user id, `users.hl`) plus, since
643. step 2, the old tracker's data — `genres.db` (27), `persons.db` (15157), `shows.db` (9453),
644. `seasons.db` (6430), `episodes.db` (231584), `follows.db` (128), `watches.db` (11692), all with
645. fresh mpackdb ids and every reference re-pointed (`tools/migrate.hl`); the one old user is this
646. app's user for ident short id `az5b2`. Since step 9 the TMDB sync adds seasons/episodes and
647. `storage/mpackdb/posters/<oldId>.<ext>` (w342, ~50 KB each), and sets `tmdbSync`/`tmdbPoster` on
648. shows. Mission 054: `shows.adult` (true/false/null = unknown → hidden from lists + search) and `shows.adultCheck` (why the
649. backfill gave up on a title, e.g. `TMDB 404`). Known in the migrated data (not caused by the sync): 12 duplicate episode tmdbIds and 1
650. episode whose `show`/`season` disagrees among the followed shows. Counts and the referential-integrity proof:
651. `STATUS.md` "ticket #2". Real data 2026-10-01: 7,814 movies + 1,639 series (`shows.type`), 232,384 episodes (935
652. series have a released one).
654. ## Files
656. | File | |
657. |---|---|
658. | `CONCEPT.md` | the creator's concept — do not edit |
659. | `project.hl` | manifest: the installable app (`appIcons`, `appTouchIcon`, `appFavicon`, theme colours from the tokens, `offline = [ Home ]`, #10), the daily TMDB run (`syncTick`, #9), routes (`/login/callback`, `/login/failed`, `/login.js`, `/posters/:name`, `/movies` + `/movies/page/:page`, `/shows` + `/shows/page/:page` (#13), `/shows/:slug`, `/my/shows`, `/my/movies` (#13), `/my/unwatched`, `/my/schedule`, `/unwatched` + `/schedule` → 301, `/search/:q?` (#14), `/` component Home), the app's own session cookie; #14: builds the search index at boot (`ensureIndex`); mission 054: the adult backfill (`backfillTick`) on the clock it shares with the daily run |
660. | `users.hl` | the ident exchange + the `usersTable`, copied from calendar.worldapi.org's `users.hl` |
661. | `login.js` | bridge between `<ident-selector>`'s `ident-login` event and the shell, copied verbatim from calendar.worldapi.org |
662. | `components/home.hl` | `/` (#13): signed in the 4 icon tiles; the text; the latest 10 movies and series (catalog.hl) |
663. | `components/movies.hl`, `components/allshows.hl` | `/movies` (+ `/movies/page/:page`), `/shows` (+ `/shows/page/:page`): every movie / series, 24 per page (#13) |
664. | `components/tilelist.hl` | the poster grid + pagination both lists compose; turns a pagination click into `navigate()` (#13) |
665. | `components/pagination.hl` | the pagination, copied verbatim from components.hybriel.worldapi.org (`components/pagination/pagination.hl`, 4b4dabe) — re-copy, don't edit |
666. | `components/mymovies.hl` | `/my/movies`: followed movies, newest follow first (#13) |
667. | `catalog.hl` | the public lists: built once at start, refreshed per synced show and per new day (#13); only `adult == false` titles (mission 054) |
668. | `components/main.hl` | the shell (header: brand, `<ident-selector>`, log in / log out; the offline note + its `netProbe` tick, #10) |
669. | `components/loginfailed.hl` | `/login/failed`: why a login didn't work |
670. | `components/show.hl` | `/shows/:slug`: the show page (header, Follow toggle, seasons with caret, episodes, watch check; signed out the sign-in modal) |
671. | `components/modal.hl` | the modal dialog, copied verbatim from components.hybriel.worldapi.org (`components/modal/modal.hl`, becbd59) — re-copy, don't edit |
672. | `components/unwatched.hl` | `/my/unwatched`: unwatched, already-released episodes of followed shows, newest first |
673. | `components/schedule.hl` | `/my/schedule`: not-yet-released episodes of followed shows, soonest first, no checks |
674. | `components/myshows.hl` | `/my/shows`: followed SERIES (#13: movies → `/my/movies`), newest follow first — small poster, title, last watched `SxxEyy` |
675. | `shows.hl` | read-only data access (shows/seasons/episodes/genres/persons), plus the list labels `episodeCode`/`titleWithYear`/`posterUrlOf` (#8), reused by the `/my/` pages; `externalLinksOf` (#12: the link icons); `isPublicTitle` (mission 054: `adult == false`) |
676. | `follows.hl` | per-user follow data (`followsOfUser`, `followedShowIds`), reused by `/unwatched`/`/schedule`/`/my/shows`; `allFollowedShowIds` (#9: every show anyone follows, for the sync); `isFollowing`/`setFollowed` (mission 047: the show page's Follow toggle) |
677. | `tmdbsync.hl` | the TMDB sync of one show (`syncShow`: seasons, episodes, poster; #12: external ids + TVmaze lookup, movies via `/movie/`; #15: TVmaze episode merge `mergeTvmaze`) + run totals — used by `project.hl`'s daily run and `tools/sync-tmdb.hl` (#9); mission 054: `adultOf`, the adult backfill (`adultBackfillQueue`, `backfillTitle`) |
678. | `watches.hl` | per-user watch state (episode/season checks, the shared check-icon class `watchClassOf`, `watchedSet` = the user's watches as a map, built once per page), reused by `/unwatched`/`/schedule`/`/my/shows` |
679. | `icons/` | #10: `icon.svg` (source) → `icon-192.png`, `icon-512.png`, `apple-touch-icon.png`; `favicon.svg`, `favicon.ico` (commands in "What it does (step 10)") |
680. | `styles.hl` | all CSS (imports the tokens from `shared/tokens.hl`; accent green `#4ec9b0`) |
681. | `shared/tokens.hl` | the WorldAPI tokens, vendored verbatim from `ident.worldapi.org/shared/tokens.hl` |
682. | `tests/browser.mjs` | the gate (above) |
683. | `tests/seed-adult.hl` | mission 054: the gate's adult/unknown-flag titles + alice's follow of the adult one, written with the app stopped |
684. | `tests/seed-show.hl` | writes the gate's fixture show (two seasons, three episodes) and, since ticket #5, a follow of it for the test user (#7: plus a 2nd, later-followed, never-watched show; #12: partial external ids + a MOVIE followed by another user), before the server starts |
685. | `tests/identkit.mjs` | starts a throwaway ident copy for the gate, copied from calendar.worldapi.org's `tests/` |
686. | `tests/faketmdb.mjs` | the gate's fake TMDB (#9): details, `append_to_response=season/N`, w342 png; records every request; #12: `external_ids`, `/3/movie/<id>`, a fake TVmaze under `/tvmaze` (`tvmazeRequests`); #15: `/tvmaze/shows/<id>/episodes` from `tvmaze.episodes` |
687. | `tests/realdata-check.mjs` | NOT the gate: the real-data browser check on a COPY of live `storage/` (STATUS.md, mission 047 / #13): times of every page incl. `/`, `/movies`, `/shows`, `/my/movies`, pagination clicks + Back, client-side navigation, screenshots to `$REAL_SHOTS`; env REAL_PORT / REAL_CHROME / REAL_OUT / REAL_SHOW (#15) |
688. | `search.hl` | #14: the search — in-memory index (`ensureIndex`, `dbSearch`), `queryOfParam`, TMDB `webSearch`, `importTitle` (new record + `syncShow`) |
689. | `components/search.hl` | #14: `/search` + `/search/<text>` — the field, our titles/people as you type, "Fetch from web", "Add" (signed out: the sign-in modal) |
690. | `tests/realdata-search.mjs` | #14, NOT the gate: the search on a COPY of live `storage/` with the real TMDB (STATUS.md "ticket #14"): keystroke times, web search, ONE import, screenshots |
691. | `tools/count-external-ids.hl` | #12: how many shows have each external id, per type, all / followed — ONLY on a copy |
692. | `tools/count-duplicate-ids.hl` | duplicate `@id`s per table — ONLY on a copy (hybriel#113 check, mission 047) |
693. | `tests/cdp.mjs`, `tests/ports.mjs` | the CDP browser driver, copied from `ident.worldapi.org/tests/` |
694. | `docker-compose.yml`, `Dockerfile`, `deploy.sh` | Byrodin container; the deploy from Loreana (section "Deploy") |
695. | `tools/migrate.hl` | one-off: old MongoDB export → `storage/mpackdb/` (step 2, idempotent, see `STATUS.md`) |
696. | `tools/verify.hl` | one-off: proves the migrated data against a COPY of `storage/mpackdb/` (counts, zero dangling references, one show end to end) |
697. | `tools/sync-tmdb.hl` | the TMDB sync from the command line (#9) — app stopped or a copy; `--limit N`; prints counts |
698. | `tools/relink-episode-seasons.hl` | one-off: links episodes with no `season` (step 2 leftovers) by `(show, seasonNumber)` — not run against the live data yet, see "What it does (step 4)" |
700. ## Vendored Hybriel
702. `bin/hybriel` + `plugins/` = **hybriel master ff51cf46** (2026-10-01 00:27, mission 047; includes
703. #113 b08384e8, #115/#116 428bc56c/6b721937, #118 f5d709a4, the client-built SVG fix, hl:markdown),
704. `bin/hybriel` sha256 `55a724a0cf8680247616a8b66b792cc0ee3ce797afe80d764347ee242d6facc5`. No local
705. patch. Built read-only from an archive (never in the hybriel checkout):
706. ```bash
707. git -C /media/STORAGE/projects/hybriel archive master | tar x -C .scratch/hybriel-047
708. cd .scratch/hybriel-047/native && /media/STORAGE/projects/hybriel/native/zig-toolchain/zig build -Dtarget=x86_64-linux-gnu.2.39 -Doptimize=ReleaseFast # → zig-out/bin/hybriel (hybriel README + ReleaseFast)
709. cp zig-out/bin/hybriel ../../../bin/hybriel
710. # plugins: copy every plugin dir the app already has (core crypto data fetch fs http http1 mpackdb proc smtp
711. # time web) from .scratch/hybriel-047/plugins/ over plugins/ — `diff -rq plugins .scratch/hybriel-047/plugins`
712. # must then list only plugins the app does not use
713. ```
714. The previous build (837fe120 via ident, sha256 `9e5e95b3…`) is kept in `.scratch/pre-047/` (bin/ +
715. plugins/). After a re-vendor run the gate.