gitoriaLog in with ident

tracker

All repositories: gitoria

ReadmeCodePull requestsReleasesTicketsSettings
Commit54796ff254796ff2tracker mission 028 (code order) 4/5 thin faces + last copies: the show page's check/follow faces call lib/watches.hl toggleWatched / toggleSeasonWatched (seasonAllWatched moved there) and lib/follows.hl toggleFollowed; both logins (header selector face, /login/callback) share lib/users.hl userOfCode; todayStr/listOf copies in components and the export readers copied into tools/migrate.hl + tools/old-short-ids.hl now once (lib/util.hl, lib/export.hl); gates 342/0, 32/0, 52/0; old-short-ids output byte-identical, migrate output identicalmre54796ff2/reports/022-tracker-headings-series-split.md

9.0 KB

  1. # Report 068 — tracker#20 typed headings + tracker#21 Series / Shows split
  2. **Commit**: `1ed5457` on branch `t20` (worktree loreana:/media/STORAGE/projects/tracker-wt20, from fa1f9df). Not pushed, not merged,
  3. not deployed. main and the other worktrees were not touched.
  4. ## Built
  5. - **#20**: every heading reads "<Type> | <name>": the type word in its colour, a muted bar, the name in white.
  6. - Title page: Movie / Series / Show. Also genre and person pages.
  7. - Lists `/movies`, `/series`, `/shows` show their type word; `/my/series` and `/my/shows` say "My Series" / "My Shows".
  8. - The same colours on the homepage row headings, the search labels and the filmography labels.
  9. - Colours: Movie yellow, Series greenDark (the accent), Show purple, Genre blue, Person orange.
  10. - Franchise (blueLight) and Timeline (greenLight) are defined as CSS classes only, for t19's pages.
  11. - **#21**: every TV title has a kind, `series` or `show`.
  12. - Show = TMDB type Reality / Talk Show / News / Video, or one of our genres Reality / Talk / News. Everything else is a series.
  13. - `kind` and `tmdbType` are stored by the daily sync, the adult backfill, the search import, and a new **kind backfill**:
  14. - one TMDB request per title that has no show genre;
  15. - it runs after the adult backfill and resumes after a restart;
  16. - `TRACKER_KINDS=0` switches it off.
  17. - Before a title has a stored kind, its genres decide.
  18. - **Pages and addresses**:
  19. - Lists: `/series` and `/shows` (+ `/page/N`).
  20. - Title pages: `/series/<slug>`, `/shows/<slug>`, and `/movies/<slug>` (new).
  21. - Old links: `/shows/<slug>` of a series → **301** `/series/<slug>`; of a movie → **301** `/movies/<slug>`.
  22. - `/my/series` and `/my/shows`.
  23. - Home: 5 tiles (Unwatched, Schedule, Series, Shows, Movies) and 3 rows.
  24. - Every link goes through `shows.hl titlePath`: lists, search, filmography, `/my/*`, `/<shortId>`, the page an import opens.
  25. - **Docs**: `docs/kinds.md` (new). README (summary, config, What it does, Test, Files). STATUS entry on top.
  26. ## Decisions
  27. - **Movies moved to `/movies/<slug>`.** `/shows/` now means TV shows; old movie links answer 301. This is my decision, not in
  28. the ticket.
  29. - **`/shows/<slug>` is a function route** (`project.hl showsRoute`). hl:web cannot redirect from a component route.
  30. - For a show, the route renders the Show component itself through hl:web internals (`server.router.match`, `server.page`,
  31. `hostOf`, `requestHeaders`, `sessions`).
  32. - Re-check this when hl:web is re-vendored. The kinds gate covers it: hydration, a season face and Follow all work on that page.
  33. - Consequence: a link to a show is a full page load. Series and movie pages are still client-side.
  34. - The search's TMDB results ("Fetch from web") say **TV** instead of a kind, because TMDB's search has no TV type.
  35. ## Files touched (merge notes: t26 and t19 also change show.hl, project.hl, styles.hl)
  36. | File | Change |
  37. |---|---|
  38. | `project.hl` | imports, `showsRoute`, routes (`/movies/:slug`, `/series*`, `/shows/:slug` as a function route, `/my/:tv` instead of `/my/shows`), the kind backfill block + one clock line + one log line |
  39. | `components/show.hl` | header comment, 1 import, `slugInfoOf` replaces `initialCollapsedOf`, `pageKind` / `typeText` / `typeClass` members, the h1 line |
  40. | `styles.hl` | 1 import line, colour tokens after `colorGenre`, a "typed headings" block before "/shows/:slug", home h2 colours, `homeNav` 3 columns (5 in the media block) |
  41. | `shows.hl` | new block "kind of a title" before `posterName` |
  42. | `catalog.hl` | two TV lists (`shows` beside `series`), `rekeyed`, `latestShows`, `showsPage`, counts, `tileOf` href |
  43. | `tmdbsync.hl` | 1 import, header comment, type stored in `syncShow` and `backfillTitle`, `needsKind` / `kindBackfillQueue` / `kindTitle` |
  44. | `search.hl` | import, `tmdbType` / `kind` on an import, `path` in its answer |
  45. | `people.hl` | import, filmography `href` / `typeLabel` / `typeClass` |
  46. | `shortids.hl` | `shortIdPath` uses `titlePath` |
  47. | `components/home.hl`, `myshows.hl`, `allshows.hl`, `movies.hl`, `genre.hl`, `person.hl`, `search.hl`, `mymovies.hl`, `unwatched.hl`, `schedule.hl` | headings, links, labels |
  48. | `tests/browser.mjs` | URLs (`/series/`, `/movies/`, `/my/series`, `#home-series`), h1 → `type-name` selectors, 5 home tiles, web label "TV", `TRACKER_KINDS: '0'` in `startServer` |
  49. | `tests/faketmdb.mjs` | `type: s.type` in TV details |
  50. New files:
  51. - `components/allseries.hl`
  52. - `docs/kinds.md`
  53. - `tests/kinds.mjs`, `tests/seed-kinds.hl`, `tests/realdata-068.mjs`
  54. - `tools/count-kinds.hl`
  55. ## Gates (final tree)
  56. ```
  57. TRACKER_GATE_SHOTS=/tmp/w068/shots TRACKER_GATE_PORT=8761 TRACKER_GATE_IDENT_PORT=8762 TRACKER_GATE_CHROME=8763-8765 TRACKER_GATE_TMDB_PORT=8766 node tests/kinds.mjs
  58. 32 passed, 0 failed
  59. TRACKER_GATE_SHOTS=/tmp/w068/main-shots TRACKER_GATE_PORT=8761 TRACKER_GATE_IDENT_PORT=8762 TRACKER_GATE_CHROME=8763-8765 TRACKER_GATE_TMDB_PORT=8766 node tests/browser.mjs
  60. 266 passed, 0 failed
  61. ```
  62. - The shots directory must exist before `tests/browser.mjs` runs (`mkdir -p` first).
  63. - One earlier run of browser.mjs hit an mpackdb LockTimeout in the short-id resume check right after a server stop. The rerun passed.
  64. `tests/kinds.mjs` covers:
  65. - Kinds from the stored data.
  66. - The sync storing the type: a followed Reality title moves to `/shows` without a restart.
  67. - The kind backfill: stop and resume, no title asked twice, ≥ 260 ms between requests, a 404 handled, nothing to do on the next start.
  68. - Both lists, the 301s, and every heading's words and computed colours.
  69. - The function-rendered show page: a season closes and opens; Follow works.
  70. - Signed in: home rows and tiles, `/my/series` and `/my/shows`.
  71. - Search labels and links; a talk-show import opens `/shows/<slug>`.
  72. - Filmography labels.
  73. - 390 / 1280 px: nothing scrolls sideways, and the long name wraps.
  74. ## Real data (copy of the live `storage/`, :8760, real TMDB; copy, sessions and `.env` link removed afterwards)
  75. - **Backfill**: 1,789 TV titles in **838 s**, 1,613 requests, 11 TMDB 404s, 0 retries.
  76. - **Kinds**: 1,577 series and 212 shows. 176 shows came from the genre (no request), 36 from TMDB's type.
  77. - **TMDB types**: Scripted 1,337, Miniseries 179, Documentary 50, Reality 16, Talk Show 15, Video 4, News 1.
  78. - **Public lists**: 1,601 series / 173 shows before the backfill (genres only), **1,565 / 209** after; 4,650 movies.
  79. - **Spot checks** (12 names; 10 are in our data, all correct):
  80. - Show: Saturday Night Live, The Tonight Show Starring Jimmy Fallon, Last Week Tonight, Late Show with Stephen Colbert, Hot Ones.
  81. - Series: Reacher, Foundation, Doctor Who, Chernobyl, Severance.
  82. - **Page times during the backfill** (curl every 5 s, 156 each, avg / max):
  83. | Page | avg | max |
  84. |---|---|---|
  85. | `/` | 0.094 s | 3.5 s |
  86. | `/series` | 0.055 s | 2.6 s |
  87. | `/shows` | 0.024 s | 0.21 s |
  88. | `/series/Reacher` | 0.19 s | 0.48 s |
  89. - 4 answers took more than 1 s, about every 170 s. The cause was not found.
  90. - Idle afterwards: average 0.029 s.
  91. - **Browser** (signed in as the creator, full load in ms):
  92. | Page | 390 | 1280 |
  93. |---|---|---|
  94. | `/series/Reacher` | 290 | 243 |
  95. | `/shows/Saturday-Night-Live` | 395 | 380 |
  96. | movie page | 251 | 246 |
  97. | `/my/series` (126 rows) | 1,410 | 384 |
  98. | `/my/shows` | 220 | 217 |
  99. | every list / genre / person / search | ≤ 351 | ≤ 351 |
  100. - Headings and colours right, nothing scrolls sideways, 0 console problems.
  101. ## Screenshots (looked at, on Loreana)
  102. - Gate: `/tmp/w068/shots/{series-page,show-page,movie-page,genre,person,series-list,shows-list,movies-list,home,my-series,my-shows,search}-{390,1280}.png`
  103. - Main gate: `/tmp/w068/main-shots/`
  104. - Real data: `/tmp/w068/real/{home,series-list,shows-list,movies-list,series,show,movie,genre,person,my-series,my-shows,search}-{390,1280}.png`
  105. - `/tmp/w068` also holds older files from other workers; mine are the names above.
  106. ## Open points
  107. 1. **Merge** with main (062), t19 and t26.
  108. - t19's franchise pages need the typed h1 (`type-franchise` / `type-timeline`).
  109. - t19's `/shows/` links should use `titlePath`. Until then they still work through the 301.
  110. 2. **TMDB type "Video"** is anime OVAs here (Halo Legends, .hack//Liminality, Carnival Phantasm). The architect's rule makes them
  111. Shows. Should Video become Series? (creator to decide)
  112. 3. **Deploy**: the first start runs the kind backfill after the adult backfill, about 14 min.
  113. 4. The 2–3.5 s answers during the backfill (probably mpackdb rewriting shows.db; not investigated).
  114. 5. Titles added by the person fill get a stored kind only at the next start. Until then the genres decide.
  115. 6. **Ports**: on Loreana 8765 (node pid 1047) and 8766 on the tailscale address belong to other services. My gates still ran (the
  116. fake TMDB binds 127.0.0.1). Nothing of mine is left running.
  117. ## Hybriel
  118. - **No component-route redirect.** hl:web cannot answer 301 from a component route or fall through from a function route.
  119. - Workaround: `showsRoute` renders the component through hl:web internals.
  120. - Candidate for a hybriel ticket: a component route should be able to answer a redirect, or a function route should be able to
  121. hand over to the next route.
  122. - No other new issues.

Branches

Latest commits

  • 54796ff2tracker mission 028 (code order) 4/5 thin faces + last copies: the show page's check/follow faces call lib/watches.hl toggleWatched / toggleSeasonWatched (seasonAllWatched moved there) and lib/follows.hl toggleFollowed; both logins (header selector face, /login/callback) share lib/users.hl userOfCode; todayStr/listOf copies in components and the export readers copied into tools/migrate.hl + tools/old-short-ids.hl now once (lib/util.hl, lib/export.hl); gates 342/0, 32/0, 52/0; old-short-ids output byte-identical, migrate output identicalmre
  • 06b078e3tracker mission 028 (code order) 3/5 project.hl is the map: config, routes, wiring and a feature → file index (914 → 258 lines); the background jobs (daily sync run, backfills, details repair, credits job, merge, short ids, collection seed) moved unchanged into lib/jobs.hl (a class: their state is reassigned every step, a static cannot be; one instance made after the server), the login callback into lib/users.hl, poster/photo serving into lib/images.hl, the /shows/<slug> rule into lib/shows.hl showsMovedPath; route handlers are thin wrappers; gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 94716fd2tracker mission 028 (code order) 2/5 util + topics: lib/util.hl holds envOr, storageDir, postersDir, profilesDir, newId, hexDigits, todayStr, dateOr, textOr, hasId, listOr, firstOf, sortDesc once (were copied into up to 5 files); tmdbsync.hl split into tmdb.hl (TMDB/TVmaze requests), sync.hl (one title's sync), sync-helpers.hl, backfill.hl; details.hl split into details.hl, credits.hl, credits-helpers.hl (isIncomplete to shows.hl); search-helpers.hl (words, query, ranking, slugs); collections.hl (the TMDB collection seed, out of franchises.hl); deltasync.hl renamed dailysync.hl; no behaviour change: gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 186079b0tracker mission 028 (code order) 1/5 move: every root .hl except project.hl into lib/ (styles.hl into components/), import paths only; gates 342/0, 32/0, 52/0; real-copy pages identicalmre
  • 4f47f181tracker: report 027mre
  • dc1d4be4tracker mission 027: Hybriel master 06617221 vendored (plugin allocator fixes 3a781359 + 413f60e4); real copy RSS through first-start jobs + 400 loads flat ~2.55 GB (190aa11d 2.3 -> 5.6 GB), page times <= 1.1x; gates 342/0, 32/0, 52/0mre
  • 84e1b3e1tracker: reports 025 + 026mre
  • dc40d859tracker#31 (mission 026): duplicate titles merged — the 68 type+tmdbId pairs held by 157 records were the old tracker's (all migrated); merge.hl repair job (own clock, before the TMDB jobs) keeps one keeper per title (follows/watches > old short id > oldest), moves follows, watches, seasons, cast, credits, timelines, tombstones the rest (mergedInto, never deleted), slugs + short ids 301 to the keeper; stray seasons merged into their listed twin (Reacher S3 watches) or linked when watched; search import re-checks before its put; deploy.sh waits up to 90 s for 200; real copy 68 -> 0 dup ids, az5b2 follows/watches equal; gates 342/0, 32/0, 52/0mre
  • 2667da05tracker#30 (mission 025): /my/ pages from slim cached title cards, episode rows and watch sets (after the jobs /my/series 1.8 s -> 0.06 s, /my/unwatched 4.1 -> 0.18 s); timeline page shows its name once; franchise widget under the poster/title; movies with TV leftovers (First Contact) go through the details repair; tools/count-tmdb-ids.hl; gates 327/0, 52/0, 32/0mre
  • 3909810dantcolony#40: mission references in README/STATUS/docs point to the moved missionsmre
  • e63b1d28antcolony#40: history (LOG.md), worker briefs (missions/) and reports moved here from antcolony, numbered per project; old numbers in antcolony docs/mission-map.mdmre
  • 1cda451dtracker: Hybriel master 190aa11d (#127 both shapes, GC correctness fc838894) — conductor adopts despite /my/series 2.4x after jobs (memory 8.0 → 1.7 GB boot); see reports/071mre
  • c1fa2f2etracker (mission 071): Hybriel master 190aa11d measured on the real copy vs the live binary 8590df63 — NOT adopted (after the first-start jobs /my/series 2.4x slower, /series 1.6x, RSS swings 7.4-12.2 GB; fresh it is flat at 1.7-2.2 GB and /my/unwatched faster), vendor stays 8efba065, candidate kept in .scratch/w071/vendor-190aa11d; tests/kinds.mjs: collection seed off (its TMDB request broke check 1 in 1 of 4 runs); tests/realdata-071.sh + realdata-071-bench.mjs + tools/realdata-071-table.py; README + STATUS (numbers, how to repeat); gates 325/0, 32/0, 50/0mre
  • 8081350atracker docs (mission 070): README (summary, Config HL_GC_BYTES — kept at Hybriel's default, the 256 MiB setting is taken out of docker-compose.yml again: the jobs grew to 12+ GB with it too, see STATUS), Test (three gates), Deploy (first start ~50 min: kinds then seed, restart once after collections done, memory numbers), Vendored Hybriel 8efba065 + #48 audit, Files; STATUS mission 070 entry (merges, migrated counts, lambda audit, gates, RSS old vs new, how to repeat, open points); docs/kinds.md + docs/franchises.md job order; tests/realdata-070-*.sh, tools/count-migrated.hl, tools/ref-params.py, tools/lambda-audit.pymre
  • 1ad19c8ctracker: re-vendor Hybriel master 8efba065 (#126 GC by bytes, #48 lambda parameters copy) (mission 070): bin/hybriel sha256 50361e95…, plugins core crypto data fetch fs http http1 mpackdb proc smtp time web; lambda audit: 13 lambdas change a passed record/list (11 through a local alias), no caller relies on it — unchanged; 319 read-only lambda parameters get & (no copy per call: /my/schedule 2.35 → 0.46 s, /my/unwatched 13.7 → 5.7 s on the real copy); /my/unwatched one merge sort instead of n² inserts; docker-compose HL_GC_BYTES=268435456; deploy.sh runs kinds.mjs + franchises.mjs too (default ports 8700–8710); tests/realdata-070.mjs; gates browser 325/0, kinds 32/0, franchises 50/0mre
  • 46b21389tracker (mission 070, conductor): the sync never destroys migrated data — the one-time summary step MOVES a copied summary to migratedSummary (marker summaries-moved.txt) instead of clearing it; a migrated record's first title/genres/homepage/tagline TMDB replaces → migratedTitle/migratedGenres/migratedHomepage/migratedTagline, a migrated season's/episode's title/summary → migratedTitle/migratedSummary (set once); tests/peek-shows.hl prints them; gate 325/0mre
  • 749019b0Merge t19 (tracker#19 franchises + timelines) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs/faketmdb.mjs, both sides kept; franchise/timeline pages get #20's typed heading (Franchise | …, Timeline | …), their title links via titlePath (/movies|/series|/shows); the collection seed waits for repair, kinds and credits too; franchises.mjs URLs + 2 new checks; gates browser 323/0, kinds 32/0, franchises 50/0mre
  • daf49feaMerge t20 (tracker#20 typed headings + #21 series/shows split) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/search.hl/components/search.hl/browser.mjs/faketmdb.mjs, both sides kept; clock order backfill → repair → kinds → credits; withDetailsFields stores tmdbType + kind; gates: browser.mjs URLs → /series|/movies, typed h1 selectors; kinds.mjs repair/credits off, fixture name = seed name; browser 323/0, kinds 32/0mre
  • 20e09d89Merge t18 (tracker#18 delta sync) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs, both sides kept; pageShowOf summary = summaryOfmre
  • f83571c3tracker#18 (mission 067): daily sync by change lists — TMDB /tv|movie/changes (since the stored day, paged) + TVmaze /updates/shows → only our changed titles (followed: full step, unfollowed: light step — changed seasons, no TVmaze), full walk on first run / gap > 14 days / failed list; show record refreshed (title, tmdbSummary, tagline, status, genres …; renamed titles re-indexed); summary = the creator's own text (page: summary > tmdbSummary > tvmazeSummary), one-time clear of copied summaries (9,647 on the live copy); gate 261, tests/realdata-018*.mjs, README + STATUSmre