gitoriaLog in with ident

tracker

All repositories: gitoria

ReadmeCodePull requestsReleasesTicketsSettings
Commitdc40d859dc40d859tracker#31 (mission 026): duplicate titles merged — the 68 type+tmdbId pairs held by 157 records were the old tracker's (all migrated); merge.hl repair job (own clock, before the TMDB jobs) keeps one keeper per title (follows/watches > old short id > oldest), moves follows, watches, seasons, cast, credits, timelines, tombstones the rest (mergedInto, never deleted), slugs + short ids 301 to the keeper; stray seasons merged into their listed twin (Reacher S3 watches) or linked when watched; search import re-checks before its put; deploy.sh waits up to 90 s for 200; real copy 68 -> 0 dup ids, az5b2 follows/watches equal; gates 342/0, 32/0, 52/0mredc40d859/reports/022-tracker-headings-series-split.md

9.0 KB

  1. # Report 068 — tracker#20 typed headings + tracker#21 Series / Shows split
  2. **Commit**: `1ed5457` on branch `t20` (worktree loreana:/media/STORAGE/projects/tracker-wt20, from fa1f9df). Not pushed, not merged,
  3. not deployed. main and the other worktrees were not touched.
  4. ## Built
  5. - **#20**: every heading reads "<Type> | <name>": the type word in its colour, a muted bar, the name in white.
  6. - Title page: Movie / Series / Show. Also genre and person pages.
  7. - Lists `/movies`, `/series`, `/shows` show their type word; `/my/series` and `/my/shows` say "My Series" / "My Shows".
  8. - The same colours on the homepage row headings, the search labels and the filmography labels.
  9. - Colours: Movie yellow, Series greenDark (the accent), Show purple, Genre blue, Person orange.
  10. - Franchise (blueLight) and Timeline (greenLight) are defined as CSS classes only, for t19's pages.
  11. - **#21**: every TV title has a kind, `series` or `show`.
  12. - Show = TMDB type Reality / Talk Show / News / Video, or one of our genres Reality / Talk / News. Everything else is a series.
  13. - `kind` and `tmdbType` are stored by the daily sync, the adult backfill, the search import, and a new **kind backfill**:
  14. - one TMDB request per title that has no show genre;
  15. - it runs after the adult backfill and resumes after a restart;
  16. - `TRACKER_KINDS=0` switches it off.
  17. - Before a title has a stored kind, its genres decide.
  18. - **Pages and addresses**:
  19. - Lists: `/series` and `/shows` (+ `/page/N`).
  20. - Title pages: `/series/<slug>`, `/shows/<slug>`, and `/movies/<slug>` (new).
  21. - Old links: `/shows/<slug>` of a series → **301** `/series/<slug>`; of a movie → **301** `/movies/<slug>`.
  22. - `/my/series` and `/my/shows`.
  23. - Home: 5 tiles (Unwatched, Schedule, Series, Shows, Movies) and 3 rows.
  24. - Every link goes through `shows.hl titlePath`: lists, search, filmography, `/my/*`, `/<shortId>`, the page an import opens.
  25. - **Docs**: `docs/kinds.md` (new). README (summary, config, What it does, Test, Files). STATUS entry on top.
  26. ## Decisions
  27. - **Movies moved to `/movies/<slug>`.** `/shows/` now means TV shows; old movie links answer 301. This is my decision, not in
  28. the ticket.
  29. - **`/shows/<slug>` is a function route** (`project.hl showsRoute`). hl:web cannot redirect from a component route.
  30. - For a show, the route renders the Show component itself through hl:web internals (`server.router.match`, `server.page`,
  31. `hostOf`, `requestHeaders`, `sessions`).
  32. - Re-check this when hl:web is re-vendored. The kinds gate covers it: hydration, a season face and Follow all work on that page.
  33. - Consequence: a link to a show is a full page load. Series and movie pages are still client-side.
  34. - The search's TMDB results ("Fetch from web") say **TV** instead of a kind, because TMDB's search has no TV type.
  35. ## Files touched (merge notes: t26 and t19 also change show.hl, project.hl, styles.hl)
  36. | File | Change |
  37. |---|---|
  38. | `project.hl` | imports, `showsRoute`, routes (`/movies/:slug`, `/series*`, `/shows/:slug` as a function route, `/my/:tv` instead of `/my/shows`), the kind backfill block + one clock line + one log line |
  39. | `components/show.hl` | header comment, 1 import, `slugInfoOf` replaces `initialCollapsedOf`, `pageKind` / `typeText` / `typeClass` members, the h1 line |
  40. | `styles.hl` | 1 import line, colour tokens after `colorGenre`, a "typed headings" block before "/shows/:slug", home h2 colours, `homeNav` 3 columns (5 in the media block) |
  41. | `shows.hl` | new block "kind of a title" before `posterName` |
  42. | `catalog.hl` | two TV lists (`shows` beside `series`), `rekeyed`, `latestShows`, `showsPage`, counts, `tileOf` href |
  43. | `tmdbsync.hl` | 1 import, header comment, type stored in `syncShow` and `backfillTitle`, `needsKind` / `kindBackfillQueue` / `kindTitle` |
  44. | `search.hl` | import, `tmdbType` / `kind` on an import, `path` in its answer |
  45. | `people.hl` | import, filmography `href` / `typeLabel` / `typeClass` |
  46. | `shortids.hl` | `shortIdPath` uses `titlePath` |
  47. | `components/home.hl`, `myshows.hl`, `allshows.hl`, `movies.hl`, `genre.hl`, `person.hl`, `search.hl`, `mymovies.hl`, `unwatched.hl`, `schedule.hl` | headings, links, labels |
  48. | `tests/browser.mjs` | URLs (`/series/`, `/movies/`, `/my/series`, `#home-series`), h1 → `type-name` selectors, 5 home tiles, web label "TV", `TRACKER_KINDS: '0'` in `startServer` |
  49. | `tests/faketmdb.mjs` | `type: s.type` in TV details |
  50. New files:
  51. - `components/allseries.hl`
  52. - `docs/kinds.md`
  53. - `tests/kinds.mjs`, `tests/seed-kinds.hl`, `tests/realdata-068.mjs`
  54. - `tools/count-kinds.hl`
  55. ## Gates (final tree)
  56. ```
  57. TRACKER_GATE_SHOTS=/tmp/w068/shots TRACKER_GATE_PORT=8761 TRACKER_GATE_IDENT_PORT=8762 TRACKER_GATE_CHROME=8763-8765 TRACKER_GATE_TMDB_PORT=8766 node tests/kinds.mjs
  58. 32 passed, 0 failed
  59. TRACKER_GATE_SHOTS=/tmp/w068/main-shots TRACKER_GATE_PORT=8761 TRACKER_GATE_IDENT_PORT=8762 TRACKER_GATE_CHROME=8763-8765 TRACKER_GATE_TMDB_PORT=8766 node tests/browser.mjs
  60. 266 passed, 0 failed
  61. ```
  62. - The shots directory must exist before `tests/browser.mjs` runs (`mkdir -p` first).
  63. - One earlier run of browser.mjs hit an mpackdb LockTimeout in the short-id resume check right after a server stop. The rerun passed.
  64. `tests/kinds.mjs` covers:
  65. - Kinds from the stored data.
  66. - The sync storing the type: a followed Reality title moves to `/shows` without a restart.
  67. - The kind backfill: stop and resume, no title asked twice, ≥ 260 ms between requests, a 404 handled, nothing to do on the next start.
  68. - Both lists, the 301s, and every heading's words and computed colours.
  69. - The function-rendered show page: a season closes and opens; Follow works.
  70. - Signed in: home rows and tiles, `/my/series` and `/my/shows`.
  71. - Search labels and links; a talk-show import opens `/shows/<slug>`.
  72. - Filmography labels.
  73. - 390 / 1280 px: nothing scrolls sideways, and the long name wraps.
  74. ## Real data (copy of the live `storage/`, :8760, real TMDB; copy, sessions and `.env` link removed afterwards)
  75. - **Backfill**: 1,789 TV titles in **838 s**, 1,613 requests, 11 TMDB 404s, 0 retries.
  76. - **Kinds**: 1,577 series and 212 shows. 176 shows came from the genre (no request), 36 from TMDB's type.
  77. - **TMDB types**: Scripted 1,337, Miniseries 179, Documentary 50, Reality 16, Talk Show 15, Video 4, News 1.
  78. - **Public lists**: 1,601 series / 173 shows before the backfill (genres only), **1,565 / 209** after; 4,650 movies.
  79. - **Spot checks** (12 names; 10 are in our data, all correct):
  80. - Show: Saturday Night Live, The Tonight Show Starring Jimmy Fallon, Last Week Tonight, Late Show with Stephen Colbert, Hot Ones.
  81. - Series: Reacher, Foundation, Doctor Who, Chernobyl, Severance.
  82. - **Page times during the backfill** (curl every 5 s, 156 each, avg / max):
  83. | Page | avg | max |
  84. |---|---|---|
  85. | `/` | 0.094 s | 3.5 s |
  86. | `/series` | 0.055 s | 2.6 s |
  87. | `/shows` | 0.024 s | 0.21 s |
  88. | `/series/Reacher` | 0.19 s | 0.48 s |
  89. - 4 answers took more than 1 s, about every 170 s. The cause was not found.
  90. - Idle afterwards: average 0.029 s.
  91. - **Browser** (signed in as the creator, full load in ms):
  92. | Page | 390 | 1280 |
  93. |---|---|---|
  94. | `/series/Reacher` | 290 | 243 |
  95. | `/shows/Saturday-Night-Live` | 395 | 380 |
  96. | movie page | 251 | 246 |
  97. | `/my/series` (126 rows) | 1,410 | 384 |
  98. | `/my/shows` | 220 | 217 |
  99. | every list / genre / person / search | ≤ 351 | ≤ 351 |
  100. - Headings and colours right, nothing scrolls sideways, 0 console problems.
  101. ## Screenshots (looked at, on Loreana)
  102. - Gate: `/tmp/w068/shots/{series-page,show-page,movie-page,genre,person,series-list,shows-list,movies-list,home,my-series,my-shows,search}-{390,1280}.png`
  103. - Main gate: `/tmp/w068/main-shots/`
  104. - Real data: `/tmp/w068/real/{home,series-list,shows-list,movies-list,series,show,movie,genre,person,my-series,my-shows,search}-{390,1280}.png`
  105. - `/tmp/w068` also holds older files from other workers; mine are the names above.
  106. ## Open points
  107. 1. **Merge** with main (062), t19 and t26.
  108. - t19's franchise pages need the typed h1 (`type-franchise` / `type-timeline`).
  109. - t19's `/shows/` links should use `titlePath`. Until then they still work through the 301.
  110. 2. **TMDB type "Video"** is anime OVAs here (Halo Legends, .hack//Liminality, Carnival Phantasm). The architect's rule makes them
  111. Shows. Should Video become Series? (creator to decide)
  112. 3. **Deploy**: the first start runs the kind backfill after the adult backfill, about 14 min.
  113. 4. The 2–3.5 s answers during the backfill (probably mpackdb rewriting shows.db; not investigated).
  114. 5. Titles added by the person fill get a stored kind only at the next start. Until then the genres decide.
  115. 6. **Ports**: on Loreana 8765 (node pid 1047) and 8766 on the tailscale address belong to other services. My gates still ran (the
  116. fake TMDB binds 127.0.0.1). Nothing of mine is left running.
  117. ## Hybriel
  118. - **No component-route redirect.** hl:web cannot answer 301 from a component route or fall through from a function route.
  119. - Workaround: `showsRoute` renders the component through hl:web internals.
  120. - Candidate for a hybriel ticket: a component route should be able to answer a redirect, or a function route should be able to
  121. hand over to the next route.
  122. - No other new issues.

Branches

Latest commits

  • dc40d859tracker#31 (mission 026): duplicate titles merged — the 68 type+tmdbId pairs held by 157 records were the old tracker's (all migrated); merge.hl repair job (own clock, before the TMDB jobs) keeps one keeper per title (follows/watches > old short id > oldest), moves follows, watches, seasons, cast, credits, timelines, tombstones the rest (mergedInto, never deleted), slugs + short ids 301 to the keeper; stray seasons merged into their listed twin (Reacher S3 watches) or linked when watched; search import re-checks before its put; deploy.sh waits up to 90 s for 200; real copy 68 -> 0 dup ids, az5b2 follows/watches equal; gates 342/0, 32/0, 52/0mre
  • 2667da05tracker#30 (mission 025): /my/ pages from slim cached title cards, episode rows and watch sets (after the jobs /my/series 1.8 s -> 0.06 s, /my/unwatched 4.1 -> 0.18 s); timeline page shows its name once; franchise widget under the poster/title; movies with TV leftovers (First Contact) go through the details repair; tools/count-tmdb-ids.hl; gates 327/0, 52/0, 32/0mre
  • 3909810dantcolony#40: mission references in README/STATUS/docs point to the moved missionsmre
  • e63b1d28antcolony#40: history (LOG.md), worker briefs (missions/) and reports moved here from antcolony, numbered per project; old numbers in antcolony docs/mission-map.mdmre
  • 1cda451dtracker: Hybriel master 190aa11d (#127 both shapes, GC correctness fc838894) — conductor adopts despite /my/series 2.4x after jobs (memory 8.0 → 1.7 GB boot); see reports/071mre
  • c1fa2f2etracker (mission 071): Hybriel master 190aa11d measured on the real copy vs the live binary 8590df63 — NOT adopted (after the first-start jobs /my/series 2.4x slower, /series 1.6x, RSS swings 7.4-12.2 GB; fresh it is flat at 1.7-2.2 GB and /my/unwatched faster), vendor stays 8efba065, candidate kept in .scratch/w071/vendor-190aa11d; tests/kinds.mjs: collection seed off (its TMDB request broke check 1 in 1 of 4 runs); tests/realdata-071.sh + realdata-071-bench.mjs + tools/realdata-071-table.py; README + STATUS (numbers, how to repeat); gates 325/0, 32/0, 50/0mre
  • 8081350atracker docs (mission 070): README (summary, Config HL_GC_BYTES — kept at Hybriel's default, the 256 MiB setting is taken out of docker-compose.yml again: the jobs grew to 12+ GB with it too, see STATUS), Test (three gates), Deploy (first start ~50 min: kinds then seed, restart once after collections done, memory numbers), Vendored Hybriel 8efba065 + #48 audit, Files; STATUS mission 070 entry (merges, migrated counts, lambda audit, gates, RSS old vs new, how to repeat, open points); docs/kinds.md + docs/franchises.md job order; tests/realdata-070-*.sh, tools/count-migrated.hl, tools/ref-params.py, tools/lambda-audit.pymre
  • 1ad19c8ctracker: re-vendor Hybriel master 8efba065 (#126 GC by bytes, #48 lambda parameters copy) (mission 070): bin/hybriel sha256 50361e95…, plugins core crypto data fetch fs http http1 mpackdb proc smtp time web; lambda audit: 13 lambdas change a passed record/list (11 through a local alias), no caller relies on it — unchanged; 319 read-only lambda parameters get & (no copy per call: /my/schedule 2.35 → 0.46 s, /my/unwatched 13.7 → 5.7 s on the real copy); /my/unwatched one merge sort instead of n² inserts; docker-compose HL_GC_BYTES=268435456; deploy.sh runs kinds.mjs + franchises.mjs too (default ports 8700–8710); tests/realdata-070.mjs; gates browser 325/0, kinds 32/0, franchises 50/0mre
  • 46b21389tracker (mission 070, conductor): the sync never destroys migrated data — the one-time summary step MOVES a copied summary to migratedSummary (marker summaries-moved.txt) instead of clearing it; a migrated record's first title/genres/homepage/tagline TMDB replaces → migratedTitle/migratedGenres/migratedHomepage/migratedTagline, a migrated season's/episode's title/summary → migratedTitle/migratedSummary (set once); tests/peek-shows.hl prints them; gate 325/0mre
  • 749019b0Merge t19 (tracker#19 franchises + timelines) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs/faketmdb.mjs, both sides kept; franchise/timeline pages get #20's typed heading (Franchise | …, Timeline | …), their title links via titlePath (/movies|/series|/shows); the collection seed waits for repair, kinds and credits too; franchises.mjs URLs + 2 new checks; gates browser 323/0, kinds 32/0, franchises 50/0mre
  • daf49feaMerge t20 (tracker#20 typed headings + #21 series/shows split) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/search.hl/components/search.hl/browser.mjs/faketmdb.mjs, both sides kept; clock order backfill → repair → kinds → credits; withDetailsFields stores tmdbType + kind; gates: browser.mjs URLs → /series|/movies, typed h1 selectors; kinds.mjs repair/credits off, fixture name = seed name; browser 323/0, kinds 32/0mre
  • 20e09d89Merge t18 (tracker#18 delta sync) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs, both sides kept; pageShowOf summary = summaryOfmre
  • f83571c3tracker#18 (mission 067): daily sync by change lists — TMDB /tv|movie/changes (since the stored day, paged) + TVmaze /updates/shows → only our changed titles (followed: full step, unfollowed: light step — changed seasons, no TVmaze), full walk on first run / gap > 14 days / failed list; show record refreshed (title, tmdbSummary, tagline, status, genres …; renamed titles re-indexed); summary = the creator's own text (page: summary > tmdbSummary > tvmazeSummary), one-time clear of copied summaries (9,647 on the live copy); gate 261, tests/realdata-018*.mjs, README + STATUSmre
  • 1ed5457etracker#20 + #21 (mission 068): typed headings "<Type> | <name>" in type colours; TV titles split into Series (/series) and Shows (/shows) by TMDB type + Reality/Talk/News genres — kind stored by sync/import/adult backfill + new kind backfill (resumes), /movies/<slug>, /shows/<slug> of a series/movie → 301, /my/series + /my/shows, home 5 tiles + 3 rows, search/filmography labels; gates kinds 32 + browser 266, tests/realdata-068.mjs, tools/count-kinds.hl, docs/kinds.md, README + STATUSmre
  • f2b00674Merge t26 (tracker#26 + #28) into main (mission 062): short ids for every new person (castPersonId, guest route), guest stars stored on the title and created as people only when opened (/person/tmdb/<id>?show=<id> → 302), lean watch/follow clicks (showRow a small object, cast/crew from the slug, watches cached per user, face rows only after a season toggle); gate 311, tests/realdata-062.mjs, README + STATUSmre
  • 0553b51ftracker#19 (mission 066): franchises and timelines — tables, /franchises, /franchises/<slug>, /timelines/<slug> (Timeline | Release sort, series by last episode), the Prequel | Timeline | Sequel widget with the franchise above, the creator's editor, TMDB collection seed in the app (resumes, paced); gate tests/franchises.mjs 48/0 + browser.mjs 266/0, tests/realdata-066.mjs, docs/franchises.md, README + STATUSmre
  • fa1f9dfatracker#29 (mission 063): unwatched check muted grey outline + check (accent only on hover), watched stays solid — no code regression, the accent outline read as ticked; gate checks real checks visibly (computed style + screenshot pixel) on /my/unwatched, show, movie, /my/movies; gate 266, tests/realdata-063.mjs, README + STATUSmre
  • 10bb3f93tracker#28 (mission 061): full cast (all seasons, main cast by episodes, guest stars) + crew (created by, directed by, written by, screenplay, story, music) — stored by the details completion, the daily sync, the search import (one details request) and a background credits job (resumes, RSS limit); show page collapsed after 20 with client-side Show all; showBySlug via a slug map; gate 249, tests/realdata-028.mjs, README + STATUSmre
  • d8b12d67tracker#27 (mission 060): short ids for movies, series and persons — old 702 kept (data/old-short-ids.json), new random [a-z0-9]{5} unique across both, claimed at creation, background backfill (resumes), shown under poster/photo, /<shortId> → 301; gate 259, tests/realdata-060*, README + STATUSmre
  • 25a50bc4tracker#26 (mission 059): titles from a filmography are completed — on open (skeleton, step-wise face showComplete, no reload) and by the in-app details repair (resumes, TMDB-paced, series in parts); cast from TMDB credits; gate 231, tests/realdata-026.mjs, README + STATUSmre