gitoriaLog in with ident

tracker

All repositories: gitoria

ReadmeCodePull requestsReleasesTicketsSettings
Commitb10f00c8b10f00c8tracker#39: double episodes — migrated episodes whose TMDB id TMDB replaced are adopted by their number in the sync (old id -> migratedTmdbId); merge.hl step 3 merges each season's doubles at start (keeper: most watches > synced > first; watches moved/parked; tombstones into mergedEpisodes, nothing deleted); tools/count-duplicate-episodes.hl; gate fixture + paths-m039; live copy 850 -> 0 in 64 s; gates 373/0, 32/0, 52/0mreb10f00c8/docs/franchises.md

6.7 KB

  1. # Franchises and timelines (tracker.worldapi.org#19, mission 020 (old 066))
  2. Creator: "we have franchises, and timelines, and some franchises have multiple timelines ... make them sortable by timeline or
  3. release date at shows the release date of the last episode counts. in a show or movie page we show a franchise widget with
  4. < Prequel | TimelineName | Sequel > and if the timeline is part of a franchise that franchise is above it and when you click on
  5. it you get the franchise index." Names: "Franchise", "Timeline"; navigation `/franchises`.
  6. ## Data (`franchises.hl`, new tables — nothing in `shows.db` changes)
  7. | Table | Fields |
  8. |---|---|
  9. | `franchises.db` (`!slug`) | `name`, `slug`, `shortId` (always null for now), `summary`, `created` |
  10. | `timelines.db` (`!slug`) | `name`, `slug`, `franchise` (@id or null), `order` [show @id …] (in-world order), `source` (`tmdb`/`editor`), `tmdbCollection` (TMDB id or null), `created` |
  11. | `collectionchecks.db` | pk = the movie's show @id; `collection` (TMDB id or null), `why` (`collection`, `collection known`, `no collection`, `TMDB 404`, `collection 404`), `at` — the seed's progress |
  12. * Slugs like the shows' urlSegment: `Star Trek — Prime` → `Star-Trek-Prime`; taken → `-2`, `-3`; never `edit`.
  13. * A title may be in several timelines → one widget per timeline. `byShow` (show @id → [timeline @id]) is built at start
  14. (`ensureFranchises`, one scan) and kept current by every edit.
  15. * Inspect: the pages (`/franchises` lists every timeline with its count), or a small .hl script opening
  16. `new MPackDB(file = '<copy>/timelines.db', primaryKey = '@id', indexes = ['!slug'])` + `find(null, null)` — ONLY on a copy
  17. or with the app stopped (opening a table rewrites its files, hybriel#40).
  18. ## Ordering
  19. * **Timeline** = `order` as edited (the seed: TMDB collection parts by `release_date`).
  20. * **Release** = a movie's `release`; a SERIES' LAST episode: the newest one already out (≤ today, UTC), else the newest
  21. announced one, else the series' own `release`. No date → last. Equal dates keep the timeline order. Shown per row as
  22. `1996-11-18` / `last episode 2020-06-15`.
  23. * Franchise index: its timelines oldest first (by their earliest release).
  24. * Public like all lists (mission 010 (old 054)): pages and prequel/sequel links skip titles that are not `adult == false`; the editor
  25. shows every title.
  26. ## Pages
  27. | Route | Component | |
  28. |---|---|---|
  29. | `/franchises` | `components/franchises.hl` | franchises (→ index, n timelines), then the timelines without one (TMDB collections) with n titles; creator: "Edit" |
  30. | `/franchises/<slug>` | `components/franchise.hl` | the index: name, summary, each timeline a header (name → its page, count, `Timeline | Release` toggle) + its rows |
  31. | `/timelines/<slug>` | `components/franchise.hl` | one timeline, its franchise above it; creator: "Edit" |
  32. | `/franchises/edit` | `components/franchiseedit.hl` | creator: new franchise (name, summary), new timeline (name → its editor), find a timeline by name |
  33. | `/timelines/<slug>/edit` | `components/timelineedit.hl` | creator: franchise chips (None + all; a tap attaches), rows ↑ ↓ ✕, "Add titles" (search.hl index) |
  34. | widget on `/shows/<slug>` | `components/franchisewidget.hl` | composed by `components/show.hl` under the header: franchise above, `‹ Prequel | name | Sequel ›`, the title under each side, a missing side greyed out (`span.off`) |
  35. The sort toggle is client-only (`flatRows` is pure; both orders come with the page). One flat row list (`head`/`item`
  36. rows) instead of a nested for (hybriel#86).
  37. **Editor = the creator only**: `isEditorSession` = the signed-in user's ident id is in `TRACKER_EDITORS` (default `az5b2`).
  38. Every editing face (`fedCreateFranchise`, `fedCreateTimeline`, `tedSetFranchise`, `tedMove`, `tedRemove`, `tedAdd`) checks it
  39. again server-side; `fedFindTimelines` / `tedSearch` are public reads. Not built: rename/delete of franchises or timelines,
  40. editing a franchise's summary (open point).
  41. ## The seed (TMDB movie collections)
  42. In the app, own `every(0.1)` clock (`project.hl collectionTick` → `franchises.hl seedStep`), ONE movie per step:
  43. 1. `GET /movie/<tmdbId>` → `belongs_to_collection` (none → check row `no collection`; 404 → `TMDB 404`; other failure →
  44. back of the queue, ≤ 2 retries per start).
  45. 2. A collection seen before (`collectionsDone`, from check rows + timelines) → just a check row, no further request.
  46. 3. Else `GET /collection/<id>` → parts sorted by `release_date`, mapped to OUR non-adult movies by tmdbId. **≥ 2 → one
  47. timeline** (`source = 'tmdb'`, no franchise); < 2 → none (`small`). Every matched part gets a check row at once, so its
  48. own details request is never made.
  49. Pause after a step: 260 ms × requests made (≤ 40 / 10 s, like the adult backfill). Waits while the daily sync or the adult
  50. backfill runs. Queue at start = every movie with a tmdbId, `adult != true`, without a check row → resumes after a restart;
  51. nothing to do → one log line. Persisted every `TRACKER_COLLECTIONS_PERSIST_EVERY` (25) steps and at the end. A hand-edited
  52. timeline is never touched. Log: `collections: start, N movies to check`, every `TRACKER_COLLECTIONS_LOG_EVERY` (500)
  53. `collections: pos/N movies, checked= timelines= titles= small= noCollection= notFound= requests= errors= retries= seconds=`,
  54. `collections done: …`. `TRACKER_COLLECTIONS=0` switches it off (off too without `TMDB_READ_TOKEN`).
  55. Re-seed (e.g. after many imports): delete `collectionchecks.db*` with the app stopped — existing collection timelines are
  56. skipped (their `tmdbCollection` is known); NEW parts of an existing collection are not added (open point).
  57. Real data: STATUS.md, ticket #19.
  58. ## Test
  59. * Gate: `tests/franchises.mjs` (own storage `.scratch/franchise-store`, fixture `tests/seed-franchises.hl` on top of
  60. `tests/seed-show.hl`, fake TMDB `collections`): seed + resume + pacing, widget (ends, client-side Sequel click, franchise
  61. above), `/franchises`, index sort both ways (series by last episode), editor only for `TRACKER_EDITORS`, 390/1280 no
  62. sideways scroll, screenshots with `TRACKER_GATE_SHOTS`.
  63. * `tests/browser.mjs` runs with `TRACKER_COLLECTIONS=0` (its fake TMDB request counts must not see the seed).
  64. * Real data: `tests/realdata-066.mjs` (procedure in STATUS.md).
  65. ## Since the merge (mission 023 (old 070))
  66. * Headings are #20's typed ones: "Franchises" (blueLight), "Franchise | <name>", "Timeline | <name>" (greenLight), also on the
  67. timeline editor. Title links use `shows.hl titlePath` (`/movies/…`, `/series/…`, `/shows/…`).
  68. * The seed waits for the daily sync, the adult backfill, the details repair, the kind backfill AND the credits job (one TMDB
  69. pacing at a time): on the first start after the deploy it runs last (~32–37 min after the kind backfill's ~16–21 min).

Branches

Latest commits

  • b10f00c8tracker#39: double episodes — migrated episodes whose TMDB id TMDB replaced are adopted by their number in the sync (old id -> migratedTmdbId); merge.hl step 3 merges each season's doubles at start (keeper: most watches > synced > first; watches moved/parked; tombstones into mergedEpisodes, nothing deleted); tools/count-duplicate-episodes.hl; gate fixture + paths-m039; live copy 850 -> 0 in 64 s; gates 373/0, 32/0, 52/0mre
  • 8751adb8tracker: report 032mre
  • 9bce1f65tracker mission 032: STATUS gate files + the hour-boundary flakemre
  • 718bfb89tracker#37 (mission 032): /people = everyone, last updated first (updatedAt stamped by the person fill; view built at boot, touched people first at once), photo + name tiles (person colour) with the /movies pagination, /people/<letter> removed; photo = our file, tmdbProfile, a cast/crew entry's profile (in-memory map at boot), else the new 'no photo' placeholder; new cast/crew/created_by people keep tmdbProfile; search people rows with the photo; /settings = the heading only; util.hl sortDesc starts from sorted runs (same result, 105k: 1.6 s -> 0.15 s); gates 369/0, 32/0, 52/0, check-theme 0; README/STATUS/LOGmre
  • 96ba683adeploy.sh: a gate without a 'passed,' line (check-theme) no longer ends the scriptmre
  • eb3b9205tracker: report 031mre
  • 9b5d2e89tracker mission 031: README (What it does, Test: four gates + the #32 checks, Files: theme/, new pages), STATUS (real copy, A/B load, how to repeat, open points), LOGmre
  • 39950e4ctracker#32 (mission 031): the WorldAPI theme (theme/ vendored verbatim from layouts.worldapi.org 85b5654; styles.hl inherits it: accent green-dark, type colours 1-6; own base/header rules, row lines, genre-pill and inverted-button frames removed, the season foldable keeps its line; check-theme 21 -> 0, 4th deploy gate; main actions class primary) and the #32 header (theme AppHeader/MainMenu/UserMenu/Sidebar/ContentFirst: desktop brand, search, Series|Shows|Movies|Genres|People, user icon with Unwatched..Settings, Logout; signed out the ident selector, phone the iD icon dropdown; phone menu in the sidebar overlay; marked entry by :has); /find -> /search/<q>, /genres, /people(/<letter>), /settings; main { ContentFirst { slot } } works around the hl:web one-line slot bug; gates 365/0, 32/0, 52/0, check-theme 0mre
  • a386dc92tracker: reports 029 + 030mre
  • 71e0fd7dtracker missions 029 + 030: README (What it does, Files, gate count), STATUS (real-copy numbers, how to repeat, open points), LOGmre
  • d36ea6eatracker#34 + #35 (mission 030): Follow directly under the poster, as wide as the poster (show.hl, styles.hl); the status pill next to a series' title — TVmaze's status (new tvmazeStatus, stored by the sync's TVmaze merge) else TMDB's, TVmaze Ended + TMDB Canceled = Canceled, inverted (filled, dark text, no border), green running / yellow pending / red canceled / muted ended (shows.hl statusOf); the daily delta asks TVmaze's status of an unfollowed series TVmaze's change list names (dailysync.hl syncRunStep, sync.hl syncTvmazeStatus); the status backfill after the details repair (backfill.hl, jobs.hl statusTick; resumable, 550 ms per TVmaze request); gates 354/0, 32/0, 52/0mre
  • 7d7d4487tracker#33 (mission 029): reduced titles — every title TMDB's details never went through this app (no detailsAt, no tmdbSync) is incomplete (shows.hl isIncomplete; the old tracker's migrated rows passed #26's test: 5,697 non-adult on the live copy, 691 series without seasons); the repair job does the visibly reduced first (shows.hl missingParts), the page completes one on open; a title TMDB has no poster for (The Remaining) shows the placeholder; tools/count-incomplete.hl; gate fixtures stand for synced titles (tmdbSync), tests/seed-reduced.hl + #33 checks; gates 347/0, 32/0, 52/0mre
  • 661c2592tracker: report 028mre
  • 27c916fatracker mission 028: README ("Code order", the new file map), STATUS (counts before/after, tests, how to repeat, open), LOGmre
  • d924f398tracker mission 028: comments name the new files (sync.hl, dailysync.hl, backfill.hl, credits.hl, jobs.hl, images.hl …); tools/ref-params.py + tools/lambda-audit.py also scan lib/ (they globbed the root only), lambda-audit counts a plain `x = p` alias like `let x = p`mre
  • 2e89b968tracker mission 028 (code order) 5/5 let: `let` only where a variable is reassigned — 667 never-reassigned lets became plain declarations (project.hl, lib/, components/, tools/, tests/); kept: 264 in loop bodies (a plain declaration there is 'Cannot reassign' on the 2nd pass), 234 reassigned, 27 whose name is also a member/outer/free name (a plain write would rebind it); tools/let-audit.py decides and fixes (README 'Code order'); tests/realdata-m028.{sh,mjs} = the page-output diff on a real copy; gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 54796ff2tracker mission 028 (code order) 4/5 thin faces + last copies: the show page's check/follow faces call lib/watches.hl toggleWatched / toggleSeasonWatched (seasonAllWatched moved there) and lib/follows.hl toggleFollowed; both logins (header selector face, /login/callback) share lib/users.hl userOfCode; todayStr/listOf copies in components and the export readers copied into tools/migrate.hl + tools/old-short-ids.hl now once (lib/util.hl, lib/export.hl); gates 342/0, 32/0, 52/0; old-short-ids output byte-identical, migrate output identicalmre
  • 06b078e3tracker mission 028 (code order) 3/5 project.hl is the map: config, routes, wiring and a feature → file index (914 → 258 lines); the background jobs (daily sync run, backfills, details repair, credits job, merge, short ids, collection seed) moved unchanged into lib/jobs.hl (a class: their state is reassigned every step, a static cannot be; one instance made after the server), the login callback into lib/users.hl, poster/photo serving into lib/images.hl, the /shows/<slug> rule into lib/shows.hl showsMovedPath; route handlers are thin wrappers; gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 94716fd2tracker mission 028 (code order) 2/5 util + topics: lib/util.hl holds envOr, storageDir, postersDir, profilesDir, newId, hexDigits, todayStr, dateOr, textOr, hasId, listOr, firstOf, sortDesc once (were copied into up to 5 files); tmdbsync.hl split into tmdb.hl (TMDB/TVmaze requests), sync.hl (one title's sync), sync-helpers.hl, backfill.hl; details.hl split into details.hl, credits.hl, credits-helpers.hl (isIncomplete to shows.hl); search-helpers.hl (words, query, ranking, slugs); collections.hl (the TMDB collection seed, out of franchises.hl); deltasync.hl renamed dailysync.hl; no behaviour change: gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 186079b0tracker mission 028 (code order) 1/5 move: every root .hl except project.hl into lib/ (styles.hl into components/), import paths only; gates 342/0, 32/0, 52/0; real-copy pages identicalmre