tracker
All repositories: gitoria
16.3 KB
// shows.hl — READ-ONLY data access for a show's page (tracker.worldapi.org#4): shows, seasons,// episodes, persons, genres (storage/mpackdb/, migrated by tools/migrate.hl, tracker.worldapi.org#2).// Kept apart from watches.hl (the per-user watch state) and from components/show.hl (the page itself)// so /unwatched and /schedule (tickets #5/#6, "reuse, don't copy") can import the same functions.//// No declared index beyond the migration's own ['!oldId'] (opening a table with indexes it wasn't// created with is untested territory, hybriel#21 "opening a table rewrites its files") — a slug// lookup is a linear scan of ~9500 shows, cheap for one page load.import { MPackDB } from 'hl:mpackdb'import { env } from 'hl:proc'import { exists } from 'hl:fs'static envOr = (&name, &fallback) => {let v = env(name)return v != null && v.trim() != '' ? v.trim() : fallback}static storageDir = envOr('TRACKER_STORAGE', './storage/mpackdb')static genresTable = new MPackDB(file = storageDir + '/genres.db', primaryKey = '@id', indexes = ['!oldId'])static personsTable = new MPackDB(file = storageDir + '/persons.db', primaryKey = '@id', indexes = ['!oldId'])static showsTable = new MPackDB(file = storageDir + '/shows.db', primaryKey = '@id', indexes = ['!oldId'])static seasonsTable = new MPackDB(file = storageDir + '/seasons.db', primaryKey = '@id', indexes = ['!oldId'])static episodesTable = new MPackDB(file = storageDir + '/episodes.db', primaryKey = '@id', indexes = ['!oldId'])// the show with this urlSegment (slug), or null. tracker#28: by a slug → id map (one scan, at the first call) — the old scan of// every record per call (twice per show page) decoded every title's cast + crew: 82 ms per call on the live copy before the// credits, growing with them. A slug the map does not know (a title added since: import, filmography) costs one scan as before// and is remembered; slugs never change. (Flat static written key by key, like people.hl's maps.)static slugIds = {}static slugIdsBuilt = { yes = false }static showBySlug = (&slug) => {if (slug == null || hlTypeName(slug) != 'String' || slug == '') { return null }if (!slugIdsBuilt.yes) {for (s of showsTable.find(null, null)) { if (s.urlSegment != null) { slugIds[s.urlSegment] = s.id } }slugIdsBuilt.yes = true}let id = slugIds[slug]if (id != null) {let hit = showsTable.fetch(id)if (hit != null && hit.urlSegment == slug) { return hit }}for (s of showsTable.find(null, null)) {if (s.urlSegment == slug) {slugIds[slug] = s.idreturn s}}return null}static genreById = (&id) => { return id == null ? null : genresTable.fetch(id) }// /genres/<genre> (mission 058): the genre with this urlSegment ('Action', 'Sci-Fi-Fantasy'), exact first, else ignoring case// — a scan of the genres table (27 rows)static genreBySlug = (&slug) => {if (slug == null || hlTypeName(slug) != 'String' || slug == '') { return null }let lower = slug.toLowerCase()let loose = nullfor (g of genresTable.find(null, null)) {if (g.urlSegment == slug) { return g }if (loose == null && g.urlSegment != null && ('' + g.urlSegment).toLowerCase() == lower) { loose = g }}return loose}static personById = (&id) => { return id == null ? null : personsTable.fetch(id) }static seasonById = (&id) => { return id == null ? null : seasonsTable.fetch(id) }static showById = (&id) => { return id == null ? null : showsTable.fetch(id) }static episodeById = (&id) => { return id == null ? null : episodesTable.fetch(id) }// EVERY show / episode (tracker.worldapi.org#13, catalog.hl): read ONCE when the app starts, to build the public lists —// never per requeststatic allShows = () => { return showsTable.find(null, null) }static allEpisodes = () => { return episodesTable.find(null, null) }// insertion sort, hand-written (no list sort(), hybriel#1) — a show has a handful of seasons at moststatic insertDesc = (&sorted, &s) => {let out = []let inserted = falsefor (x of sorted) {if (!inserted && s.seasonNumber >= x.seasonNumber) { out.push(s) inserted = true }out.push(x)}if (!inserted) { out.push(s) }return out}static insertAsc = (&sorted, &e) => {let out = []let inserted = falsefor (x of sorted) {if (!inserted && e.episodeNumber <= x.episodeNumber) { out.push(e) inserted = true }out.push(x)}if (!inserted) { out.push(e) }return out}// a show's seasons, latest (highest seasonNumber) first — creator: "all seasons in reverse order,// latest on top"static seasonsOfShow = (&show) => {let sorted = []for (id of show.seasons) {let s = seasonsTable.fetch(id)if (s != null) { sorted = insertDesc(sorted, s) }}return sorted}// a season's episodes, in episode-number orderstatic episodesOfSeason = (&season) => {let sorted = []for (id of season.episodes) {let e = episodesTable.fetch(id)if (e != null) { sorted = insertAsc(sorted, e) }}return sorted}// ---- labels shared by every list (tracker.worldapi.org#8) -------------------------------------static pad2 = (&n) => {let s = '' + nwhile (s.length < 2) { s = '0' + s }return s}// an episode as `S01E01` (creator: "write it like that: S01E01") — /my/shows, /my/unwatched, /my/schedulestatic episodeCode = (&seasonNumber, &episodeNumber) => { return 'S' + pad2(seasonNumber) + 'E' + pad2(episodeNumber) }// the show's year: `year`, else the first 4 characters of `release` ('YYYY-MM-DD'; 197 migrated shows have// no year, a few of those an empty release) — null when neither gives onestatic showYear = (&show) => {if (show.year != null) { return '' + show.year }if (show.release != null && hlTypeName(show.release) == 'String' && show.release.length >= 4) { return show.release.slice(0, 4) }return null}// a show's title as every list shows it: `Doctor Who (2005)` (creator: so same-named shows can be told apart)static titleWithYear = (&show) => {let y = showYear(show)return y != null ? show.title + ' (' + y + ')' : show.title}// ---- the description (tracker#18) ---------------------------------------------------------------// creator: "i put the summary in an own field for tmdb because some summaries i wanted to rewrite ... in that case the summary// has priority over maze or tmdb" — `summary` (the creator's own text, never written by the sync), else TMDB's `tmdbSummary`,// else TVmaze's `tvmazeSummary` (HTML stripped); '' when nonestatic hasText = (&v) => { return v != null && hlTypeName(v) == 'String' && v.trim() != '' }static summaryOf = (&show) => {if (show == null) { return '' }if (hasText(show.summary)) { return show.summary }if (hasText(show.tmdbSummary)) { return show.tmdbSummary }if (hasText(show.tvmazeSummary)) { return show.tvmazeSummary }return ''}// ---- adult titles (mission 054) -----------------------------------------------------------------// `adult` on every title: true / false from TMDB, null = not known yet (the migrated rows; the app's backfill, tmdbsync.hl// `backfillTitle`, asks TMDB). The PUBLIC lists (catalog.hl: homepage rows, /movies, /shows) and the search (search.hl) show// ONLY a title TMDB says is not adult — unknown counts as hidden. A title the user follows stays on /my/* and its page.static isPublicTitle = (&show) => { return show != null && show.adult == false }// ---- the kind of a title (tracker.worldapi.org#21, mission 068) ----------------------------------------------// A TV title (`type = 'series'`) is either a SERIES (TMDB's TV type Scripted, Miniseries, Documentary) or a SHOW (Reality,// Talk Show, News, Video — or one of our genres Reality / Talk / News, whatever the type). Stored on the title as `kind`// ('series' | 'show') and TMDB's type as `tmdbType`: by the kind backfill (tmdbsync.hl `kindTitle`, one TMDB request per// title without a show genre), the daily sync and the search import. Until a title has `kind`, its genres decide (no// type known yet = series). Movies are `type = 'movie'` and have no kind.static showTmdbTypes = ['Reality', 'Talk Show', 'News', 'Video']static showGenreNames = ['Reality', 'Talk', 'News']static showGenreIdsOf = () => {let out = {}for (g of genresTable.find(null, null)) { if (g.name != null && showGenreNames.includes('' + g.name)) { out[g.id] = true } }return out}// the genre ids that make a title a show — read once (the genres table never changes while the app runs)static showGenreIds = showGenreIdsOf()static hasShowGenre = (&genres) => {if (genres == null) { return false }for (g of genres) { if (showGenreIds[g] == true) { return true } }return false}// 'show' | 'series' from TMDB's type (null = not known) and the title's genre idsstatic kindOfTitle = (&tmdbType, &genres) => {if (hasShowGenre(genres)) { return 'show' }if (tmdbType != null && showTmdbTypes.includes('' + tmdbType)) { return 'show' }return 'series'}// 'movie' | 'series' | 'show' of a stored titlestatic titleKind = (&show) => {if (show.type == 'movie') { return 'movie' }if (show.kind == 'show' || show.kind == 'series') { return show.kind }return kindOfTitle(show.tmdbType, show.genres)}// a title's own page: /movies/<slug>, /series/<slug>, /shows/<slug> (the old /shows/<slug> of a movie or series answers 301,// project.hl showsRoute)static titlePath = (&show) => {let k = titleKind(show)return (k == 'movie' ? '/movies/' : (k == 'show' ? '/shows/' : '/series/')) + show.urlSegment}// tracker.worldapi.org#20: the type word of a page heading / list label and its colour class (styles.hl `type-*`)static kindWord = (&kind) => { return kind == 'movie' ? 'Movie' : (kind == 'show' ? 'Show' : 'Series') }static typeWordOf = (&show) => { return kindWord(titleKind(show)) }static typeClassOf = (&show) => { return 'type-word type-' + titleKind(show) }// the poster file name (oldId + the migrated extension) — null when either is missing; served at// /posters/<name> (project.hl), which falls back to a placeholder when the file itself isn't there// yet (the real poster files are not part of the mongo export, see the report's `open`)static posterName = (&show) => {if (show.oldId == null || show.image == null || show.image == '') { return null }return show.oldId + '.' + show.image}// the poster's URL for a LIST (/my/shows, tracker.worldapi.org#8): `/posters/<name>` only when the file really// is in storage/mpackdb/posters/ (project.hl posterRoute reads the same folder), else the ONE placeholder URL// `/posters/none` — the real data has no poster files yet, and 128 different URLs all answering the same// placeholder were 128 requests per page load (the browser can't cache them as one)static posterUrlOf = (&show) => { return posterUrlOfName(posterName(show)) }static posterUrlOfName = (&name) => { return name != null && exists(storageDir + '/posters/' + name) ? '/posters/' + name : '/posters/none' }// ---- slim per-title caches for the lists (tracker.worldapi.org#30, mission 025) ------------------------------------// /my/series took 4.4 s on a process that had run the first-start jobs (0.4 s fresh): every load fetched each followed title's// WHOLE record (cast + crew, up to 300 entries) and one episode per watch (11.7k fetches); /my/unwatched every season and// episode of every followed title. Each fetch decodes a new object, and with a big live heap (after the jobs) every byte// allocated feeds the byte-triggered GC. Now a list reads two small cached things per title:// - `cardOf(id)`: { id type kind title year titleYear href posterName } — what a row shows;// - `episodeRowsOf(id)`: every episode of the title as { id s e title release }, seasons latest first, episodes in order// (the same order as seasonsOfShow/episodesOfSeason);// - `episodeKeyOf(id)`: one episode as { show s e } — /my/series' last watched episode goes by the WATCHED episode's own// `show` (as before): migrated episodes the sync left outside the title's seasons (a duplicate season, e.g. Reacher S03 on// the real data) still count. project.hl warms every watched episode at boot.// Filled at the first ask (project.hl warms the followed titles at boot) and kept in sync by catalog.hl `refreshCatalogShow`,// which every writer of a title already calls after it changed one (daily sync, repair, kinds, adult backfill, imports):// it calls `refreshTitleCaches(show)`, which rebuilds the entries that exist. Cached values are only ever RETURNED from a// function (a `let x = map[k]` deep-copies the value; a return does not — measured: 3.7 ms per 20k-row copy).static cardCache = {}static episodeRowsCache = {}static episodeKeyCache = {}static cardFrom = (&show) => {let y = showYear(show)return {id = show.id type = show.type kind = titleKind(show) title = show.title year = y != null ? y : ''titleYear = titleWithYear(show) href = titlePath(show) posterName = posterName(show)}}static cardOf = (&id) => {if (id == null) { return null }if (cardCache[id] != null) { return cardCache[id] }let show = showsTable.fetch(id)if (show == null) { return null }cardCache[id] = cardFrom(show)return cardCache[id]}static episodeRowsFrom = (&show) => {let out = []if (show.seasons == null) { return out }// loop names apart from the row's own `s`/`e` keys: inside an object literal a key set earlier shadows the variablefor (sn of seasonsOfShow(show)) {for (ep of episodesOfSeason(sn)) {out.push({ id = ep.id s = ep.seasonNumber e = ep.episodeNumber title = ep.title != null ? ep.title : '' release = ep.release })}}return out}static episodeRowsOf = (&id) => {if (id == null) { return [] }if (episodeRowsCache[id] != null) { return episodeRowsCache[id] }let show = showsTable.fetch(id)if (show == null) { return [] }episodeRowsCache[id] = episodeRowsFrom(show)return episodeRowsCache[id]}// an episode that is not stored is { show = null } (never fetched again)static episodeKeyOf = (&id) => {if (id == null) { return { show = null s = null e = null } }if (episodeKeyCache[id] != null) { return episodeKeyCache[id] }let ep = episodesTable.fetch(id)episodeKeyCache[id] = ep != null ? { show = ep.show s = ep.seasonNumber e = ep.episodeNumber } : { show = null s = null e = null }return episodeKeyCache[id]}// a title changed (catalog.hl refreshCatalogShow): its cached entries again from the stored record; the episode keys of its// episodes from the new rows (numbers the sync changed). Keys of a title nobody follows (no rows cached) are not refreshed —// /my/series only reads followed titles, and warmTitleCaches covers every followed one at boot.static refreshTitleCaches = (&show) => {if (cardCache[show.id] != null) { cardCache[show.id] = cardFrom(show) }if (episodeRowsCache[show.id] != null) {episodeRowsCache[show.id] = episodeRowsFrom(show)let sid = show.idfor (r of episodeRowsOf(sid)) {if (episodeKeyCache[r.id] != null) { episodeKeyCache[r.id] = { show = sid s = r.s e = r.e } }}}return null}// project.hl at boot: the titles anyone follows and every watched episode, so the first /my/ load after a start is not the one// that fills themstatic warmTitleCaches = (&ids) => {let n = 0for (id of ids) {if (cardOf(id) != null) { episodeRowsOf(id) n = n + 1 }}return n}static warmEpisodeKeys = (&ids) => {for (id of ids) { episodeKeyOf(id) }return ids.length}// ---- the show on other sites (tracker.worldapi.org#12) ------------------------------------------// the header's link icons: TMDB, IMDb, TheTVDB, TVmaze — ONLY for the ids the show has; a movie (`type = 'movie'`)// gets TMDB + IMDb. TheTVDB's id form is `/dereferrer/series/<id>` (checked 2026-10-01: it and the old// `?tab=series&id=` both 301 to the slug page); the others are the sites' own id URLs (they too 301 to a slug).static hasExternalId = (&v) => { return v != null && ('' + v).trim() != '' }static externalLinksOf = (&show) => {let out = []let movie = show.type == 'movie'if (hasExternalId(show.tmdbId)) { out.push({ site = 'tmdb' label = 'TMDB' title = 'On TMDB' href = 'https://www.themoviedb.org/' + (movie ? 'movie/' : 'tv/') + show.tmdbId }) }if (hasExternalId(show.imdbId)) { out.push({ site = 'imdb' label = 'IMDb' title = 'On IMDb' href = 'https://www.imdb.com/title/' + show.imdbId + '/' }) }if (!movie && hasExternalId(show.tvdbId)) { out.push({ site = 'tvdb' label = 'TVDB' title = 'On TheTVDB' href = 'https://thetvdb.com/dereferrer/series/' + show.tvdbId }) }if (!movie && hasExternalId(show.tvmzId)) { out.push({ site = 'tvmaze' label = 'TVmaze' title = 'On TVmaze' href = 'https://www.tvmaze.com/shows/' + show.tvmzId }) }return out}
Branches
- mainmain branch
Latest commits
- 2667da05tracker#30 (mission 025): /my/ pages from slim cached title cards, episode rows and watch sets (after the jobs /my/series 1.8 s -> 0.06 s, /my/unwatched 4.1 -> 0.18 s); timeline page shows its name once; franchise widget under the poster/title; movies with TV leftovers (First Contact) go through the details repair; tools/count-tmdb-ids.hl; gates 327/0, 52/0, 32/0mre
- 3909810dantcolony#40: mission references in README/STATUS/docs point to the moved missionsmre
- e63b1d28antcolony#40: history (LOG.md), worker briefs (missions/) and reports moved here from antcolony, numbered per project; old numbers in antcolony docs/mission-map.mdmre
- 1cda451dtracker: Hybriel master 190aa11d (#127 both shapes, GC correctness fc838894) — conductor adopts despite /my/series 2.4x after jobs (memory 8.0 → 1.7 GB boot); see reports/071mre
- c1fa2f2etracker (mission 071): Hybriel master 190aa11d measured on the real copy vs the live binary 8590df63 — NOT adopted (after the first-start jobs /my/series 2.4x slower, /series 1.6x, RSS swings 7.4-12.2 GB; fresh it is flat at 1.7-2.2 GB and /my/unwatched faster), vendor stays 8efba065, candidate kept in .scratch/w071/vendor-190aa11d; tests/kinds.mjs: collection seed off (its TMDB request broke check 1 in 1 of 4 runs); tests/realdata-071.sh + realdata-071-bench.mjs + tools/realdata-071-table.py; README + STATUS (numbers, how to repeat); gates 325/0, 32/0, 50/0mre
- 8081350atracker docs (mission 070): README (summary, Config HL_GC_BYTES — kept at Hybriel's default, the 256 MiB setting is taken out of docker-compose.yml again: the jobs grew to 12+ GB with it too, see STATUS), Test (three gates), Deploy (first start ~50 min: kinds then seed, restart once after collections done, memory numbers), Vendored Hybriel 8efba065 + #48 audit, Files; STATUS mission 070 entry (merges, migrated counts, lambda audit, gates, RSS old vs new, how to repeat, open points); docs/kinds.md + docs/franchises.md job order; tests/realdata-070-*.sh, tools/count-migrated.hl, tools/ref-params.py, tools/lambda-audit.pymre
- 1ad19c8ctracker: re-vendor Hybriel master 8efba065 (#126 GC by bytes, #48 lambda parameters copy) (mission 070): bin/hybriel sha256 50361e95…, plugins core crypto data fetch fs http http1 mpackdb proc smtp time web; lambda audit: 13 lambdas change a passed record/list (11 through a local alias), no caller relies on it — unchanged; 319 read-only lambda parameters get & (no copy per call: /my/schedule 2.35 → 0.46 s, /my/unwatched 13.7 → 5.7 s on the real copy); /my/unwatched one merge sort instead of n² inserts; docker-compose HL_GC_BYTES=268435456; deploy.sh runs kinds.mjs + franchises.mjs too (default ports 8700–8710); tests/realdata-070.mjs; gates browser 325/0, kinds 32/0, franchises 50/0mre
- 46b21389tracker (mission 070, conductor): the sync never destroys migrated data — the one-time summary step MOVES a copied summary to migratedSummary (marker summaries-moved.txt) instead of clearing it; a migrated record's first title/genres/homepage/tagline TMDB replaces → migratedTitle/migratedGenres/migratedHomepage/migratedTagline, a migrated season's/episode's title/summary → migratedTitle/migratedSummary (set once); tests/peek-shows.hl prints them; gate 325/0mre
- 749019b0Merge t19 (tracker#19 franchises + timelines) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs/faketmdb.mjs, both sides kept; franchise/timeline pages get #20's typed heading (Franchise | …, Timeline | …), their title links via titlePath (/movies|/series|/shows); the collection seed waits for repair, kinds and credits too; franchises.mjs URLs + 2 new checks; gates browser 323/0, kinds 32/0, franchises 50/0mre
- daf49feaMerge t20 (tracker#20 typed headings + #21 series/shows split) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/search.hl/components/search.hl/browser.mjs/faketmdb.mjs, both sides kept; clock order backfill → repair → kinds → credits; withDetailsFields stores tmdbType + kind; gates: browser.mjs URLs → /series|/movies, typed h1 selectors; kinds.mjs repair/credits off, fixture name = seed name; browser 323/0, kinds 32/0mre
- 20e09d89Merge t18 (tracker#18 delta sync) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs, both sides kept; pageShowOf summary = summaryOfmre
- f83571c3tracker#18 (mission 067): daily sync by change lists — TMDB /tv|movie/changes (since the stored day, paged) + TVmaze /updates/shows → only our changed titles (followed: full step, unfollowed: light step — changed seasons, no TVmaze), full walk on first run / gap > 14 days / failed list; show record refreshed (title, tmdbSummary, tagline, status, genres …; renamed titles re-indexed); summary = the creator's own text (page: summary > tmdbSummary > tvmazeSummary), one-time clear of copied summaries (9,647 on the live copy); gate 261, tests/realdata-018*.mjs, README + STATUSmre
- 1ed5457etracker#20 + #21 (mission 068): typed headings "<Type> | <name>" in type colours; TV titles split into Series (/series) and Shows (/shows) by TMDB type + Reality/Talk/News genres — kind stored by sync/import/adult backfill + new kind backfill (resumes), /movies/<slug>, /shows/<slug> of a series/movie → 301, /my/series + /my/shows, home 5 tiles + 3 rows, search/filmography labels; gates kinds 32 + browser 266, tests/realdata-068.mjs, tools/count-kinds.hl, docs/kinds.md, README + STATUSmre
- f2b00674Merge t26 (tracker#26 + #28) into main (mission 062): short ids for every new person (castPersonId, guest route), guest stars stored on the title and created as people only when opened (/person/tmdb/<id>?show=<id> → 302), lean watch/follow clicks (showRow a small object, cast/crew from the slug, watches cached per user, face rows only after a season toggle); gate 311, tests/realdata-062.mjs, README + STATUSmre
- 0553b51ftracker#19 (mission 066): franchises and timelines — tables, /franchises, /franchises/<slug>, /timelines/<slug> (Timeline | Release sort, series by last episode), the Prequel | Timeline | Sequel widget with the franchise above, the creator's editor, TMDB collection seed in the app (resumes, paced); gate tests/franchises.mjs 48/0 + browser.mjs 266/0, tests/realdata-066.mjs, docs/franchises.md, README + STATUSmre
- fa1f9dfatracker#29 (mission 063): unwatched check muted grey outline + check (accent only on hover), watched stays solid — no code regression, the accent outline read as ticked; gate checks real checks visibly (computed style + screenshot pixel) on /my/unwatched, show, movie, /my/movies; gate 266, tests/realdata-063.mjs, README + STATUSmre
- 10bb3f93tracker#28 (mission 061): full cast (all seasons, main cast by episodes, guest stars) + crew (created by, directed by, written by, screenplay, story, music) — stored by the details completion, the daily sync, the search import (one details request) and a background credits job (resumes, RSS limit); show page collapsed after 20 with client-side Show all; showBySlug via a slug map; gate 249, tests/realdata-028.mjs, README + STATUSmre
- d8b12d67tracker#27 (mission 060): short ids for movies, series and persons — old 702 kept (data/old-short-ids.json), new random [a-z0-9]{5} unique across both, claimed at creation, background backfill (resumes), shown under poster/photo, /<shortId> → 301; gate 259, tests/realdata-060*, README + STATUSmre
- 25a50bc4tracker#26 (mission 059): titles from a filmography are completed — on open (skeleton, step-wise face showComplete, no reload) and by the in-app details repair (resumes, TMDB-paced, series in parts); cast from TMDB credits; gate 231, tests/realdata-026.mjs, README + STATUSmre
- f14db671tracker#22-#25 (mission 058): episode air dates, season check = all episodes watched, movie watched check (+ /my/movies count), /genres/<genre> pages (movies + series, newest first, paginated); gate 238, tests/realdata-058.mjs, README + STATUSmre