gitoriaLog in with ident

tracker

All repositories: gitoria

ReadmeCodePull requestsReleasesTicketsSettings
Commit9bce1f659bce1f65tracker mission 032: STATUS gate files + the hour-boundary flakemre9bce1f65/lib/people.hl

25.1 KB

  1. // lib/people.hl — THE PERSON PAGE's data (tracker.worldapi.org#16, components/person.hl `/person/<slug>`): a person by slug,
  2. // their facts and filmography, and the ONE-TIME fetch of their whole filmography from TMDB on the first visit. Creator:
  3. // "the movies come from artist scrolls. when one clicks on an artist that has no dataset yet the old version fetched that
  4. // artist and its entire bibilogrqphy so its profile was complete".
  5. //
  6. // personBySlug(slug) the person (slug = `urlSegment`) — a slug → id map built once (15k people, one scan)
  7. // personInfoOf(p) name, photo URL, born/died lines, bio
  8. // filmographyOf(p) their credits (`p.shows`): poster, title, year, role, kind — newest first, adult titles hidden
  9. // personFillStep(id) ONE step of the fetch, called again and again by the page until it answers `done`:
  10. // 1st call: GET <TMDB>/person/<tmdbId>?append_to_response=combined_credits (ONE request: details +
  11. // every cast/crew credit) + the photo (w185, like the posters); the person's facts are updated;
  12. // every next call stores the next `fillBatch` titles; the last one links the credits and marks
  13. // the person complete (`filmographyAt`, ms). hl:web serves one request at a time — between two
  14. // steps every other request is served, so a person with 300 credits never holds the server.
  15. // peoplePage(page) /people (tracker#37): everyone, the one updated last first, 24 per page — a view built once at boot
  16. // personPhotoOf(p) the photo of a /people tile and a search row: our file, else TMDB's (`tmdbProfile`), else the placeholder
  17. //
  18. // COMPLETE = `filmographyAt` set: never fetched again (the daily sync may refresh followed people later — not built).
  19. // Migrated people are NOT complete (153 of them carry the old tracker's credits — kept, the fetch adds the rest).
  20. // A title we don't have → a MINIMAL show record (title, year, type, tmdbId, `adult`, overview, genres by TMDB id, language;
  21. // no seasons, no poster file; `minimal = true` — details.hl completes it when it is opened or by the repair job, tracker#26), shaped like search.hl importTitle's
  22. // (`oldId tmdb-<tv|movie>-<id>`, a free slug), indexed for the search and put into the public lists (catalog.hl).
  23. // The person's credits (`shows`, the migrated field): [{ show, character, posterPath, adult }] — `character` is the role
  24. // text (the characters, then crew jobs like "Director", " / " between them); `posterPath` = TMDB's, so the page can show
  25. // TMDB's small poster (w92, like search's web rows) until we have the file. ADULT (TMDB's flag, mission 054): a row is
  26. // shown only for a PUBLIC title (shows.hl `isPublicTitle`: `adult == false`, unknown hidden like the public lists) whose
  27. // credit is not adult either (mission 056 merge).
  28. // ONE PROCESS PER STORAGE (hybriel#21); new rows get self-assigned 16-hex ids (hybriel#113), like sync.hl.
  29. import { now } from 'hl:time'
  30. import { fetch } from 'hl:fetch'
  31. import { writeFile, mkDir, exists } from 'hl:fs'
  32. import { profilesDir, newId, textOr, listOr, sortDesc } from './util.hl'
  33. import { personsTable, showsTable, genresTable, showById, showYear, posterUrlOf, isPublicTitle, titlePath, typeWordOf, typeClassOf, liveShowOf } from './shows.hl'
  34. import { tmdbGet, tmdbImageBase } from './tmdb.hl'
  35. import { titleIdByTmdb, indexShow, slugOf, indexPerson } from './search.hl'
  36. import { slugBaseOf } from './search-helpers.hl'
  37. import { refreshCatalogShow, pagesOf, pageNumberOf, pageSize } from './catalog.hl'
  38. import { claimShortId } from './shortids.hl'
  39. // titles stored per step (real data: see STATUS.md ticket #16 for the per-step time)
  40. static fillBatch = 40
  41. // ---- the person -------------------------------------------------------------------------------------------------
  42. static slugIndex = { built = false ids = {} }
  43. static ensureSlugIndex = () => {
  44. if (!slugIndex.built) {
  45. m = {}
  46. for (p of personsTable.find(null, null)) { if (p.urlSegment != null) { m[p.urlSegment] = p.id } }
  47. slugIndex.ids = m
  48. slugIndex.built = true
  49. }
  50. return true
  51. }
  52. static personBySlug = (&slug) => {
  53. if (slug == null || hlTypeName(slug) != 'String' || slug == '') { return null }
  54. ensureSlugIndex()
  55. id = slugIndex.ids[slug]
  56. return id == null ? null : personsTable.fetch(id)
  57. }
  58. static isComplete = (&p) => { return p.filmographyAt != null }
  59. // the page asks TMDB for this person: not complete yet, and TMDB knows them
  60. static needsFill = (&p) => { return !isComplete(p) && p.tmdbId != null }
  61. // the photo: /profiles/<oldId>.<ext> (images.hl) — only when the file is there (the migrated `image` = 'jpg'
  62. // of 129 people never came with a file)
  63. static profileName = (&p) => {
  64. if (p.oldId == null || p.image == null || p.image == '') { return null }
  65. return p.oldId + '.' + p.image
  66. }
  67. static photoUrlOf = (&p) => {
  68. name = profileName(p)
  69. return name != null && exists(profilesDir + '/' + name) ? '/profiles/' + name : ''
  70. }
  71. static monthNames = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December']
  72. // '1986-03-04' → '4 March 1986'; a bare year or anything else as it is
  73. static dateText = (&d) => {
  74. t = textOr(d)
  75. if (t == null) { return null }
  76. if (t.length < 10) { return t }
  77. m = toNumber(t.slice(5, 7))
  78. day = toNumber(t.slice(8, 10))
  79. if (m == null || day == null || m < 1 || m > 12) { return t }
  80. return day + ' ' + monthNames[m - 1] + ' ' + t.slice(0, 4)
  81. }
  82. // { name, photoUrl, born, died, bio } — every field a plain string ('' = none)
  83. static personInfoOf = (&p) => {
  84. born = dateText(p.birthDate)
  85. place = textOr(p.birthPlace)
  86. let bornText = ''
  87. if (born != null && place != null) { bornText = 'Born ' + born + ' in ' + place } else if (born != null) { bornText = 'Born ' + born } else if (place != null) { bornText = 'Born in ' + place }
  88. died = dateText(p.deathDate)
  89. return { name = p.name != null ? p.name : '' photoUrl = photoUrlOf(p) born = bornText died = died != null ? 'Died ' + died : ''
  90. bio = textOr(p.biography) != null ? p.biography : '' }
  91. }
  92. // ---- the filmography --------------------------------------------------------------------------------------------
  93. // the credits, one row per title (a title twice — the migrated credit and TMDB's — is ONE row, roles merged), newest
  94. // first by the title's release ('YYYY-MM-DD', else its year), undated last; only public titles (no adult/unknown)
  95. static filmographyOf = (&p) => {
  96. byShow = {}
  97. order = []
  98. for (c of listOr(p.shows)) {
  99. if (c != null && c.show != null) {
  100. let role = textOr(c.character)
  101. let prev = byShow[c.show]
  102. if (prev == null) {
  103. byShow[c.show] = { role = role != null ? role : '' posterPath = textOr(c.posterPath) adult = c.adult == true }
  104. order.push(c.show)
  105. } else {
  106. let x = prev
  107. if (role != null && !(' / ' + x.role + ' / ').includes(' / ' + role + ' / ')) { x.role = x.role == '' ? role : x.role + ' / ' + role }
  108. if (x.posterPath == null) { x.posterPath = textOr(c.posterPath) }
  109. if (c.adult == true) { x.adult = true }
  110. byShow[c.show] = x
  111. }
  112. }
  113. }
  114. list = []
  115. listed = {}
  116. for (sid of order) {
  117. let c = byShow[sid]
  118. // tracker#31: a credit on a merged tombstone is the keeper's (merge.hl repoints them; a keeper listed once)
  119. let s = liveShowOf(showById(sid))
  120. if (isPublicTitle(s) && !c.adult && listed[s.id] != true) {
  121. listed[s.id] = true
  122. let y = showYear(s)
  123. let release = textOr(s.release)
  124. let key = release != null && release.length >= 4 ? release : (y != null ? y : '')
  125. let poster = posterUrlOf(s)
  126. if (poster == '/posters/none' && c.posterPath != null) { poster = tmdbImageBase + '/w92' + c.posterPath }
  127. list.push({ key = key id = s.id title = s.title year = y != null ? y : '' href = titlePath(s) posterUrl = poster
  128. role = c.role typeLabel = typeWordOf(s) typeClass = typeClassOf(s) })
  129. }
  130. }
  131. out = []
  132. undated = []
  133. for (r of sortDesc(list)) { if (r.key == '') { undated.push(r) } else { out.push(r) } }
  134. for (r of undated) { out.push(r) }
  135. return out
  136. }
  137. // ---- the fetch ------------------------------------------------------------------------------------------------
  138. // person id → { items, pos, credits, added, requests, started, steps } while a fetch is under way (in memory: a restart
  139. // loses it — the titles stored so far are found again by TMDB id, nothing is duplicated)
  140. static jobs = {}
  141. // TMDB genre id → ours (our genres carry TMDB's id)
  142. static genreMap = { built = false ids = {} }
  143. static genreIdsOfTmdb = (&tmdbIds) => {
  144. if (!genreMap.built) {
  145. m = {}
  146. for (g of genresTable.find(null, null)) { if (g.tmdbId != null) { m['' + g.tmdbId] = g.id } }
  147. genreMap.ids = m
  148. genreMap.built = true
  149. }
  150. out = []
  151. for (t of listOr(tmdbIds)) {
  152. let id = genreMap.ids['' + t]
  153. if (id != null && !out.includes(id)) { out.push(id) }
  154. }
  155. return out
  156. }
  157. // TMDB's combined_credits → one item per title (cast AND crew of the same title merged), TV + movies only
  158. static creditItems = (&cc) => {
  159. byKey = {}
  160. order = []
  161. all = []
  162. if (cc != null) {
  163. for (c of listOr(cc.cast)) { all.push({ credit = c role = textOr(c.character) }) }
  164. for (c of listOr(cc.crew)) { all.push({ credit = c role = textOr(c.job) }) }
  165. }
  166. for (a of all) {
  167. let c = a.credit
  168. let kind = c.media_type
  169. if ((kind == 'tv' || kind == 'movie') && c.id != null && hlTypeName(c.id) == 'Number') {
  170. let k = kind + ':' + c.id
  171. let known = byKey[k]
  172. if (known == null) {
  173. let name = kind == 'tv' ? textOr(c.name) : textOr(c.title)
  174. if (name != null) {
  175. byKey[k] = { kind = kind tmdbId = c.id title = name release = textOr(kind == 'tv' ? c.first_air_date : c.release_date)
  176. overview = textOr(c.overview) language = textOr(c.original_language) genreIds = listOr(c.genre_ids)
  177. posterPath = textOr(c.poster_path) adult = c.adult == true role = a.role != null ? a.role : '' }
  178. order.push(k)
  179. }
  180. } else if (a.role != null && !(' / ' + known.role + ' / ').includes(' / ' + a.role + ' / ')) {
  181. let x = known
  182. x.role = x.role == '' ? a.role : x.role + ' / ' + a.role
  183. byKey[k] = x
  184. }
  185. }
  186. }
  187. out = []
  188. for (k of order) { out.push(byKey[k]) }
  189. return out
  190. }
  191. // the photo: w185 → storage/mpackdb/profiles/<oldId>.<ext> — { requests, image, tmdbProfile, failed }
  192. static syncProfile = (&p, profilePath) => {
  193. out = { requests = 0 image = p.image tmdbProfile = p.tmdbProfile failed = null }
  194. if (textOr(profilePath) == null || p.oldId == null) { return out }
  195. parts = profilePath.split('.')
  196. ext = parts[parts.length - 1].toLowerCase()
  197. if (ext != 'jpg' && ext != 'jpeg' && ext != 'png' && ext != 'webp') { return out }
  198. file = profilesDir + '/' + p.oldId + '.' + ext
  199. if (exists(file) && p.tmdbProfile == profilePath && p.image == ext) { return out }
  200. out.requests = 1
  201. r = fetch(tmdbImageBase + '/w185' + profilePath, { headers = { 'user-agent' = 'tracker.worldapi.org (person page)' } timeoutMs = 20000 })
  202. if (r == null || r.status != 200 || r.body == null || r.body.length == 0) {
  203. out.failed = 'photo ' + profilePath + ': ' + (r == null ? 'no answer' : 'HTTP ' + r.status)
  204. return out
  205. }
  206. mkDir(profilesDir)
  207. writeFile(file, toBytes(r.body), 420)
  208. out.image = ext
  209. out.tmdbProfile = profilePath
  210. return out
  211. }
  212. // step 1: TMDB's person + credits; the facts (a TMDB value that is empty never overwrites ours) and the photo
  213. static startFill = (p) => {
  214. t0 = now()
  215. d = tmdbGet('/person/' + p.tmdbId + '?append_to_response=combined_credits')
  216. if (d.failed != null) { return { error = 'TMDB: ' + d.failed } }
  217. items = creditItems(d.combined_credits)
  218. photo = syncProfile(p, d.profile_path)
  219. x = p
  220. if (textOr(d.biography) != null) { x.biography = d.biography }
  221. if (textOr(d.birthday) != null) { x.birthDate = d.birthday }
  222. if (textOr(d.deathday) != null) { x.deathDate = d.deathday }
  223. if (textOr(d.place_of_birth) != null) { x.birthPlace = d.place_of_birth }
  224. if (textOr(x.imdbId) == null && textOr(d.imdb_id) != null) { x.imdbId = d.imdb_id }
  225. if (d.adult != null) { x.adult = d.adult == true }
  226. x.image = photo.image
  227. x.tmdbProfile = photo.tmdbProfile
  228. x.updatedAt = now()
  229. if (personsTable.update(x.id, x) == null) { return { error = 'could not store the person: ' + personsTable.lastError() } }
  230. personTouched(x.id)
  231. if (photo.failed != null) { console.log('person fill: ' + p.name + ': ' + photo.failed) }
  232. jobs[p.id] = { items = items pos = 0 credits = [] added = 0 requests = 1 + photo.requests started = t0 steps = 1 tmdbMs = now() - t0 }
  233. return { done = false pos = 0 total = items.length }
  234. }
  235. // a title we don't have: the minimal record (shape: search.hl importTitle) → its id, or null
  236. static addMinimalTitle = (&cr, &pid, pname) => {
  237. kindType = cr.kind == 'tv' ? 'series' : 'movie'
  238. rel = cr.release
  239. yr = rel != null && rel.length >= 4 ? toNumber(rel.slice(0, 4)) : null
  240. slug = slugOf(cr.title, cr.tmdbId)
  241. record = {
  242. id = newId() title = cr.title summary = null tmdbSummary = cr.overview tmdbId = cr.tmdbId
  243. language = cr.language type = kindType release = rel year = yr image = null
  244. urlSegment = slug episodesCount = null homepage = null imdbId = null seasonsCount = null
  245. status = null tagline = null tvdbId = null tvmzId = null genres = genreIdsOfTmdb(cr.genreIds)
  246. cast = [{ person = pid character = cr.role actor = pname }] seasons = [] oldId = 'tmdb-' + cr.kind + '-' + cr.tmdbId
  247. adult = cr.adult imported = now() minimal = true
  248. }
  249. // mission 060: its short id (shortids.hl — unique across shows and persons)
  250. record.shortId = claimShortId('show', record.id, record.oldId)
  251. if (showsTable.put(record) == null) {
  252. console.log('person fill: "' + cr.title + '" not stored: ' + showsTable.lastError())
  253. return null
  254. }
  255. indexShow(record)
  256. refreshCatalogShow(record.id)
  257. return record.id
  258. }
  259. // every next step: the next `fillBatch` titles; the last one links the credits and marks the person complete
  260. static fillStep = (p, jobIn) => {
  261. job = jobIn
  262. total = job.items.length
  263. let end = job.pos + fillBatch
  264. if (end > total) { end = total }
  265. credits = job.credits
  266. let added = job.added
  267. pid = p.id
  268. pname = p.name
  269. let i = job.pos
  270. while (i < end) {
  271. let item = job.items[i]
  272. let sid = titleIdByTmdb(item.kind == 'tv' ? 'series' : 'movie', item.tmdbId)
  273. if (sid == null) {
  274. sid = addMinimalTitle(item, pid, pname)
  275. if (sid != null) { added = added + 1 }
  276. }
  277. if (sid != null) { credits.push({ show = sid character = item.role posterPath = item.posterPath adult = item.adult }) }
  278. i = i + 1
  279. }
  280. job.pos = end
  281. job.credits = credits
  282. job.added = added
  283. job.steps = job.steps + 1
  284. jobs[pid] = job
  285. if (end < total) { return { done = false pos = end total = total } }
  286. // done: the credits (+ the migrated ones TMDB no longer lists), complete
  287. have = {}
  288. all = []
  289. for (c of credits) { have[c.show] = true all.push(c) }
  290. for (c of listOr(p.shows)) { if (c != null && c.show != null && have[c.show] != true) { all.push(c) } }
  291. x = p
  292. x.shows = all
  293. x.filmographyAt = now()
  294. x.updatedAt = x.filmographyAt
  295. if (personsTable.update(x.id, x) == null) { return { error = 'could not store the filmography: ' + personsTable.lastError() } }
  296. personTouched(x.id)
  297. personsTable.persist()
  298. showsTable.persist()
  299. jobs[pid] = null
  300. console.log('person fill: ' + pname + ' (tmdb ' + p.tmdbId + ') credits=' + total + ' added=' + added + ' requests=' + job.requests + ' steps=' + job.steps + ' tmdbMs=' + job.tmdbMs + ' ms=' + (now() - job.started))
  301. return { done = true pos = total total = total added = added }
  302. }
  303. // one step for the person with this id: { done, pos, total } or { error }
  304. static personFillStep = (&personId) => {
  305. p = personsTable.fetch(personId)
  306. if (p == null) { return { error = 'no such person' } }
  307. if (isComplete(p)) { return { done = true } }
  308. if (p.tmdbId == null) { return { error = 'TMDB does not know this person' } }
  309. job = jobs[personId]
  310. if (job == null) { return startFill(p) }
  311. return fillStep(p, job)
  312. }
  313. // ---- people from a title's cast (tracker#26, details.hl completeStep) ------------------------------------------
  314. // TMDB person id → our person id, built once (15k people, one scan), kept up to date by castPersonId
  315. // (flat statics, written key by key like search.hl's haveTmdb: `let m = …ids` would copy the whole map — plain data
  316. // deep-copies on assignment)
  317. static personTmdbBuilt = { yes = false }
  318. static personTmdbIds = {}
  319. static personIdByTmdb = (&tmdbId) => {
  320. if (!personTmdbBuilt.yes) {
  321. for (p of personsTable.find(null, null)) { if (p.tmdbId != null) { personTmdbIds['' + p.tmdbId] = p.id } }
  322. personTmdbBuilt.yes = true
  323. }
  324. return personTmdbIds['' + tmdbId]
  325. }
  326. // a free person slug: "Maya-Rudolph", taken → "-2", … (the migrated people's form)
  327. static personSlugOf = (&name, &tmdbId) => {
  328. ensureSlugIndex()
  329. base = slugBaseOf(name, 'person-' + tmdbId)
  330. if (slugIndex.ids[base] == null) { return base }
  331. let n = 2
  332. while (slugIndex.ids[base + '-' + n] != null) { n = n + 1 }
  333. return base + '-' + n
  334. }
  335. // how many people castPersonId added since the start (details.hl counts a title's new people by the difference, tracker#28)
  336. static castPeopleAdded = { n = 0 }
  337. static castPeopleCount = () => { return castPeopleAdded.n }
  338. // our person for one cast entry of TMDB ({ tmdbId, name, character, gender, profile }) — the one we have by TMDB id, else a NEW
  339. // person (name, tmdbId, a free slug, this one credit, TMDB's profile path as `tmdbProfile` (tracker#37: the photo of /people and
  340. // the search); not complete: their page fetches the rest on the first visit like any migrated person). Answers the id, or null
  341. // when it could not be stored. Also a CREW person (tracker#28: `character` = the job, "Director").
  342. static castPersonId = (&c, &showId) => {
  343. have = personIdByTmdb(c.tmdbId)
  344. if (have != null) { return have }
  345. record = { id = newId() name = c.name tmdbId = c.tmdbId urlSegment = personSlugOf(c.name, c.tmdbId) oldId = 'tmdb-person-' + c.tmdbId
  346. gender = c.gender image = null tmdbProfile = textOr(c.profile) adult = false birthDate = null birthPlace = null deathDate = null imdbId = null
  347. shows = showId != null ? [{ show = showId character = c.character }] : [] added = now() }
  348. // mission 062 (tracker#27 for #26/#28): every person the app creates gets its short id at once (shortids.hl)
  349. record.shortId = claimShortId('person', record.id, record.oldId)
  350. if (personsTable.put(record) == null) {
  351. console.log('details: person "' + c.name + '" not stored: ' + personsTable.lastError())
  352. return null
  353. }
  354. personTmdbIds['' + c.tmdbId] = record.id
  355. slugIndex.ids[record.urlSegment] = record.id
  356. indexPerson(record)
  357. personTouched(record.id)
  358. castPeopleAdded.n = castPeopleAdded.n + 1
  359. return record.id
  360. }
  361. // ---- GUEST STARS (mission 062): a series' guests are NOT person records — details.hl stores them on the title ({ tmdbId, actor,
  362. // character, profile, episodes, guest }, `person` only when we have them already). 160k people (most of them one-episode guests)
  363. // cost 3.3 GB of RSS at boot (w061). A guest's link is `/person/tmdb/<tmdbId>?show=<show id>` (project.hl guestRoute): the FIRST
  364. // open creates the person (name + role from that title's guest entry; without one — an old link — TMDB's /person/<id>), with its
  365. // short id, then the person page fills it like any other.
  366. static guestOf = (&showId, &tmdbId) => {
  367. s = showId != null && hlTypeName(showId) == 'String' && showId != '' ? showsTable.fetch(showId) : null
  368. if (s == null) { return null }
  369. for (c of listOr(s.cast)) { if (c != null && c.tmdbId == tmdbId && textOr(c.actor) != null) { return c } }
  370. return null
  371. }
  372. // our person for this TMDB person (created now when we have none) → { id, created } or { error }
  373. static guestPersonId = (tmdbId, showId) => {
  374. have = personIdByTmdb(tmdbId)
  375. if (have != null) { return { id = have created = false } }
  376. g = guestOf(showId, tmdbId)
  377. let c = null
  378. let inShow = null
  379. if (g != null) {
  380. c = { tmdbId = tmdbId name = g.actor character = g.character gender = null }
  381. inShow = showId
  382. } else {
  383. d = tmdbGet('/person/' + tmdbId)
  384. if (d.failed != null) { return { error = 'TMDB: ' + d.failed } }
  385. if (textOr(d.name) == null || d.adult == true) { return { error = 'no such person' } }
  386. c = { tmdbId = tmdbId name = d.name character = null gender = d.gender }
  387. }
  388. pid = castPersonId(c, inShow)
  389. if (pid == null) { return { error = 'could not store the person' } }
  390. personsTable.persist()
  391. console.log('guest opened: ' + c.name + ' (tmdb ' + tmdbId + ') → new person ' + pid + (inShow != null ? ' from show ' + inShow : ' from TMDB'))
  392. return { id = pid created = true }
  393. }
  394. // ---- /people (tracker#37, mission 032): everyone we have, the one updated last first ------------------------------------------
  395. // Creator: "/people just lists persons by last one updated with poster name and pagination". LAST UPDATED = the newest of
  396. // `updatedAt` (stamped by the person page's fetch, startFill / fillStep), `filmographyAt` and `added` (a person the app created);
  397. // the migrated people with none of them come last (newest stored first). No adult filter (tracker#32's /people had none either).
  398. // A CACHED VIEW like tracker#30's list caches (mission 025): the order is built ONCE at boot (project.hl warmPeopleView: one pass
  399. // over the people + one sort, and the cast photos below — 105k people, 3–4 s on the live copy), a page only slices its 24 ids and
  400. // fetches those people (~15 ms server time, any page). Kept current without a rebuild: every person the app creates or updates is
  401. // `personTouched` → `recent`; the next page read moves them to the front (one pass over the ids, ~50 ms, only after a change).
  402. static peopleView = { built = false ids = [] recent = [] castProfiles = 0 }
  403. static updatedKeyOf = (&p) => {
  404. let k = p.updatedAt != null ? p.updatedAt : 0
  405. if (p.filmographyAt != null && p.filmographyAt > k) { k = p.filmographyAt }
  406. if (p.added != null && p.added > k) { k = p.added }
  407. return k
  408. }
  409. static warmPeopleView = () => {
  410. if (peopleView.built) { return peopleView.ids.length }
  411. peopleView.castProfiles = castProfilesOf()
  412. stored = []
  413. for (p of personsTable.iterate(null, null)) { stored.push({ id = p.id key = updatedKeyOf(p) }) }
  414. // newest stored first: the table is in the order the people were stored, so the list is nearly sorted already (util.hl sortDesc
  415. // keeps runs) and equal keys (the migrated people: 0) end newest stored first
  416. newestFirst = []
  417. let i = stored.length - 1
  418. while (i >= 0) { newestFirst.push(stored[i]) i = i - 1 }
  419. ids = []
  420. for (x of sortDesc(newestFirst)) { ids.push(x.id) }
  421. peopleView.ids = ids
  422. peopleView.built = true
  423. return ids.length
  424. }
  425. // a person was created or updated: first in /people from the next page read on
  426. static personTouched = (&id) => {
  427. if (peopleView.built && id != null) { peopleView.recent.push(id) }
  428. return null
  429. }
  430. static peopleViewCurrent = () => {
  431. if (!peopleView.built) { warmPeopleView() }
  432. if (peopleView.recent.length == 0) { return true }
  433. seen = {}
  434. out = []
  435. let i = peopleView.recent.length - 1
  436. while (i >= 0) {
  437. let id = peopleView.recent[i]
  438. if (seen[id] != true) { seen[id] = true out.push(id) }
  439. i = i - 1
  440. }
  441. for (id of peopleView.ids) { if (seen[id] != true) { out.push(id) } }
  442. peopleView.ids = out
  443. peopleView.recent = []
  444. return true
  445. }
  446. static peopleViewStats = () => { return { people = peopleView.ids.length castProfiles = peopleView.castProfiles } }
  447. static peopleCount = () => {
  448. peopleViewCurrent()
  449. return peopleView.ids.length
  450. }
  451. // a person's photo for a tile or a search row: our file (photoUrlOf), else TMDB's (`tmdbProfile`, w185 — like the filmography's
  452. // TMDB posters), else TMDB's from a title's cast entry (castProfiles), else the placeholder
  453. static personPhotoOf = (&p) => {
  454. local = photoUrlOf(p)
  455. if (local != '') { return local }
  456. path = textOr(p.tmdbProfile) != null ? p.tmdbProfile : castProfiles[p.id]
  457. return path != null ? tmdbImageBase + '/w185' + path : '/profiles/none'
  458. }
  459. // person id → TMDB's profile path from a title's cast or crew entry (credits.hl stores `profile` there: cast since tracker#28, crew
  460. // since tracker#37) — for the people the credits job created before castPersonId kept it as `tmdbProfile` (live copy 2026-10-04:
  461. // 51,922 of 105,018 people; the crew entries had none yet). Built at boot with the view (one pass over the titles, ~1–3 s). IN
  462. // MEMORY, not written to the people: 51,907 record updates took 511 s at a start (mission 032, measured on the live copy).
  463. // (Flat static written key by key, like the maps above.)
  464. static castProfiles = {}
  465. static castProfilesOf = () => {
  466. let n = 0
  467. // iterate, not find: one title decoded at a time
  468. for (s of showsTable.iterate(null, null)) {
  469. for (c of listOr(s.cast)) {
  470. if (c != null && c.person != null && textOr(c.profile) != null && castProfiles[c.person] == null) {
  471. castProfiles[c.person] = c.profile
  472. n = n + 1
  473. }
  474. }
  475. for (c of listOr(s.crew)) {
  476. if (c != null && c.person != null && textOr(c.profile) != null && castProfiles[c.person] == null) {
  477. castProfiles[c.person] = c.profile
  478. n = n + 1
  479. }
  480. }
  481. }
  482. return n
  483. }
  484. // one page: { page, pages, tiles } — a tile like the title grids' (components/tilelist.hl): photo, name, no year
  485. static peoplePage = (&raw) => {
  486. peopleViewCurrent()
  487. pages = pagesOf(peopleView.ids.length)
  488. page = pageNumberOf(raw, pages)
  489. tiles = []
  490. for (id of peopleView.ids.slice((page - 1) * pageSize, page * pageSize)) {
  491. let p = personsTable.fetch(id)
  492. if (p != null && p.urlSegment != null) {
  493. tiles.push({ href = '/person/' + p.urlSegment posterUrl = personPhotoOf(p) title = p.name != null ? '' + p.name : '' year = '' })
  494. }
  495. }
  496. return { page = page pages = pages tiles = tiles }
  497. }

Branches

Latest commits

  • 9bce1f65tracker mission 032: STATUS gate files + the hour-boundary flakemre
  • 718bfb89tracker#37 (mission 032): /people = everyone, last updated first (updatedAt stamped by the person fill; view built at boot, touched people first at once), photo + name tiles (person colour) with the /movies pagination, /people/<letter> removed; photo = our file, tmdbProfile, a cast/crew entry's profile (in-memory map at boot), else the new 'no photo' placeholder; new cast/crew/created_by people keep tmdbProfile; search people rows with the photo; /settings = the heading only; util.hl sortDesc starts from sorted runs (same result, 105k: 1.6 s -> 0.15 s); gates 369/0, 32/0, 52/0, check-theme 0; README/STATUS/LOGmre
  • 96ba683adeploy.sh: a gate without a 'passed,' line (check-theme) no longer ends the scriptmre
  • eb3b9205tracker: report 031mre
  • 9b5d2e89tracker mission 031: README (What it does, Test: four gates + the #32 checks, Files: theme/, new pages), STATUS (real copy, A/B load, how to repeat, open points), LOGmre
  • 39950e4ctracker#32 (mission 031): the WorldAPI theme (theme/ vendored verbatim from layouts.worldapi.org 85b5654; styles.hl inherits it: accent green-dark, type colours 1-6; own base/header rules, row lines, genre-pill and inverted-button frames removed, the season foldable keeps its line; check-theme 21 -> 0, 4th deploy gate; main actions class primary) and the #32 header (theme AppHeader/MainMenu/UserMenu/Sidebar/ContentFirst: desktop brand, search, Series|Shows|Movies|Genres|People, user icon with Unwatched..Settings, Logout; signed out the ident selector, phone the iD icon dropdown; phone menu in the sidebar overlay; marked entry by :has); /find -> /search/<q>, /genres, /people(/<letter>), /settings; main { ContentFirst { slot } } works around the hl:web one-line slot bug; gates 365/0, 32/0, 52/0, check-theme 0mre
  • a386dc92tracker: reports 029 + 030mre
  • 71e0fd7dtracker missions 029 + 030: README (What it does, Files, gate count), STATUS (real-copy numbers, how to repeat, open points), LOGmre
  • d36ea6eatracker#34 + #35 (mission 030): Follow directly under the poster, as wide as the poster (show.hl, styles.hl); the status pill next to a series' title — TVmaze's status (new tvmazeStatus, stored by the sync's TVmaze merge) else TMDB's, TVmaze Ended + TMDB Canceled = Canceled, inverted (filled, dark text, no border), green running / yellow pending / red canceled / muted ended (shows.hl statusOf); the daily delta asks TVmaze's status of an unfollowed series TVmaze's change list names (dailysync.hl syncRunStep, sync.hl syncTvmazeStatus); the status backfill after the details repair (backfill.hl, jobs.hl statusTick; resumable, 550 ms per TVmaze request); gates 354/0, 32/0, 52/0mre
  • 7d7d4487tracker#33 (mission 029): reduced titles — every title TMDB's details never went through this app (no detailsAt, no tmdbSync) is incomplete (shows.hl isIncomplete; the old tracker's migrated rows passed #26's test: 5,697 non-adult on the live copy, 691 series without seasons); the repair job does the visibly reduced first (shows.hl missingParts), the page completes one on open; a title TMDB has no poster for (The Remaining) shows the placeholder; tools/count-incomplete.hl; gate fixtures stand for synced titles (tmdbSync), tests/seed-reduced.hl + #33 checks; gates 347/0, 32/0, 52/0mre
  • 661c2592tracker: report 028mre
  • 27c916fatracker mission 028: README ("Code order", the new file map), STATUS (counts before/after, tests, how to repeat, open), LOGmre
  • d924f398tracker mission 028: comments name the new files (sync.hl, dailysync.hl, backfill.hl, credits.hl, jobs.hl, images.hl …); tools/ref-params.py + tools/lambda-audit.py also scan lib/ (they globbed the root only), lambda-audit counts a plain `x = p` alias like `let x = p`mre
  • 2e89b968tracker mission 028 (code order) 5/5 let: `let` only where a variable is reassigned — 667 never-reassigned lets became plain declarations (project.hl, lib/, components/, tools/, tests/); kept: 264 in loop bodies (a plain declaration there is 'Cannot reassign' on the 2nd pass), 234 reassigned, 27 whose name is also a member/outer/free name (a plain write would rebind it); tools/let-audit.py decides and fixes (README 'Code order'); tests/realdata-m028.{sh,mjs} = the page-output diff on a real copy; gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 54796ff2tracker mission 028 (code order) 4/5 thin faces + last copies: the show page's check/follow faces call lib/watches.hl toggleWatched / toggleSeasonWatched (seasonAllWatched moved there) and lib/follows.hl toggleFollowed; both logins (header selector face, /login/callback) share lib/users.hl userOfCode; todayStr/listOf copies in components and the export readers copied into tools/migrate.hl + tools/old-short-ids.hl now once (lib/util.hl, lib/export.hl); gates 342/0, 32/0, 52/0; old-short-ids output byte-identical, migrate output identicalmre
  • 06b078e3tracker mission 028 (code order) 3/5 project.hl is the map: config, routes, wiring and a feature → file index (914 → 258 lines); the background jobs (daily sync run, backfills, details repair, credits job, merge, short ids, collection seed) moved unchanged into lib/jobs.hl (a class: their state is reassigned every step, a static cannot be; one instance made after the server), the login callback into lib/users.hl, poster/photo serving into lib/images.hl, the /shows/<slug> rule into lib/shows.hl showsMovedPath; route handlers are thin wrappers; gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 94716fd2tracker mission 028 (code order) 2/5 util + topics: lib/util.hl holds envOr, storageDir, postersDir, profilesDir, newId, hexDigits, todayStr, dateOr, textOr, hasId, listOr, firstOf, sortDesc once (were copied into up to 5 files); tmdbsync.hl split into tmdb.hl (TMDB/TVmaze requests), sync.hl (one title's sync), sync-helpers.hl, backfill.hl; details.hl split into details.hl, credits.hl, credits-helpers.hl (isIncomplete to shows.hl); search-helpers.hl (words, query, ranking, slugs); collections.hl (the TMDB collection seed, out of franchises.hl); deltasync.hl renamed dailysync.hl; no behaviour change: gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 186079b0tracker mission 028 (code order) 1/5 move: every root .hl except project.hl into lib/ (styles.hl into components/), import paths only; gates 342/0, 32/0, 52/0; real-copy pages identicalmre
  • 4f47f181tracker: report 027mre
  • dc1d4be4tracker mission 027: Hybriel master 06617221 vendored (plugin allocator fixes 3a781359 + 413f60e4); real copy RSS through first-start jobs + 400 loads flat ~2.55 GB (190aa11d 2.3 -> 5.6 GB), page times <= 1.1x; gates 342/0, 32/0, 52/0mre