tracker

5.4 KB

1. # w062: tracker, t26 (#26 + #28) merged into main + 3 fixes (mission 018 (old 062))
3. **Commit**: `f2b0067` on main (a merge commit: fa1f9df + t26 10bb3f9), in loreana:/media/STORAGE/projects/tracker.worldapi.org. Not pushed and not deployed.
4. - The worktree tracker-wt26 and branch t26 are removed. It had nothing uncommitted, only `.scratch` gate files.
5. - Note: the worktree tracker-wt18 (branch t18) is at 10bb3f9, so it is based on t26.
7. ## What was done
8. 1. **Merge**: conflicts were in README, STATUS, `components/show.hl` (header comment; skeleton poster + `#shortid`) and `tests/browser.mjs` (startServer env, import checks, the #27 and #26/#28 blocks). I kept both sides. `buildRows` and `syncMovie`/`syncShowPart` merged without conflict.
9. 2. **Short ids for every new record**:
10. - Shows were already covered (import, filmography).
11. - `people.hl castPersonId` now calls `claimShortId('person', …)`. It is the only place that creates people, so it covers completion, the credits job, the daily sync, `tools/sync-tmdb.hl`, the search import and the guest route.
12. - `shortids.hl` finds `../data/old-short-ids.json` when it runs from `tools/`.
13. - Gate:
14. - `count-short-ids` runs right after the sync tool and after the credits job, both before the next start: no record without a short id, 0 duplicates.
15. - The #26 person, the import's 2 people and the guest have their short id at once.
16. - The next start logs "nothing to do".
17. 3. **Guests are no person records**:
18. - `details.hl mergedCast` stores a guest on the title as `{ person: null|existing, tmdbId, actor, character, profile, episodes, guest }`.
19. - The link is `/person/tmdb/<id>?show=<show id>`. `project.hl guestRoute` calls `people.hl guestPersonId`, which creates the person (short id, persisted) and answers 302 to `/person/<slug>`. That page fills from TMDB.
20. - Without `?show=`, the name comes from TMDB `/person/<id>`. An unknown id answers 404.
21. - Main cast and crew still get records.
22. 4. **Watch/follow clicks**: the real payload was `showRow`, the whole show record (300 cast + crew entries). It went out with the page and again with every session sync. Now:
23. - `showRow` is a small `pageShowOf` object.
24. - Cast and crew derive from the slug. An adult title shows none; adult titles never get credits.
25. - `watchesOfUser` is cached per user until `setWatched`. It was a full scan of 11.7k watches, 3× per click.
26. - The episode/season faces return rows only after the page opened or closed a season (`collapsedChanged`). Otherwise the session sync's rows are the answer.
28. ## Gate
29. `TRACKER_GATE_SHOTS=/tmp/w062-shots TRACKER_GATE_PORT=8720 TRACKER_GATE_IDENT_PORT=8721 TRACKER_GATE_CHROME=8722-8726 TRACKER_GATE_TMDB_PORT=8727 node tests/browser.mjs`
30. → **`311 passed, 0 failed`**
31. - That is 266 from main, 35 from t26 and 10 new. The sync-tool expectation changed from +27 to +25 people.
32. - I looked at the screenshots `/tmp/w062-shots/guest-person-{390,1280}`, `credits-series-all-390`.
34. ## Real data
35. Copy of Byrodin `storage/`, real TMDB, server on :8730. The copy is deleted and nothing of mine is running.
37. **Jobs**
38. - Details repair: 557/557 in 836 s.
39. - Credits job: 5,867 titles, 0 failed.
40. - The first process ended silently at ~22:41 after 1,961 titles. I found no cause and no OOM line visible.
41. - The restart resumed the job and finished it (3,906 titles, 2,028 s).
42. - The next start found nothing to do. 112,298 records, all with a short id.
44. **People and RSS after boot**
46. | | people | RSS |
47. |---|---|---|
48. | main fa1f9df, same copy | 15,157 | 4.3–4.5 GB |
49. | merged, after the jobs, jobs off | 102,424 | 6.4–7.0 GB |
50. | merged, after the jobs, jobs on (nothing to do) | 102,287 | 8.1–8.8 GB |
51. | w061 (all guests as people) | 160,084 | — |
53. - I did not reproduce w061's "4.9 GB with 160k people". Today's main alone already boots at 4.3 GB.
54. - **RSS curve during the jobs** (sampled every minute, `/tmp/w062/rss.log` on Loreana): 1.2–6.3 GB, swinging. Twice it dropped by GBs around a 5 s pause.
56. **Pages and clicks**
57. - Picard: 202 cast (94 of them lazy guests) + 45 crew. Full load 201–342 ms; the first after boot took 2.3 s.
58. - Superman (movie): 117–173 ms.
59. - Guest click (Stephanie Czajkowski): created (short id `401md`) and filled from TMDB (18 films) in 0.74–1.3 s. Afterwards 29–66 ms.
60. - **Watch click** (page clock, click → answer):
61. - Picard: 159–174 ms (was 184–200).
62. - The Simpsons: 156–214 ms (was 276–731). Payload 15 kB, was 29 kB.
63. - Follow: 119–221 ms.
64. - About 1 click in 6 took 550–970 ms (the runtime pauses).
65. - Screenshots on Loreana, looked at: `/tmp/w062/real/{Star-Trek-Picard,Superman}{,-credits}-{390,1280}.png`, `guest-person-{390,1280}.png`.
67. ## Docs
68. - README: new section "What it does (mission 018 (old 062))", the gate notes, a deploy bullet, the Data section, the Files table.
69. - STATUS: a new top entry (deploy timing, what to watch, how to repeat).
71. ## Open points
72. 1. Deploy. On the first start the repair takes ~14 min, then the credits job ~50 min. Watch `docker stats`: expect about 2× RSS at boot.
73. 2. There are still 102k people (6.8× before), mostly movie casts. If that is too much memory, the next step is to make a movie's cast after the first N lazy too (not built: the mission said to keep the main cast).
74. 3. Hybriel issues for the architect:
75. - RSS swings by GBs.
76. - Steps of about 5 s that block pages.
77. - The silent process end in run 1.
78. - About 120 ms per session face inside hl:web's re-mount and sync. This is the floor for watch clicks.
79. 4. Adult titles no longer show a migrated cast.