gitoriaLog in with ident

tracker

All repositories: gitoria

ReadmeCodePull requestsReleasesTicketsSettings
Commit84e1b3e184e1b3e1tracker: reports 025 + 026mre84e1b3e1/reports/012-tracker-final-merge.md

5.0 KB

  1. # w056: tracker, #16 merged, 3 fixes, Hybriel master (mission 012 (old 056))
  2. The work is in loreana:/media/STORAGE/projects/tracker.worldapi.org, on main. Nothing was pushed and nothing was deployed.
  3. **Commits**
  4. - `2c53d5e`: merges t16-person into main.
  5. - All conflicts only added code on both sides, so both sides were kept.
  6. - The filmography now lists only public titles (`isPublicTitle`, which means `adult == false`) whose TMDB credit is not adult either.
  7. - The worktree tracker-wt16 and the branch are removed. Their `.scratch/w055` notes were moved to `.scratch/w055-wt16`.
  8. - `2b7fdd6`: the header fix and the backfill fix.
  9. - Signed out on a phone, the header is now one row. The buttons do not wrap their text, and the phone shows "Log in" ("Log in with ident" from 40rem up). It is 46 px high at 320 and 390 px (it was 73 px).
  10. - The backfill no longer downloads the posters of adult titles. Their flag and ids are still stored.
  11. - `f40c250`: the new Hybriel and the hidden adult show page.
  12. - Hybriel master is re-vendored.
  13. - The page of an adult title says "Not found" to everyone who does not follow it.
  14. - Two flaky gate checks are fixed.
  15. - `f2fe3e3`: README, STATUS, `tests/realdata-056.mjs` and `tools/check-public-slugs.hl`.
  16. **Gate**
  17. `TRACKER_GATE_SHOTS=/tmp/w056/shots TRACKER_GATE_PORT=8720 TRACKER_GATE_IDENT_PORT=8721 TRACKER_GATE_CHROME=8722-8726 TRACKER_GATE_TMDB_PORT=8727 node tests/browser.mjs`
  18. - Last line: **`208 passed, 0 failed`**. That is 203 after the merge plus 5 new checks.
  19. - I made two gate fixes:
  20. - The backfill's "start" line is written at the first clock tick, which comes after the server's first answer. The check now waits for that line; without the wait it failed every time after the merge.
  21. - The tab now goes to about:blank before each server stop. Without that, websocket reconnect errors sometimes failed the console check.
  22. - I proved two of the new checks catch the bug they are for:
  23. - With the old poster download put back, the poster check fails.
  24. - With the old Hybriel binary, the "Unfollow hides the page" check fails.
  25. **Hybriel**
  26. - Master **8590df63** (it includes 73267707 / #122 and #121 426944c7). Built read-only, ReleaseFast, sha256 `ca830ef8…`.
  27. - The old copy is in `.scratch/pre-056/`.
  28. - Duplicate @ids in the live copy: 0 in all 8 tables.
  29. - No workaround was removed. The SVG check was already in the gate.
  30. **Hiding the adult show page: it works now**
  31. - Why mission 010 (old 054)'s attempt failed: `collapsed` (which seasons are open, kept in the browser) was derived from the session-dependent `found`.
  32. - After every session face, the server re-derived it from the page's starting state. The seasons snapped back and the second click did nothing.
  33. - Now `collapsed` comes from the slug only. `showRow` itself is null when the page is hidden, because every member is sent to the browser.
  34. - What a user sees:
  35. - A follower sees the page.
  36. - If the follower unfollows on the page, it changes to "Not found" at once.
  37. - No minimal repro for Anton was needed.
  38. **Real data** (copy of the live `storage/`, one tar, real TMDB, the backfill run on the copy; the copy was deleted afterwards)
  39. | | |
  40. |---|---|
  41. | Backfill | 78 min: adult 3,450, not adult 5,866, unknown 137 |
  42. | Posters | **5,433 downloaded (was 8,740), 269 MB (was 459 MB)** |
  43. Full page loads in ms, at 390 / 1280 px:
  44. | page | 390 | 1280 |
  45. |---|---|---|
  46. | `/` | 426 | 91 |
  47. | `/movies` | 92 | 90 |
  48. | `/movies/page/2` | 93 | 37 |
  49. | `/shows` | 287 | 41 |
  50. | `/shows/page/2` | 92 | 42 |
  51. | `/search/the` | 336 | 126 |
  52. | `/search/sex` | 442 | 91 |
  53. | `/person/Jared-Harris` | 242 | 102 |
  54. | `/shows/Foundation` | 189 | 145 |
  55. | `/my/shows` | 650 | 219 |
  56. | `/my/movies` | 41 | 38 |
  57. | `/my/unwatched` | 1478 | 1655 |
  58. | `/my/schedule` | 457 | 322 |
  59. - The first visit of the person page loaded the filmography from TMDB: 97 credits, 80 titles added, 487 ms.
  60. - `/my/unwatched` took 1.5–2.4 s over repeated loads (the server part is 1.33 s, 1,595 rows). The old Hybriel gives the same times.
  61. - The header is one row on every page.
  62. - The adult page (`/shows/This-Aint-Star-Trek-XXX`) says "Not found" signed out, and its slug is not in the HTML. It also says "Not found" for az5b2, who follows no adult title.
  63. - I collected every link on all 251 list pages, `/`, 6 searches and the person page: 5,914 slugs. All of them are `adult == false`; 0 adult, 0 unknown.
  64. - 0 console problems.
  65. - Screenshots on Loreana: `/tmp/w056/real/{out,in}-*-{390,1280}.png` and the gate's `header-signed-out-{320,390,1280}.png`. I looked at them.
  66. **Open points**
  67. 1. Deploy: the backfill runs on the live data for about 80 min and adds about 270 MB of posters.
  68. 2. `/my/unwatched` is at the 2 s target (1.5–2.4 s).
  69. 3. hl:fetch blocks the whole server while a TMDB answer is slow (the slowest step in this run took 5 s).
  70. 4. When someone unfollows an adult title on its page, the page turns into "Not found" at once. That follows from the rule; tell me if a softer behaviour is wanted.
  71. 5. Still open from before: 137 TMDB-404 titles stay hidden, adult people are not hidden, and people are never refreshed.

Branches

Latest commits

  • 84e1b3e1tracker: reports 025 + 026mre
  • dc40d859tracker#31 (mission 026): duplicate titles merged — the 68 type+tmdbId pairs held by 157 records were the old tracker's (all migrated); merge.hl repair job (own clock, before the TMDB jobs) keeps one keeper per title (follows/watches > old short id > oldest), moves follows, watches, seasons, cast, credits, timelines, tombstones the rest (mergedInto, never deleted), slugs + short ids 301 to the keeper; stray seasons merged into their listed twin (Reacher S3 watches) or linked when watched; search import re-checks before its put; deploy.sh waits up to 90 s for 200; real copy 68 -> 0 dup ids, az5b2 follows/watches equal; gates 342/0, 32/0, 52/0mre
  • 2667da05tracker#30 (mission 025): /my/ pages from slim cached title cards, episode rows and watch sets (after the jobs /my/series 1.8 s -> 0.06 s, /my/unwatched 4.1 -> 0.18 s); timeline page shows its name once; franchise widget under the poster/title; movies with TV leftovers (First Contact) go through the details repair; tools/count-tmdb-ids.hl; gates 327/0, 52/0, 32/0mre
  • 3909810dantcolony#40: mission references in README/STATUS/docs point to the moved missionsmre
  • e63b1d28antcolony#40: history (LOG.md), worker briefs (missions/) and reports moved here from antcolony, numbered per project; old numbers in antcolony docs/mission-map.mdmre
  • 1cda451dtracker: Hybriel master 190aa11d (#127 both shapes, GC correctness fc838894) — conductor adopts despite /my/series 2.4x after jobs (memory 8.0 → 1.7 GB boot); see reports/071mre
  • c1fa2f2etracker (mission 071): Hybriel master 190aa11d measured on the real copy vs the live binary 8590df63 — NOT adopted (after the first-start jobs /my/series 2.4x slower, /series 1.6x, RSS swings 7.4-12.2 GB; fresh it is flat at 1.7-2.2 GB and /my/unwatched faster), vendor stays 8efba065, candidate kept in .scratch/w071/vendor-190aa11d; tests/kinds.mjs: collection seed off (its TMDB request broke check 1 in 1 of 4 runs); tests/realdata-071.sh + realdata-071-bench.mjs + tools/realdata-071-table.py; README + STATUS (numbers, how to repeat); gates 325/0, 32/0, 50/0mre
  • 8081350atracker docs (mission 070): README (summary, Config HL_GC_BYTES — kept at Hybriel's default, the 256 MiB setting is taken out of docker-compose.yml again: the jobs grew to 12+ GB with it too, see STATUS), Test (three gates), Deploy (first start ~50 min: kinds then seed, restart once after collections done, memory numbers), Vendored Hybriel 8efba065 + #48 audit, Files; STATUS mission 070 entry (merges, migrated counts, lambda audit, gates, RSS old vs new, how to repeat, open points); docs/kinds.md + docs/franchises.md job order; tests/realdata-070-*.sh, tools/count-migrated.hl, tools/ref-params.py, tools/lambda-audit.pymre
  • 1ad19c8ctracker: re-vendor Hybriel master 8efba065 (#126 GC by bytes, #48 lambda parameters copy) (mission 070): bin/hybriel sha256 50361e95…, plugins core crypto data fetch fs http http1 mpackdb proc smtp time web; lambda audit: 13 lambdas change a passed record/list (11 through a local alias), no caller relies on it — unchanged; 319 read-only lambda parameters get & (no copy per call: /my/schedule 2.35 → 0.46 s, /my/unwatched 13.7 → 5.7 s on the real copy); /my/unwatched one merge sort instead of n² inserts; docker-compose HL_GC_BYTES=268435456; deploy.sh runs kinds.mjs + franchises.mjs too (default ports 8700–8710); tests/realdata-070.mjs; gates browser 325/0, kinds 32/0, franchises 50/0mre
  • 46b21389tracker (mission 070, conductor): the sync never destroys migrated data — the one-time summary step MOVES a copied summary to migratedSummary (marker summaries-moved.txt) instead of clearing it; a migrated record's first title/genres/homepage/tagline TMDB replaces → migratedTitle/migratedGenres/migratedHomepage/migratedTagline, a migrated season's/episode's title/summary → migratedTitle/migratedSummary (set once); tests/peek-shows.hl prints them; gate 325/0mre
  • 749019b0Merge t19 (tracker#19 franchises + timelines) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs/faketmdb.mjs, both sides kept; franchise/timeline pages get #20's typed heading (Franchise | …, Timeline | …), their title links via titlePath (/movies|/series|/shows); the collection seed waits for repair, kinds and credits too; franchises.mjs URLs + 2 new checks; gates browser 323/0, kinds 32/0, franchises 50/0mre
  • daf49feaMerge t20 (tracker#20 typed headings + #21 series/shows split) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/search.hl/components/search.hl/browser.mjs/faketmdb.mjs, both sides kept; clock order backfill → repair → kinds → credits; withDetailsFields stores tmdbType + kind; gates: browser.mjs URLs → /series|/movies, typed h1 selectors; kinds.mjs repair/credits off, fixture name = seed name; browser 323/0, kinds 32/0mre
  • 20e09d89Merge t18 (tracker#18 delta sync) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs, both sides kept; pageShowOf summary = summaryOfmre
  • f83571c3tracker#18 (mission 067): daily sync by change lists — TMDB /tv|movie/changes (since the stored day, paged) + TVmaze /updates/shows → only our changed titles (followed: full step, unfollowed: light step — changed seasons, no TVmaze), full walk on first run / gap > 14 days / failed list; show record refreshed (title, tmdbSummary, tagline, status, genres …; renamed titles re-indexed); summary = the creator's own text (page: summary > tmdbSummary > tvmazeSummary), one-time clear of copied summaries (9,647 on the live copy); gate 261, tests/realdata-018*.mjs, README + STATUSmre
  • 1ed5457etracker#20 + #21 (mission 068): typed headings "<Type> | <name>" in type colours; TV titles split into Series (/series) and Shows (/shows) by TMDB type + Reality/Talk/News genres — kind stored by sync/import/adult backfill + new kind backfill (resumes), /movies/<slug>, /shows/<slug> of a series/movie → 301, /my/series + /my/shows, home 5 tiles + 3 rows, search/filmography labels; gates kinds 32 + browser 266, tests/realdata-068.mjs, tools/count-kinds.hl, docs/kinds.md, README + STATUSmre
  • f2b00674Merge t26 (tracker#26 + #28) into main (mission 062): short ids for every new person (castPersonId, guest route), guest stars stored on the title and created as people only when opened (/person/tmdb/<id>?show=<id> → 302), lean watch/follow clicks (showRow a small object, cast/crew from the slug, watches cached per user, face rows only after a season toggle); gate 311, tests/realdata-062.mjs, README + STATUSmre
  • 0553b51ftracker#19 (mission 066): franchises and timelines — tables, /franchises, /franchises/<slug>, /timelines/<slug> (Timeline | Release sort, series by last episode), the Prequel | Timeline | Sequel widget with the franchise above, the creator's editor, TMDB collection seed in the app (resumes, paced); gate tests/franchises.mjs 48/0 + browser.mjs 266/0, tests/realdata-066.mjs, docs/franchises.md, README + STATUSmre
  • fa1f9dfatracker#29 (mission 063): unwatched check muted grey outline + check (accent only on hover), watched stays solid — no code regression, the accent outline read as ticked; gate checks real checks visibly (computed style + screenshot pixel) on /my/unwatched, show, movie, /my/movies; gate 266, tests/realdata-063.mjs, README + STATUSmre
  • 10bb3f93tracker#28 (mission 061): full cast (all seasons, main cast by episodes, guest stars) + crew (created by, directed by, written by, screenplay, story, music) — stored by the details completion, the daily sync, the search import (one details request) and a background credits job (resumes, RSS limit); show page collapsed after 20 with client-side Show all; showBySlug via a slug map; gate 249, tests/realdata-028.mjs, README + STATUSmre
  • d8b12d67tracker#27 (mission 060): short ids for movies, series and persons — old 702 kept (data/old-short-ids.json), new random [a-z0-9]{5} unique across both, claimed at creation, background backfill (resumes), shown under poster/photo, /<shortId> → 301; gate 259, tests/realdata-060*, README + STATUSmre