gitoriaLog in with ident

tracker

All repositories: gitoria

ReadmeCodePull requestsReleasesTicketsSettingsCommitdc40d859dc40d859tracker#31 (mission 026): duplicate titles merged — the 68 type+tmdbId pairs held by 157 records were the old tracker's (all migrated); merge.hl repair job (own clock, before the TMDB jobs) keeps one keeper per title (follows/watches > old short id > oldest), moves follows, watches, seasons, cast, credits, timelines, tombstones the rest (mergedInto, never deleted), slugs + short ids 301 to the keeper; stray seasons merged into their listed twin (Reacher S3 watches) or linked when watched; search import re-checks before its put; deploy.sh waits up to 90 s for 200; real copy 68 -> 0 dup ids, az5b2 follows/watches equal; gates 342/0, 32/0, 52/0mredc40d859/tests/seed-dupes.hl

9.0 KB

  1. // tests/seed-dupes.hl — GATE FIXTURE for mission 026 (tracker#31, duplicate titles): written by tests/browser.mjs into the gate's
  2. // storage while the app is STOPPED (one process per storage, hybriel#21), late in the gate. Self-assigned ids, one put() each.
  3. // Shaped like the live data's migrated duplicates (Mongo-like oldIds in creation order, shared slugs):
  4. // Dupe Gate Series (tv 1301) ×3:
  5. // A oldest, slug Dupe-Gate-Series, short id dupa1: S1 (E1, E2) + S3 (E1 — B has no S3 → moved), cast Dupe Cast A
  6. // B slug Dupe-Gate-Series-2, short id dupb1: alice FOLLOWS it, watched S1E1 → the keeper; S1 (E1, E2) + S2 (E1), cast Dupe Cast B
  7. // C slug Dupe-Gate-Series-2 (shared, like Archive 81), short id dupc1: S1 (E1, E2, E3 — B has no E3 → moved), alice watched
  8. // S1E2 there (→ B's E2); a person's credit (Dupe Credit Person) and a timeline (Dupe Gate Line) name C
  9. // Dupe Gate Movie (movie 1303) ×2: M1 older (slug Dupe-Gate-Movie, dupm1), M2 (slug Dupe-Gate-Movie-2, dupm2) alice watched → keeper
  10. // Stray Gate Series (tv 1302), one record: listed S1 (E1, E2, the sync's kind: tmdb-season oldId); stray seasons naming it but
  11. // not listed — S1' (E1, E2: alice watched both + checked the season) → merged into S1; S2' (E1: alice watched) → linked;
  12. // S3' (E1, nobody watched) → stays as it is
  13. import { MPackDB } from 'hl:mpackdb'
  14. import { env } from 'hl:proc'
  15. import { now } from 'hl:time'
  17. storageDir = env('TRACKER_STORAGE')
  18. if (storageDir == null || !storageDir.startsWith('/')) { hlError('set TRACKER_STORAGE, absolute') }
  19. userId = env('TRACKER_SEED_USER')
  20. if (userId == null || userId == '') { hlError('set TRACKER_SEED_USER') }
  22. showsTable = new MPackDB(file = storageDir + '/shows.db', primaryKey = '@id', indexes = ['!oldId'])
  23. seasonsTable = new MPackDB(file = storageDir + '/seasons.db', primaryKey = '@id', indexes = ['!oldId'])
  24. episodesTable = new MPackDB(file = storageDir + '/episodes.db', primaryKey = '@id', indexes = ['!oldId'])
  25. followsTable = new MPackDB(file = storageDir + '/follows.db', primaryKey = '@id', indexes = ['!oldId'])
  26. watchesTable = new MPackDB(file = storageDir + '/watches.db', primaryKey = '@id', indexes = ['!oldId'])
  27. personsTable = new MPackDB(file = storageDir + '/persons.db', primaryKey = '@id', indexes = ['!oldId'])
  28. timelinesTable = new MPackDB(file = storageDir + '/timelines.db', primaryKey = '@id', indexes = ['!slug'])
  30. title = (id, tmdbId, type, name, slug, sid, oldId, seasons, cast) => {
  31. showsTable.put({ adult = false id = id title = name summary = null tmdbSummary = 'a duplicate' tmdbId = tmdbId
  32. language = 'en' type = type release = '2019-04-04' year = 2019 image = null urlSegment = slug
  33. episodesCount = null homepage = null imdbId = null seasonsCount = null status = 'Ended' tagline = null
  34. tvdbId = null tvmzId = null genres = [] cast = cast seasons = seasons oldId = oldId shortId = sid kind = type == 'series' ? 'series' : null
  35. creditsAt = 1 detailsAt = 1 })
  36. return true
  37. }
  38. season = (id, show, n, eps, oldId) => {
  39. seasonsTable.put({ id = id seasonNumber = n title = 'Season ' + n release = '2019-04-04' summary = '' show = show episodes = eps oldId = oldId })
  40. return true
  41. }
  42. episode = (id, show, seasonId, n, e, name) => {
  43. episodesTable.put({ id = id episodeNumber = e seasonNumber = n title = name summary = '' release = '2019-04-0' + e show = show season = seasonId oldId = 'o' + id })
  44. return true
  45. }
  46. watch = (id, type, target) => {
  47. watchesTable.put({ id = id targetType = type target = target user = userId at = now() oldId = 'gw-' + id })
  48. return true
  49. }
  51. // Dupe Gate Series
  52. episode('dpae000000000a11', 'dpa000000000000a', 'dpas00000000000a', 1, 1, 'Pilot A')
  53. episode('dpae000000000a12', 'dpa000000000000a', 'dpas00000000000a', 1, 2, 'Second A')
  54. season('dpas00000000000a', 'dpa000000000000a', 1, ['dpae000000000a11', 'dpae000000000a12'], '5f00000000000000000000a1')
  55. episode('dpae000000000a31', 'dpa000000000000a', 'dpas0000000000a3', 3, 1, 'Third Season Start')
  56. season('dpas0000000000a3', 'dpa000000000000a', 3, ['dpae000000000a31'], '5f00000000000000000000a3')
  57. title('dpa000000000000a', 1301, 'series', 'Dupe Gate Series', 'Dupe-Gate-Series', 'dupa1', '5f0000000000000000000001', ['dpas00000000000a', 'dpas0000000000a3'],
  58. [{ person = 'dpp000000000000a' character = 'Role A' actor = 'Dupe Cast A' }])
  59. episode('dpbe000000000b11', 'dpb000000000000b', 'dpbs00000000000b', 1, 1, 'Pilot')
  60. episode('dpbe000000000b12', 'dpb000000000000b', 'dpbs00000000000b', 1, 2, 'Second')
  61. season('dpbs00000000000b', 'dpb000000000000b', 1, ['dpbe000000000b11', 'dpbe000000000b12'], '5f00000000000000000000b1')
  62. episode('dpbe000000000b21', 'dpb000000000000b', 'dpbs0000000000b2', 2, 1, 'Second Season Start')
  63. season('dpbs0000000000b2', 'dpb000000000000b', 2, ['dpbe000000000b21'], '5f00000000000000000000b2')
  64. title('dpb000000000000b', 1301, 'series', 'Dupe Gate Series', 'Dupe-Gate-Series-2', 'dupb1', '5f0000000000000000000002', ['dpbs00000000000b', 'dpbs0000000000b2'],
  65. [{ person = 'dpp000000000000b' character = 'Role B' actor = 'Dupe Cast B' }])
  66. episode('dpce000000000c11', 'dpc000000000000c', 'dpcs00000000000c', 1, 1, 'Pilot C')
  67. episode('dpce000000000c12', 'dpc000000000000c', 'dpcs00000000000c', 1, 2, 'Second C')
  68. episode('dpce000000000c13', 'dpc000000000000c', 'dpcs00000000000c', 1, 3, 'Only In C')
  69. season('dpcs00000000000c', 'dpc000000000000c', 1, ['dpce000000000c11', 'dpce000000000c12', 'dpce000000000c13'], '5f00000000000000000000c1')
  70. title('dpc000000000000c', 1301, 'series', 'Dupe Gate Series', 'Dupe-Gate-Series-2', 'dupc1', '5f0000000000000000000003', ['dpcs00000000000c'], [])
  71. personsTable.put({ id = 'dpp000000000000a' name = 'Dupe Cast A' urlSegment = 'Dupe-Cast-A' tmdbId = null oldId = 'dpp-a' shortId = 'dupp1' filmographyAt = 1
  72. shows = [{ show = 'dpa000000000000a' character = 'Role A' }] })
  73. personsTable.put({ id = 'dpp000000000000b' name = 'Dupe Cast B' urlSegment = 'Dupe-Cast-B' tmdbId = null oldId = 'dpp-b' shortId = 'dupp2' filmographyAt = 1
  74. shows = [{ show = 'dpb000000000000b' character = 'Role B' }] })
  75. personsTable.put({ id = 'dpp000000000000c' name = 'Dupe Credit Person' urlSegment = 'Dupe-Credit-Person' tmdbId = null oldId = 'dpp-c' shortId = 'dupp3' filmographyAt = 1
  76. shows = [{ show = 'dpc000000000000c' character = 'Guest In C' }] })
  77. // tests/paths-m026.hl: a person whose filmography is not fetched yet (fake TMDB person 3301: tv 1301, movie 1303, NEW movie 1304)
  78. personsTable.put({ id = 'dpp000000000000d' name = 'Dupe Fill Person' urlSegment = 'Dupe-Fill-Person' tmdbId = 3301 oldId = 'dpp-d' shortId = 'dupp4' })
  79. followsTable.put({ id = 'dpf00000000000f1' show = 'dpb000000000000b' user = userId at = now() oldId = 'gate-follow-dupe' })
  80. watch('dpw00000000000w1', 'episode', 'dpbe000000000b11')
  81. watch('dpw00000000000w2', 'episode', 'dpce000000000c12')
  82. timelinesTable.put({ id = 'dpt00000000000t1' name = 'Dupe Gate Line' slug = 'dupe-gate-line' franchise = null order = ['dpc000000000000c', 'dpm00000000000m2'] source = 'editor'
  83. tmdbCollection = null created = now() })
  85. // Dupe Gate Movie
  86. title('dpm00000000000m1', 1303, 'movie', 'Dupe Gate Movie', 'Dupe-Gate-Movie', 'dupm1', '5f0000000000000000000011', [], [])
  87. title('dpm00000000000m2', 1303, 'movie', 'Dupe Gate Movie', 'Dupe-Gate-Movie-2', 'dupm2', '5f0000000000000000000012', [], [])
  88. watch('dpw00000000000w3', 'movie', 'dpm00000000000m2')
  90. // Stray Gate Series
  91. episode('dpse000000000s11', 'dps000000000000s', 'dpss00000000000s', 1, 1, 'Stray Pilot')
  92. episode('dpse000000000s12', 'dps000000000000s', 'dpss00000000000s', 1, 2, 'Stray Second')
  93. season('dpss00000000000s', 'dps000000000000s', 1, ['dpse000000000s11', 'dpse000000000s12'], 'tmdb-season:1302:1')
  94. episode('dpse00000000x111', 'dps000000000000s', 'dpss0000000000x1', 1, 1, 'Old Stray Pilot')
  95. episode('dpse00000000x112', 'dps000000000000s', 'dpss0000000000x1', 1, 2, 'Old Stray Second')
  96. season('dpss0000000000x1', 'dps000000000000s', 1, ['dpse00000000x111', 'dpse00000000x112'], '5f00000000000000000000x1')
  97. episode('dpse00000000x211', 'dps000000000000s', 'dpss0000000000x2', 2, 1, 'Watched Special')
  98. season('dpss0000000000x2', 'dps000000000000s', 2, ['dpse00000000x211'], '5f00000000000000000000x2')
  99. episode('dpse00000000x311', 'dps000000000000s', 'dpss0000000000x3', 3, 1, 'Nobody Watched')
  100. season('dpss0000000000x3', 'dps000000000000s', 3, ['dpse00000000x311'], '5f00000000000000000000x3')
  101. title('dps000000000000s', 1302, 'series', 'Stray Gate Series', 'Stray-Gate-Series', 'dups1', '5f0000000000000000000021', ['dpss00000000000s'], [])
  102. followsTable.put({ id = 'dpf00000000000f2' show = 'dps000000000000s' user = userId at = now() - 1000 oldId = 'gate-follow-stray' })
  103. watch('dpw00000000000w4', 'episode', 'dpse00000000x111')
  104. watch('dpw00000000000w5', 'episode', 'dpse00000000x112')
  105. watch('dpw00000000000w6', 'season', 'dpss0000000000x1')
  106. watch('dpw00000000000w7', 'episode', 'dpse00000000x211')
  108. showsTable.persist()
  109. seasonsTable.persist()
  110. episodesTable.persist()
  111. followsTable.persist()
  112. watchesTable.persist()
  113. personsTable.persist()
  114. timelinesTable.persist()
  115. console.log('seed-dupes: Dupe Gate Series ×3, Dupe Gate Movie ×2, Stray Gate Series with 3 stray seasons; ' + userId + ' follows 2, 7 watches')

Branches

Latest commits

  • dc40d859tracker#31 (mission 026): duplicate titles merged — the 68 type+tmdbId pairs held by 157 records were the old tracker's (all migrated); merge.hl repair job (own clock, before the TMDB jobs) keeps one keeper per title (follows/watches > old short id > oldest), moves follows, watches, seasons, cast, credits, timelines, tombstones the rest (mergedInto, never deleted), slugs + short ids 301 to the keeper; stray seasons merged into their listed twin (Reacher S3 watches) or linked when watched; search import re-checks before its put; deploy.sh waits up to 90 s for 200; real copy 68 -> 0 dup ids, az5b2 follows/watches equal; gates 342/0, 32/0, 52/0mre
  • 2667da05tracker#30 (mission 025): /my/ pages from slim cached title cards, episode rows and watch sets (after the jobs /my/series 1.8 s -> 0.06 s, /my/unwatched 4.1 -> 0.18 s); timeline page shows its name once; franchise widget under the poster/title; movies with TV leftovers (First Contact) go through the details repair; tools/count-tmdb-ids.hl; gates 327/0, 52/0, 32/0mre
  • 3909810dantcolony#40: mission references in README/STATUS/docs point to the moved missionsmre
  • e63b1d28antcolony#40: history (LOG.md), worker briefs (missions/) and reports moved here from antcolony, numbered per project; old numbers in antcolony docs/mission-map.mdmre
  • 1cda451dtracker: Hybriel master 190aa11d (#127 both shapes, GC correctness fc838894) — conductor adopts despite /my/series 2.4x after jobs (memory 8.0 → 1.7 GB boot); see reports/071mre
  • c1fa2f2etracker (mission 071): Hybriel master 190aa11d measured on the real copy vs the live binary 8590df63 — NOT adopted (after the first-start jobs /my/series 2.4x slower, /series 1.6x, RSS swings 7.4-12.2 GB; fresh it is flat at 1.7-2.2 GB and /my/unwatched faster), vendor stays 8efba065, candidate kept in .scratch/w071/vendor-190aa11d; tests/kinds.mjs: collection seed off (its TMDB request broke check 1 in 1 of 4 runs); tests/realdata-071.sh + realdata-071-bench.mjs + tools/realdata-071-table.py; README + STATUS (numbers, how to repeat); gates 325/0, 32/0, 50/0mre
  • 8081350atracker docs (mission 070): README (summary, Config HL_GC_BYTES — kept at Hybriel's default, the 256 MiB setting is taken out of docker-compose.yml again: the jobs grew to 12+ GB with it too, see STATUS), Test (three gates), Deploy (first start ~50 min: kinds then seed, restart once after collections done, memory numbers), Vendored Hybriel 8efba065 + #48 audit, Files; STATUS mission 070 entry (merges, migrated counts, lambda audit, gates, RSS old vs new, how to repeat, open points); docs/kinds.md + docs/franchises.md job order; tests/realdata-070-*.sh, tools/count-migrated.hl, tools/ref-params.py, tools/lambda-audit.pymre
  • 1ad19c8ctracker: re-vendor Hybriel master 8efba065 (#126 GC by bytes, #48 lambda parameters copy) (mission 070): bin/hybriel sha256 50361e95…, plugins core crypto data fetch fs http http1 mpackdb proc smtp time web; lambda audit: 13 lambdas change a passed record/list (11 through a local alias), no caller relies on it — unchanged; 319 read-only lambda parameters get & (no copy per call: /my/schedule 2.35 → 0.46 s, /my/unwatched 13.7 → 5.7 s on the real copy); /my/unwatched one merge sort instead of n² inserts; docker-compose HL_GC_BYTES=268435456; deploy.sh runs kinds.mjs + franchises.mjs too (default ports 8700–8710); tests/realdata-070.mjs; gates browser 325/0, kinds 32/0, franchises 50/0mre
  • 46b21389tracker (mission 070, conductor): the sync never destroys migrated data — the one-time summary step MOVES a copied summary to migratedSummary (marker summaries-moved.txt) instead of clearing it; a migrated record's first title/genres/homepage/tagline TMDB replaces → migratedTitle/migratedGenres/migratedHomepage/migratedTagline, a migrated season's/episode's title/summary → migratedTitle/migratedSummary (set once); tests/peek-shows.hl prints them; gate 325/0mre
  • 749019b0Merge t19 (tracker#19 franchises + timelines) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs/faketmdb.mjs, both sides kept; franchise/timeline pages get #20's typed heading (Franchise | …, Timeline | …), their title links via titlePath (/movies|/series|/shows); the collection seed waits for repair, kinds and credits too; franchises.mjs URLs + 2 new checks; gates browser 323/0, kinds 32/0, franchises 50/0mre
  • daf49feaMerge t20 (tracker#20 typed headings + #21 series/shows split) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/search.hl/components/search.hl/browser.mjs/faketmdb.mjs, both sides kept; clock order backfill → repair → kinds → credits; withDetailsFields stores tmdbType + kind; gates: browser.mjs URLs → /series|/movies, typed h1 selectors; kinds.mjs repair/credits off, fixture name = seed name; browser 323/0, kinds 32/0mre
  • 20e09d89Merge t18 (tracker#18 delta sync) into main (mission 070): conflicts README/STATUS/show.hl/project.hl/browser.mjs, both sides kept; pageShowOf summary = summaryOfmre
  • f83571c3tracker#18 (mission 067): daily sync by change lists — TMDB /tv|movie/changes (since the stored day, paged) + TVmaze /updates/shows → only our changed titles (followed: full step, unfollowed: light step — changed seasons, no TVmaze), full walk on first run / gap > 14 days / failed list; show record refreshed (title, tmdbSummary, tagline, status, genres …; renamed titles re-indexed); summary = the creator's own text (page: summary > tmdbSummary > tvmazeSummary), one-time clear of copied summaries (9,647 on the live copy); gate 261, tests/realdata-018*.mjs, README + STATUSmre
  • 1ed5457etracker#20 + #21 (mission 068): typed headings "<Type> | <name>" in type colours; TV titles split into Series (/series) and Shows (/shows) by TMDB type + Reality/Talk/News genres — kind stored by sync/import/adult backfill + new kind backfill (resumes), /movies/<slug>, /shows/<slug> of a series/movie → 301, /my/series + /my/shows, home 5 tiles + 3 rows, search/filmography labels; gates kinds 32 + browser 266, tests/realdata-068.mjs, tools/count-kinds.hl, docs/kinds.md, README + STATUSmre
  • f2b00674Merge t26 (tracker#26 + #28) into main (mission 062): short ids for every new person (castPersonId, guest route), guest stars stored on the title and created as people only when opened (/person/tmdb/<id>?show=<id> → 302), lean watch/follow clicks (showRow a small object, cast/crew from the slug, watches cached per user, face rows only after a season toggle); gate 311, tests/realdata-062.mjs, README + STATUSmre
  • 0553b51ftracker#19 (mission 066): franchises and timelines — tables, /franchises, /franchises/<slug>, /timelines/<slug> (Timeline | Release sort, series by last episode), the Prequel | Timeline | Sequel widget with the franchise above, the creator's editor, TMDB collection seed in the app (resumes, paced); gate tests/franchises.mjs 48/0 + browser.mjs 266/0, tests/realdata-066.mjs, docs/franchises.md, README + STATUSmre
  • fa1f9dfatracker#29 (mission 063): unwatched check muted grey outline + check (accent only on hover), watched stays solid — no code regression, the accent outline read as ticked; gate checks real checks visibly (computed style + screenshot pixel) on /my/unwatched, show, movie, /my/movies; gate 266, tests/realdata-063.mjs, README + STATUSmre
  • 10bb3f93tracker#28 (mission 061): full cast (all seasons, main cast by episodes, guest stars) + crew (created by, directed by, written by, screenplay, story, music) — stored by the details completion, the daily sync, the search import (one details request) and a background credits job (resumes, RSS limit); show page collapsed after 20 with client-side Show all; showBySlug via a slug map; gate 249, tests/realdata-028.mjs, README + STATUSmre
  • d8b12d67tracker#27 (mission 060): short ids for movies, series and persons — old 702 kept (data/old-short-ids.json), new random [a-z0-9]{5} unique across both, claimed at creation, background backfill (resumes), shown under poster/photo, /<shortId> → 301; gate 259, tests/realdata-060*, README + STATUSmre
  • 25a50bc4tracker#26 (mission 059): titles from a filmography are completed — on open (skeleton, step-wise face showComplete, no reload) and by the in-app details repair (resumes, TMDB-paced, series in parts); cast from TMDB credits; gate 231, tests/realdata-026.mjs, README + STATUSmre