gitoriaLog in with ident

tracker

All repositories: gitoria

ReadmeCodePull requestsReleasesTicketsSettings
Commit661c2592661c2592tracker: report 028mre661c2592/plugins/crypto/server.hl

4.5 KB

  1. \* hl:crypto — passwords first. Native realm wrapper (see server.js for the JS twin).
  2. The whole surface is eight calls, and six of them exist to serve the first two.
  3. What this plugin is FOR is that an application never stores a password: it
  4. stores the answer to "could this password have produced that", and the answer
  5. carries its own algorithm and cost so it stays readable when both change.
  6. hash(password, options) the stored value — a PHC string
  7. verify(password, stored) true / false, in constant time
  8. parsePhc(stored) what a stored value says about itself
  9. kdf() which algorithm THIS host hashes with
  10. sha256(data) content hashing (fast on purpose — not for passwords)
  11. randomBytes(n, encoding) n bytes from the kernel CSPRNG
  12. toBase64(data) a String's or a Bytes' bytes as base64 text
  13. fromBase64(text) base64 text back to a Bytes (null if it is not base64)
  14. The realm is SERVER (plugin.json). A password never crosses to the client, so
  15. importing this file is also a declaration about where the importing code runs. *\
  16. \* Hash a password for storage. Returns a self-describing PHC string:
  17. $argon2id$v=19$m=32768,t=2,p=1$<salt>$<hash>
  18. $scrypt$ln=15,r=8,p=1$<salt>$<hash>
  19. Every call salts freshly, so hashing the same password twice gives two
  20. different strings and both verify.
  21. `options` is optional: { cost = 15; kdf = "argon2id" }
  22. cost base-2 log of the working memory in KiB — 15 is 32 MiB, the default.
  23. CAPPED to 10..17 (1 MiB .. 128 MiB); the string records what was
  24. actually used, so asking for 999 and reading the result back is how
  25. you see the cap rather than being told about it.
  26. kdf force an algorithm instead of taking the host's best one. Normally
  27. unnecessary: `hash` picks argon2id when the system libcrypto has it
  28. (OpenSSL >= 3.2) and scrypt otherwise, and `verify` reads both. *\
  29. hash(password, options) {
  30. return __native("crypto.hash", password, options)
  31. }
  32. \* Check a password against a stored PHC string. The comparison is constant-time
  33. and the answer is a plain boolean: a wrong password, a truncated string, a
  34. flipped character and a string that is not PHC at all are all `false`. A
  35. stored string whose ALGORITHM this host cannot compute is a loud error
  36. instead, because answering `false` to that would read as "wrong password". *\
  37. verify(password, stored) {
  38. return __native("crypto.verify", password, stored)
  39. }
  40. \* Read a stored string without the password:
  41. { kdf = "argon2id"; version = 19; params = { m; t; p }; saltLen; hashLen }
  42. { kdf = "scrypt"; version = null; params = { ln; r; p }; saltLen; hashLen }
  43. `null` when the string is not a PHC string this plugin understands — which is
  44. also the cheapest way to spot a store that was never migrated. *\
  45. parsePhc(stored) {
  46. return __native("crypto.parse", stored)
  47. }
  48. \* Which algorithm `hash()` writes with on this host. Reporting only — nothing
  49. needs to branch on it, because every stored string names its own. *\
  50. kdf() {
  51. return __native("crypto.kdf")
  52. }
  53. \* SHA-256 of a string, as 64 lowercase hex characters. Content hashing: fast by
  54. design, and therefore exactly the wrong tool for a password. *\
  55. sha256(data) {
  56. return __native("crypto.sha256", data)
  57. }
  58. \* n random bytes from the kernel CSPRNG, rendered as "hex" (the default) or
  59. "base64". n is 1..1024. Suitable for session ids, one-time tokens and nonces. *\
  60. randomBytes(n, encoding) {
  61. return __native("crypto.random_bytes", n, encoding)
  62. }
  63. \* Base64 (the standard alphabet, padded) of a String's bytes — its UTF-8 — or
  64. of a Bytes, byte for byte:
  65. toBase64('user:pa55') \\ "dXNlcjpwYTU1"
  66. toBase64(req.bytes) \\ any bytes at all, NUL and 0xff included *\
  67. toBase64(
  68. data \\ a String or a Bytes
  69. ) {
  70. if (hlTypeName(data) == 'Bytes') {
  71. return __native("crypto.base64_encode_hex", data.hex())
  72. }
  73. return __native("crypto.base64_encode", data)
  74. }
  75. \* Base64 text back to its bytes, as a Bytes; `.toString()` of it is the text
  76. when the bytes are UTF-8. Padded and unpadded text both decode; anything
  77. that is not base64 in the standard alphabet is null, not an error — a
  78. malformed `Authorization: Basic` header is an answer, not a crash:
  79. let b = fromBase64(req.headers.authorization.slice(6))
  80. if (b != null) { let pair = b.toString() } \\ "user:pa55" *\
  81. fromBase64(String text) {
  82. let raw = __native("crypto.base64_decode", text)
  83. if (raw == null) {
  84. return null
  85. }
  86. return toBytes(raw)
  87. }

Branches

Latest commits

  • 661c2592tracker: report 028mre
  • 27c916fatracker mission 028: README ("Code order", the new file map), STATUS (counts before/after, tests, how to repeat, open), LOGmre
  • d924f398tracker mission 028: comments name the new files (sync.hl, dailysync.hl, backfill.hl, credits.hl, jobs.hl, images.hl …); tools/ref-params.py + tools/lambda-audit.py also scan lib/ (they globbed the root only), lambda-audit counts a plain `x = p` alias like `let x = p`mre
  • 2e89b968tracker mission 028 (code order) 5/5 let: `let` only where a variable is reassigned — 667 never-reassigned lets became plain declarations (project.hl, lib/, components/, tools/, tests/); kept: 264 in loop bodies (a plain declaration there is 'Cannot reassign' on the 2nd pass), 234 reassigned, 27 whose name is also a member/outer/free name (a plain write would rebind it); tools/let-audit.py decides and fixes (README 'Code order'); tests/realdata-m028.{sh,mjs} = the page-output diff on a real copy; gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 54796ff2tracker mission 028 (code order) 4/5 thin faces + last copies: the show page's check/follow faces call lib/watches.hl toggleWatched / toggleSeasonWatched (seasonAllWatched moved there) and lib/follows.hl toggleFollowed; both logins (header selector face, /login/callback) share lib/users.hl userOfCode; todayStr/listOf copies in components and the export readers copied into tools/migrate.hl + tools/old-short-ids.hl now once (lib/util.hl, lib/export.hl); gates 342/0, 32/0, 52/0; old-short-ids output byte-identical, migrate output identicalmre
  • 06b078e3tracker mission 028 (code order) 3/5 project.hl is the map: config, routes, wiring and a feature → file index (914 → 258 lines); the background jobs (daily sync run, backfills, details repair, credits job, merge, short ids, collection seed) moved unchanged into lib/jobs.hl (a class: their state is reassigned every step, a static cannot be; one instance made after the server), the login callback into lib/users.hl, poster/photo serving into lib/images.hl, the /shows/<slug> rule into lib/shows.hl showsMovedPath; route handlers are thin wrappers; gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 94716fd2tracker mission 028 (code order) 2/5 util + topics: lib/util.hl holds envOr, storageDir, postersDir, profilesDir, newId, hexDigits, todayStr, dateOr, textOr, hasId, listOr, firstOf, sortDesc once (were copied into up to 5 files); tmdbsync.hl split into tmdb.hl (TMDB/TVmaze requests), sync.hl (one title's sync), sync-helpers.hl, backfill.hl; details.hl split into details.hl, credits.hl, credits-helpers.hl (isIncomplete to shows.hl); search-helpers.hl (words, query, ranking, slugs); collections.hl (the TMDB collection seed, out of franchises.hl); deltasync.hl renamed dailysync.hl; no behaviour change: gates 342/0, 32/0, 52/0, real-copy pages identicalmre
  • 186079b0tracker mission 028 (code order) 1/5 move: every root .hl except project.hl into lib/ (styles.hl into components/), import paths only; gates 342/0, 32/0, 52/0; real-copy pages identicalmre
  • 4f47f181tracker: report 027mre
  • dc1d4be4tracker mission 027: Hybriel master 06617221 vendored (plugin allocator fixes 3a781359 + 413f60e4); real copy RSS through first-start jobs + 400 loads flat ~2.55 GB (190aa11d 2.3 -> 5.6 GB), page times <= 1.1x; gates 342/0, 32/0, 52/0mre
  • 84e1b3e1tracker: reports 025 + 026mre
  • dc40d859tracker#31 (mission 026): duplicate titles merged — the 68 type+tmdbId pairs held by 157 records were the old tracker's (all migrated); merge.hl repair job (own clock, before the TMDB jobs) keeps one keeper per title (follows/watches > old short id > oldest), moves follows, watches, seasons, cast, credits, timelines, tombstones the rest (mergedInto, never deleted), slugs + short ids 301 to the keeper; stray seasons merged into their listed twin (Reacher S3 watches) or linked when watched; search import re-checks before its put; deploy.sh waits up to 90 s for 200; real copy 68 -> 0 dup ids, az5b2 follows/watches equal; gates 342/0, 32/0, 52/0mre
  • 2667da05tracker#30 (mission 025): /my/ pages from slim cached title cards, episode rows and watch sets (after the jobs /my/series 1.8 s -> 0.06 s, /my/unwatched 4.1 -> 0.18 s); timeline page shows its name once; franchise widget under the poster/title; movies with TV leftovers (First Contact) go through the details repair; tools/count-tmdb-ids.hl; gates 327/0, 52/0, 32/0mre
  • 3909810dantcolony#40: mission references in README/STATUS/docs point to the moved missionsmre
  • e63b1d28antcolony#40: history (LOG.md), worker briefs (missions/) and reports moved here from antcolony, numbered per project; old numbers in antcolony docs/mission-map.mdmre
  • 1cda451dtracker: Hybriel master 190aa11d (#127 both shapes, GC correctness fc838894) — conductor adopts despite /my/series 2.4x after jobs (memory 8.0 → 1.7 GB boot); see reports/071mre
  • c1fa2f2etracker (mission 071): Hybriel master 190aa11d measured on the real copy vs the live binary 8590df63 — NOT adopted (after the first-start jobs /my/series 2.4x slower, /series 1.6x, RSS swings 7.4-12.2 GB; fresh it is flat at 1.7-2.2 GB and /my/unwatched faster), vendor stays 8efba065, candidate kept in .scratch/w071/vendor-190aa11d; tests/kinds.mjs: collection seed off (its TMDB request broke check 1 in 1 of 4 runs); tests/realdata-071.sh + realdata-071-bench.mjs + tools/realdata-071-table.py; README + STATUS (numbers, how to repeat); gates 325/0, 32/0, 50/0mre
  • 8081350atracker docs (mission 070): README (summary, Config HL_GC_BYTES — kept at Hybriel's default, the 256 MiB setting is taken out of docker-compose.yml again: the jobs grew to 12+ GB with it too, see STATUS), Test (three gates), Deploy (first start ~50 min: kinds then seed, restart once after collections done, memory numbers), Vendored Hybriel 8efba065 + #48 audit, Files; STATUS mission 070 entry (merges, migrated counts, lambda audit, gates, RSS old vs new, how to repeat, open points); docs/kinds.md + docs/franchises.md job order; tests/realdata-070-*.sh, tools/count-migrated.hl, tools/ref-params.py, tools/lambda-audit.pymre
  • 1ad19c8ctracker: re-vendor Hybriel master 8efba065 (#126 GC by bytes, #48 lambda parameters copy) (mission 070): bin/hybriel sha256 50361e95…, plugins core crypto data fetch fs http http1 mpackdb proc smtp time web; lambda audit: 13 lambdas change a passed record/list (11 through a local alias), no caller relies on it — unchanged; 319 read-only lambda parameters get & (no copy per call: /my/schedule 2.35 → 0.46 s, /my/unwatched 13.7 → 5.7 s on the real copy); /my/unwatched one merge sort instead of n² inserts; docker-compose HL_GC_BYTES=268435456; deploy.sh runs kinds.mjs + franchises.mjs too (default ports 8700–8710); tests/realdata-070.mjs; gates browser 325/0, kinds 32/0, franchises 50/0mre
  • 46b21389tracker (mission 070, conductor): the sync never destroys migrated data — the one-time summary step MOVES a copied summary to migratedSummary (marker summaries-moved.txt) instead of clearing it; a migrated record's first title/genres/homepage/tagline TMDB replaces → migratedTitle/migratedGenres/migratedHomepage/migratedTagline, a migrated season's/episode's title/summary → migratedTitle/migratedSummary (set once); tests/peek-shows.hl prints them; gate 325/0mre