feat(fetch): portable wasm build with passthrough and meta fixes #1

Merged

w33ble merged 13 commits from feat/wasm-portable-build into main

2026-10-06 12:24:40 -06:00

hijynx commented

2026-10-06 12:14:55 -06:00

Collaborator

Summary

Replace napi @kreuzberg/html-to-markdown dep (per-OS .node binaries) with @kreuzberg/html-to-markdown-wasm; embed 3.2MB wasm via initSync into single 4.5MB skill/resources/fetch.js runnable on Linux + macOS with no install step, no --compile, no per-arch matrix
Vendor wasm-bindgen loader in src/vendor (gen-wasm.ts regenerates gitignored base64 blob; chained into build/build:dev)
--meta: hybrid extraction (document.* from full-page HTML, headers/links/images from cleaned content fragment), plain-JSON serialization of wasm getters, markdown-mode parsing, empty envelope for plain/JSON
Strip heading permalink anchors pre-conversion (keeps MDN anchor-wrapped heading text, unwraps link syntax in header strings)
Pass through text/plain and JSON API responses unconverted (JSON retry with proper Accept); non-OK responses exit 1 with server body truncated to 500 chars on stderr; late markdown responses route through passthrough instead of HTML parser
README + SKILL.md updated (wasm setup, passthrough, --meta scope: content-fragment only, nav/footer excluded)

Why

The bundled script only ran on the build OS due to the native html-to-markdown runtime. This makes the skill downloadable and runnable anywhere Bun runs.

## Summary - Replace napi `@kreuzberg/html-to-markdown` dep (per-OS .node binaries) with `@kreuzberg/html-to-markdown-wasm`; embed 3.2MB wasm via `initSync` into single 4.5MB `skill/resources/fetch.js` runnable on Linux + macOS with no install step, no `--compile`, no per-arch matrix - Vendor wasm-bindgen loader in `src/vendor` (`gen-wasm.ts` regenerates gitignored base64 blob; chained into `build`/`build:dev`) - `--meta`: hybrid extraction (document.* from full-page HTML, headers/links/images from cleaned content fragment), plain-JSON serialization of wasm getters, markdown-mode parsing, empty envelope for plain/JSON - Strip heading permalink anchors pre-conversion (keeps MDN anchor-wrapped heading text, unwraps link syntax in header strings) - Pass through text/plain and JSON API responses unconverted (JSON retry with proper Accept); non-OK responses exit 1 with server body truncated to 500 chars on stderr; late markdown responses route through passthrough instead of HTML parser - README + SKILL.md updated (wasm setup, passthrough, --meta scope: content-fragment only, nav/footer excluded) ## Why The bundled script only ran on the build OS due to the native html-to-markdown runtime. This makes the skill downloadable and runnable anywhere Bun runs.

hijynx added 11 commits

2026-10-06 12:14:56 -06:00

feat(build): vendor kreuzberg wasm loader for portable builds 9a66418132

feat(convert): portable wasm build with embedded binary 39646a7e44

Replace napi html-to-markdown dep (per-OS .node binaries) with
@kreuzberg/html-to-markdown-wasm. Loader + base64 wasm vendored
in src/vendor, initSync embeds 3.2MB wasm into single 4.5MB
fetch.js runnable on linux + darwin without install step.

Serialize wasm-bindgen metadata getters to plain JSON for --meta.

fix(build): chain wasm vendoring into build, document portable setup 71d232236e

refactor(build): dedup wasm vendor step via bun run eace013622

fix(meta): extract document metadata from full page html a0ec800290

fix(extract): strip heading permalink anchors before conversion e7cecb1a14

fix(meta): preserve anchor-wrapped headings, support markdown-mode meta d29d763d52

feat(fetch): pass through text/plain responses unconverted 5de34d3717

feat(fetch): pass through JSON API responses unconverted 592ef12160

fix(fetch): fail on non-OK responses, preserve error body on stderr a927986f0f

fix(fetch): route late markdown responses through passthrough 93f0738fea

hijynx added 1 commit

2026-10-06 12:16:12 -06:00

docs(skill): document passthrough and meta scope 4556946e6b