How Automatic Subtitle Generation Works
AnyTool transcribes speech entirely in the browser without any cloud API. The audio is decoded in-browser via the Web Audio API (OfflineAudioContext) into a 16 kHz mono PCM array, then transcribed by a self-hosted Whisper model running on transformers.js / onnxruntime-web — the model and the ORT wasm are served same-origin from /models and /ort with env.allowRemoteModels=false, so nothing is fetched from Hugging Face at runtime. Inference uses WebGPU when available and falls back to WASM, and the segment timestamps build the SRT/VTT. The audio never leaves the device, unlike cloud auto-caption services. An optional burn-in step renders the subtitles into the picture using the self-hosted ffmpeg libass subtitles filter (no CDN). Single-threaded ffmpeg is deliberate: the multi-threaded core needs SharedArrayBuffer + cross-origin isolation (COOP/COEP), which would block cross-origin ad iframes.
- Runs client-side — the audio never leaves the device
- Web Audio API (OfflineAudioContext) → 16 kHz mono PCM
- Self-hosted Whisper (transformers.js / onnxruntime-web), served from /models + /ort
- env.allowRemoteModels=false — nothing fetched from Hugging Face at runtime
- WebGPU when available, WASM fallback; timestamps build SRT/VTT
- Optional burn-in via self-hosted ffmpeg libass subtitles filter (no CDN)
