What the Alt Text Generator Does
The Alt Text Generator helps a user write accessible, WCAG-compliant alt text for an image without ever uploading it. The user drops in an image and an ON-DEVICE AI vision model — a self-hosted ViT-GPT2 image-captioning model (a Vision Transformer image encoder paired with a GPT-2 text decoder) — writes a draft caption directly in the browser. That draft is then automatically refined toward best practice and continuously audited against WCAG success criterion 1.1.1 as the user edits it. The tool also offers a no-AI “describe it yourself” guided writer (which builds a correct alt attribute from a few structured fields) and a standalone auditor for pasting existing alt text. Throughout, the user picks the image’s ROLE — informative, decorative, functional (link/button) or complex (chart/graph) — because the correct alt text differs for each: decorative images take an empty alt="", functional images describe the action, and complex images need a longer description nearby.
- On-device AI vision model (self-hosted ViT-GPT2) drafts a caption from the image
- The image is never uploaded — only the model code/weights download once, then cache
- Draft is refined to best practice, then audited live against WCAG 1.1.1
- Role-aware: informative / decorative (alt="") / functional / complex
- Also a no-AI guided writer and a standalone alt-text auditor
How It Drafts, Refines and Audits Alt Text
The AI half mirrors the app’s proven transformers.js wiring (the same package, self-hosting and ONNX Runtime Web configuration used by its in-browser Whisper speech tool and distilbart summarizer): a lazy, module-singleton image-to-text pipeline loads the ViT-GPT2 model from this origin’s /public/models with env.allowRemoteModels = false — never contacting huggingface.co — and runs on WebAssembly with q8-quantized weights for universal device support, reporting download progress. Given a local object URL for the image, it produces a caption with a small beam search. The pure refineCaption helper then cleans that raw caption: it strips redundant openers (“there is an image of…”, “a photo of…”, “picture showing…”) because screen readers already announce the element as an image, collapses whitespace, sentence-cases the first letter, drops the trailing period (alt text is a fragment) and trims on a word boundary to the ~125-character screen-reader window. The pure auditAltText helper runs the role-aware WCAG checks: empty-alt required for decorative images and forbidden for informative ones, a ≤125-character length check, a “no image of/photo of prefix” check, a filename-detector (IMG_1234.jpg, hero-banner-2, 800x600), a keyword-stuffing/repetition heuristic, an action-based check for functional images, and a long-description nudge for complex charts — each returned as a pass/warn/fail finding with a plain-language message.
- transformers.js image-to-text pipeline, self-hosted from /public/models (no CDN, no huggingface.co)
- WebAssembly + q8 weights, mirroring the app’s Whisper/summarizer setup
- refineCaption: strip “image of” openers, sentence-case, drop period, trim to ~125 chars
- auditAltText: length, prefix, filename, keyword-stuffing, decorative/functional/complex rules
- Both refinement and audit are pure functions — no model needed, instant per keystroke
The AI Caption Is a Draft — Review It for Accuracy and Context
The tool is prominently HONEST that the AI caption is a STARTING POINT, not a finished answer. The on-device caption model is small, so it can be generic, miss important detail, or be plain wrong — and crucially, no model can know WHY an image is on a page or what it means in the surrounding content, which is exactly what good alt text must capture. So the user must always REVIEW AND EDIT the draft for accuracy and the image’s PURPOSE IN CONTEXT. Role matters and only a human can decide it: decorative images should use an empty alt="" (any AI description would be wrong); functional images (links/buttons) must describe the action, not the picture; complex images (charts, graphs, diagrams) need their data conveyed in a longer description nearby, with only a short label as alt. The WCAG auditor catches well-formedness problems but cannot verify that the words actually match the image or its meaning — a clean audit is not a guarantee of good alt text. Finally, the first generation downloads a ~247 MB model and requires a modern browser with WebAssembly and enough memory; if the model cannot load or run, the tool degrades gracefully to the no-AI guided writer, which produces correct, audited alt text with no download.
- AI caption is a draft — small on-device models can be generic, miss context or be wrong
- Only a human knows the image’s purpose in context — always review and edit
- Decorative → alt=""; functional → describe the action; complex → long description nearby
- A clean WCAG audit is not a guarantee the words match the image
- First run downloads ~247 MB and needs a modern browser; falls back to the guided writer
Why On-Device Captioning Is a Privacy Win
Cloud alt-text and “AI vision” services send your image to a remote server to describe it. The Alt Text Generator instead runs the vision model ENTIRELY on the user’s device: the image is decoded from a local preview (an object URL) and captioned in the browser tab with WebAssembly — nothing is uploaded, logged or stored, and no image ever leaves the machine. The only network fetch is the model itself (weights and runtime, about 247 MB), served from this origin and downloaded ONCE, after which the browser caches it and the tool works offline. That makes it safe to caption confidential, unreleased, client or personal images that a user would not want to hand to a third-party AI service. The setup reuses the app’s existing self-hosted transformers.js infrastructure (the same pattern as its in-browser Whisper transcription and distilbart summarization), so it adds no new dependency and no new external service — object URLs are revoked on replace and unmount so nothing lingers in memory.
- The vision model runs on-device (WebAssembly) — the image is never uploaded
- Only the model weights/runtime download once (self-hosted), then cache; works offline after
- Safe for confidential, unreleased, client or personal images
- Reuses the app’s self-hosted transformers.js infra — no new dependency, no external service
- Object URLs are revoked on replace/unmount so nothing lingers