Digital Twin / how the answers work

How a question becomes a reply

Select a box to see what it does. You can then show that part in the animated example.

Digital Twin system mapA browser question passes through server-side professional context and an external provider, then returns as text with optional speech and character performance.BROWSER / INPUTSERVER FUNCTION + EXTERNAL PROVIDERBROWSER / DELIVERY + OPTIONAL PERFORMANCE1. Your questiontyped / optional microphone2. My work profileNetlify chat function3. AI providerGemini · external service4. Text replybrowser chat5. Voiceaudio energy / transcript6. Text fallbackno microphone / no 3D7. 3D characterThree.js / blended gesturesSolid path: reply route · dashed node: external service
Digital Twin compact system mapA browser question uses server context and an external provider, returns as text, and can additionally drive voice and avatar performance while a text fallback remains.1. Your questionbrowser input2. My work profileserver function3. AI providerexternal service4. Text replyback in the browser5. Voiceoptional timing6. Text fallbackno mic / no 3D7. 3D characteroptional gesturesOptional performance never replaces text.
Select a box, then use the button below to watch its example.

Inspect a component

  1. 1 · Your question

    Type a question about my work. You can also use the microphone when your browser supports it.

    Browser speech recognition and recorded-audio transcription depend on capability and permission.

    Source: mascot.js ↗ · pinned revision cc7771f

  2. 2 · My work profile

    The app uses a maintained profile of my work to guide the reply. AI can still be wrong.

    The Netlify chat function prepares the request. Provider credentials belong on the server, not in browser code.

    Source: netlify/functions/chat.js ↗ · pinned revision cc7771f

  3. 3 · AI provider

    An external AI service receives your question and relevant profile information, then generates a reply.

    The source documents Gemini through a server-side function. Profile guidance cannot guarantee factual answers.

    Source: README.md ↗ · pinned revision cc7771f

  4. 4 · Text reply

    The answer appears as text you can read, whether or not you use voice or the character.

    The browser keeps a chat transcript alongside optional audio playback.

    Source: mascot.js ↗ · pinned revision cc7771f

  5. 5 · Voice

    The reply can be read aloud. The mouth moves with the audio and text timing; the lip sync is approximate.

    Audio energy and transcript timing drive smoothed facial targets. The result is not phoneme-perfect lip sync.

    Source: README.md ↗ · pinned revision cc7771f

  6. 6 · Text fallback

    Keep typing and reading when the microphone or 3D character is unavailable.

    Microphone permission, browser support, WebGL and model loading can affect the optional experience.

    Source: README.md ↗ · pinned revision cc7771f

  7. 7 · 3D character

    The optional 3D character moves while the reply is spoken. Its gestures blend as the conversation changes.

    Authored gesture clips and facial targets are blended and smoothed. This original SVG puppet does not reuse the production identity model.

    Source: movement.mjs ↗ · pinned revision cc7771f