dsh-voice

DeepSeek's chat API is text-only — it can neither hear audio nor speak. dsh-voice bridges sound at the input/output boundary so the model never sees raw audio, yet gains a full voice loop:

🎤 speech → text → DeepSeek (text-only) → text → 🔊 speech

Same idea as dsh-vision-bridge — but for audio, the multimodal gap nobody has filled for DeepSeek Harness yet.

✨ Features

• ✅ Zero API key for the Web UI — pure browser speech, works out of the box.
• ✅ Zero new model — DeepSeek stays text-only; speech is handled at the edge.
• ✅ Configurable backends — point at local whisper.cpp / Kokoro for a fully free, keyless stack.

🧭 How it works

┌──────────────────────────────────────────────────────────────┐
│                         dsh Web GUI                           │
│                                                              │
│   you speak  ──🎤 SpeechRecognition──►  text  ──► input box   │
│                                                              │
│   reply text ──🔊 speechSynthesis──►  you hear                │
└──────────────────────────────────────────────────────────────┘
         │                                  ▲
         │ text (STT)                       │ text (TTS)
         ▼                                  │
┌──────────────────────────────────────────────────────────────┐
│              DeepSeek (text-only model)                       │
└──────────────────────────────────────────────────────────────┘

attached audio ── voice_transcribe (Whisper-compatible) ──► text ──► model
model wants to speak ── voice_speak (OpenAI-compatible TTS) ──► audio file

📦 Install

# from a local checkout
dsh plugin --profile web add "file:/path/to/dsh-voice"

# or, once published to npm
dsh plugin --profile web add dsh-voice

Activation is automatic: the package ships a bundle patch (cordis.patch.yml) and declares dsh.bundle.patch, so dsh plugin add registers it into the profile's bundles for you.

Then restart dsh web (or wait for HMR). You should see 🎤 in the composer and 🔊 on each reply.

⚙️ Configuration

The 🎤 mic button needs voice.stt.apiBase (the browser records audio and sends it to the host's Whisper-compatible backend). The 🔊 read-aloud needs nothing (browser speechSynthesis). To customize read-aloud language/rate/pitch, edit the constants at the top of lib/client.js (TTS_LANG, TTS_RATE, TTS_PITCH).

settings.yaml:

voice:
  stt:                        # mic button + voice_transcribe tool
    enabled: true
    apiBase: ""               # REQUIRED for the mic. Examples:
                              #   SiliconFlow: https://api.siliconflow.cn/v1
                              #   local whisper.cpp: http://127.0.0.1:8080/v1
    apiKeyEnv: VOICE_STT_API_KEY
    model: whisper-1
    language: ""              # zh / en / ... ; empty = auto-detect
  tts:                        # voice_speak tool
    enabled: true
    apiBase: ""               # empty = https://api.openai.com/v1
    apiKeyEnv: VOICE_TTS_API_KEY
    model: tts-1
    voice: alloy              # alloy/echo/fable/onyx/nova/shimmer, or a local voice id
    format: mp3

Why the mic needs a backend: Chrome's built-in SpeechRecognition uploads audio to Google, which is unreachable in some regions (you'd see 识别出错：network). dsh-voice records with MediaRecorder and transcribes through your Whisper-compatible backend instead. Two free, keyless options:

• SiliconFlow (China-friendly, free tier) — apiBase: https://api.siliconflow.cn/v1, model FunAudioLLM/SenseVoiceSmall or whisper-1.
• Local whisper.cpp — fully offline, apiBase: http://127.0.0.1:8080/v1 (no key).

🧰 Agent tools

🗂 Project layout

dsh-voice/
├── package.json          # dual-half plugin: host (main) + browser (client)
├── cordis.patch.yml      # bundle activation layer
├── lib/
│   ├── index.js          # host half: settings + voice_transcribe/voice_speak tools
│   ├── client.js         # browser half: 🎤 / 🔊 buttons
│   └── types/
│       ├── index.d.ts
│       └── client/index.d.ts
├── README.md             # this file
└── README.zh-CN.md       # 中文版

🗺 Roadmap

• Wire browser-UI language / rate / pitch / auto-read into the voice: settings page (currently code constants)
• autoRead: auto read-aloud on reply completion
• Built-in free edge-tts backend (no OpenAI key)
• Local Whisper STT via @xenova/transformers
• Sentence-level reading with streaming interruption

🙏 Credits

Inspired by these established voice solutions for other agents:

• slopus/happy (~23k★) — realtime voice interaction UX
• mbailey/voicemode (~1.3k★) — Claude Code voice mode
• caiovicentino/claude-call — local Whisper STT + edge-tts, no API key
• edge-tts — free Microsoft Edge neural voices
• ggerganov/whisper.cpp / OpenAI Whisper — speech recognition
• hexgrad/kokoro — local neural TTS

📄 License

MIT