dsh-web-tools

A unified multi-provider Web Runtime for DeepSeek Harness.

Preserves the native DSH web_search / web_fetch tool contracts while providing multi-provider aggregation, automatic fallback, multi-key pooling, quota and health monitoring, provider-native preference tuning, page content extraction, and per-session Search Mode at the host runtime layer.

English | 简体中文

Features

• Zero-Overhead SearchHints Semantic Layer: Built-in deterministic intent extraction directly from queries (coding, academic research, news freshness, and domain constraints like site:github.com or after:YYYY-MM-DD), intelligently mapping to native parameters and specialized indices without LLM planning overhead or latency.
• 8 Search Providers Deeply Adapted:
  • Parallel: Dual-layer semantics (objective soft-steering + clean search_queries) and source_policy domain/freshness filters.
  • Firecrawl: Coding/technical queries query the Developer Index (categories: ["developer"]), supporting research and tbs time filters.
  • Exa: Exact category mapping (publication / news / financial report), ISO-8601 date ranges, and domain constraints.
  • You.com: Native boost_domains soft-weighting, freshness presets, and geo/language targeting.
  • Brave Search: LLM Context endpoint with pd/pw/pm/py freshness filters, country, and search language.
  • Tavily: Full-tier chunks_per_source chunking control, news topic and time range filtering (gracefully falls back to general search for finance/code).
  • SearXNG: Self-hosted metasearch with categories (it/science/news) and time_range.
  • Jina: Query noise reduction and ReaderLM-v2 high-precision markdown extraction.
• Platform Search Sources (Xiaohongshu & Twitter / X):
  • Browser Bridge Architecture: Leverages user authenticated browser sessions via a lightweight MV3 extension, with 0 raw cookie storage on the Host process, bypassing fragile reverse-engineered APIs.
  • Xiaohongshu: Waterfall DOM incremental extraction preserving complete signed xsec_token URLs, note text, and engagement metrics.
  • Twitter / X: Native query operator mapping (from:, since:, lang:) and semantic DOM tweet extraction.
  • Graceful Web Fallback: Seamlessly falls back to site:xiaohongshu.com / (site:x.com OR site:twitter.com) when disconnected.
• Native DSH Tool Compatibility: No bespoke tools like web_search_exa; agents invoke standard web_search and web_fetch contracts seamlessly.
• Multi-Query Support: Handles DSH queries[] payloads concurrently across independent search dimensions.
• Multi-API-Key Pooling: Assigns keys per provider, balances concurrent requests by lowest in-flight count, and fails over across keys on authentication errors.
• Deterministic Provider Fallback: Automatically cascades through the fallback chain on network failures, timeouts, 5xx server errors, 429 rate limits, or exhausted quotas.
• 429 Retry-After Cooldown: Enforces zero-request cooldown windows when servers return Retry-After headers, skipping rate-limited providers immediately without redundant network overhead.
• Configurable Routing Policies: Supports Ordered, Round-Robin, and Random initial provider routing.
• One-Click Preference Presets: Offers Fast, Deep, Economy, and Recommended presets to quickly adjust execution parameters across providers.
• Native Page Extraction (Extract / Scrape): Transparently routes web_fetch to provider-native scraping backends (Exa /contents, Tavily /extract, Firecrawl /scrape, Parallel /v1/extract, You.com /v1/contents, Jina Reader).
• Session-Level Search Mode: Chat input toggle that forces the agent to complete web research before generating answers.
• Proxy and Self-Hosted Support: Full support for system proxies, HTTP(S)_PROXY, NO_PROXY, and self-hosted SearXNG instances without API keys.

dsh-web-tools does not provide shared API keys or proxy services. All requests originate directly from the local DSH host process to upstream APIs.

Installation & Updates

Install

dsh plugin --profile web add github:A3Boy/dsh-web-tools

Restart dsh web and navigate to:

Settings → Web Search

Update

dsh plugin --profile web update dsh-web-tools

The settings card checks GitHub Releases in the background and prompts when a new stable version is available.

Uninstall

dsh plugin --profile web remove dsh-web-tools

Provider Capabilities

Configuration & Feature Matrix

Quick Selection Guide

Exa is the default provider for new installations based on project P5 benchmark evaluation.

Routing & Resilience

1. Search Routing Policy

Choose how the runtime selects the initial search provider:

• Ordered (Default): Always starts with the first configured provider in the list.
• Round-Robin: Rotates the initial provider sequentially per query to distribute request load across multiple providers.
• Random: Randomly selects a provider as the starting point.

When the initial provider fails, the runtime always cascades through the remaining providers in order regardless of the routing policy.

2. Automatic Fallback Chain

Drag and drop providers in the settings page to customize the fallback order. Failover triggers on:

• Request timeouts (configurable per-attempt timeout, default 10s)
• Network errors or DNS lookup failures
• Upstream 5xx server errors
• 429 rate limits or exhausted credits (402, 432, 433)

3. 429 Retry-After Cooldown

When an upstream provider responds with HTTP 429 and a Retry-After header, it is placed in temporary cooldown. Subsequent queries skip this provider with zero network overhead until the cooldown expires.

4. Multi-Key Pooling & Isolation

Add multiple API keys to any hosted provider:

• Requests prioritize keys with the lowest active in-flight count.
• Authentication errors (401/403) mark that specific key unhealthy and immediately try alternative keys under the same provider before cascading to the next provider.
• Keys are masked in the UI to prevent credential exposure.

Search Mode

The web interface includes a session-level Search Mode toggle beside the chat input:

• Auto: The agent autonomously decides whether to use web retrieval tools based on context.
• Required: Enforces at least one web_search or web_fetch call before finalizing a response. If web retrieval fails entirely, the agent is instructed to state what could not be verified.

Interactive Test Search

The settings page provides an integrated test console to verify the live fallback chain:

• Inspect the selected winning provider
• View end-to-end latency and result count
• Review detailed attempt logs (success, timeout, authentication failure, rate limit)
• Preview retrieved titles, URLs, and snippet excerpts

Proxy Configuration

Built-in proxy resolution supports:

HTTP_PROXY / HTTPS_PROXY
Windows System Proxy settings
NO_PROXY

Loopback addresses (localhost, 127.0.0.1, ::1, *.local) automatically bypass proxies.

Security & Privacy

• Local Storage & Direct Transport: API keys remain in the local DSH credential vault. The host process communicates directly with provider endpoints without intermediate relays.
• Masked Credentials: Full API keys are never exposed back to the client UI. Authorization headers are sanitized in logs and test outputs.
• Self-Hosting: Supports air-gapped or private environments using a local SearXNG instance.

Development

# Install dependencies
pnpm install

# Run test suite
pnpm test

# Type checking
pnpm run typecheck

# Build bundle
pnpm run build

After making changes in src/, run pnpm run build to generate the lib/ artifacts. Refer to CONTRIBUTING.md for more details.

FAQ

If upgrading via file: or package store encounters caching, run pnpm install in your DSH profile directory:

cd ~/.dsh/profiles/web
pnpm install

For local plugin development, use link: instead of file: to ensure immediate updates.

Certain providers (e.g., Exa, Parallel) do not currently expose a lightweight public balance endpoint for individual API keys. Quota display is an informational helper and does not affect search functionality or failover.

License

MIT © A3Boy