multi-search-engine

A unified CLI for querying several web search engines in parallel and returning a normalized result list. Built on httpx and beautifulsoup4 (both already in OpenSquilla default dependencies, so no extra install beyond pip install opensquilla).

Use cases

• Building a deep-research round with diverse engine coverage
• Fact-check a claim against >1 engine
• Compare what Bing returns vs DuckDuckGo for the same query
• Search Chinese-language sources via Baidu/Sogou/360 alongside global engines

Limitations

• A single engine sufficient → call its API directly instead
• Need headless-browser DOM rendering → this skill is HTTP-only

Quick start

python {baseDir}/scripts/search.py \
    --query "openclaw skill registry" \
    --engines duckduckgo,brave \
    --limit 10 \
    --json

Output:

{
  "query": "...",
  "results": [
    {
      "engine": "duckduckgo",
      "title": "...",
      "url": "https://...",
      "snippet": "...",
      "rank": 1
    }
  ],
  "errors": [
    {"engine": "brave", "reason": "BRAVE_SEARCH_API_KEY/BRAVE_API_KEY not set; skipping"}
  ]
}

Engines

The script never errors out when an API-key engine's key is missing — it records a per-engine errors entry and continues with the rest. Pass --strict to fail fast when any requested engine is unavailable.

Routing tips

The host should pick engines by language and availability:

• English queries → duckduckgo, brave, bing (one or two for triangulation)
• Chinese queries → baidu plus optionally sogou for cross-check
• Time-sensitive (last 24h) → brave (recency filter) or tavily
• Long-tail academic → fall back to direct arXiv / Google Scholar; this skill targets general web search

engines.md has the full per-engine guidance.

Boundaries

• HTTP-only. JS-rendered pages will not be readable; use a headless-browser skill if needed.
• Scraping engines (DuckDuckGo, Bing, Baidu, Sogou, 360) are best-effort — HTML structure changes break them. The script logs parse failures individually and keeps the run going.
• Rate limiting is not handled inside the script. Calling the same engine 10x/sec from a loop will get blocked. Add jitter and back-off in the caller.
• Captcha-protected results are not bypassed. If an engine returns a challenge page, the parser will return zero results for that engine and log a warning.