Skip to content

Feature Request: Optional JSON search-API engine as fallback provider alongside SearXNG instances #234

Description

@gefsikatsinelou

Hi, thanks for maintaining this — I've been running mcp-searxng as the search tool for a couple of research agents and the instance failover has saved me more than once.

The pain point

Public SearXNG instances are the weak link. Even with the cooldown logic in searxng-instances.ts, I regularly hit runs where every configured instance is either rate-limited or returns 403 Access blocked (bot detection). I saw the Flaresolverr/Byparr request in #166 — same experience here. Standing up Flaresolverr works, but it's another service to operate, and for Google results specifically it's a lot of machinery just to get ten organic links back.

Suggestion

Would you consider an optional API-based engine as an additional provider? Something like a SEARCH_API_URL env var that gets consulted as a fallback when all SearXNG instances are in cooldown, or as an extra source in fanout mode. SerpBase (https://serpbase.dev) is one option — it returns Google organic results as JSON (title, link, snippet, position) that maps almost 1:1 onto the existing result type, so the integration surface is small. If no key is configured, the engine is skipped entirely and SearXNG-only users are unaffected.

Why it fits

  • The server already models providers with failover + cooldown; an API engine slots into that same "try in order" model as a last-resort provider that doesn't get 403-blocked.
  • It directly addresses the Flaresolverr burden from Flaresolverr/Byparr support #166 for the Google case — no extra service to run for the most common blocking scenario.
  • Privacy stays intact: SearXNG remains the default, the API engine is strictly opt-in via env var and never enabled by default.

Happy to draft a PR if there's interest — the result mapping and env handling are both small enough to keep the diff tight.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions