feat: Add SearXNG as web search provider#534
Conversation
Implements SearXNG as a third web search provider to address Oracle Cloud datacenter IP blocking issues and provide a cost-free, self-hosted alternative to commercial search APIs. Changes: - Add SearXNGConfig struct with Enabled, BaseURL, and MaxResults fields - Implement SearXNGSearchProvider with JSON API integration - Update provider priority: Perplexity > Brave > SearXNG > DuckDuckGo - Wire SearXNG configuration through agent tool registration - Add default config values (disabled by default, empty BaseURL) Benefits: - Solves DuckDuckGo datacenter IP blocking (138 bytes redirect responses) - Zero-cost alternative to Brave Search API ($5/1000 queries) - Self-hosted solution with 70+ aggregated search engines - Privacy-focused with no rate limits or API keys required - Ideal for Oracle Cloud, GCP, AWS, and Azure VM deployments The implementation follows the existing provider interface pattern and maintains backward compatibility with all existing search providers. Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Update README to document the new SearXNG search provider option alongside existing Brave, DuckDuckGo, and Perplexity providers. Changes: - Document provider priority order: Perplexity > Brave > SearXNG > DuckDuckGo - Add SearXNG configuration examples in Quick Start and Full Config sections - Expand "Get API Keys" section with all 4 search provider options - Enhance troubleshooting section with detailed setup instructions for each provider - Add SearXNG to API Key Comparison table (unlimited/self-hosted) SearXNG benefits documented: - Zero cost with no API fees or rate limits - Privacy-focused self-hosted solution - Aggregates 70+ search engines for comprehensive results - Solves datacenter IP blocking issues on Oracle Cloud, GCP, AWS, Azure - No API key required, just deploy and configure base URL This documentation complements the code implementation in commit e7d8975. Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Update config.example.json to include SearXNG web search provider configuration alongside existing Brave, DuckDuckGo, and Perplexity options. This ensures users have a complete reference for all available search providers when setting up their PicoClaw instance. Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Brave Search discontinued free tier on Feb 12, 2026. Updated all README references to reflect paid pricing. Emphasized SearXNG as free alternative. Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
|
I think adding this will be convenient for users to aggregate their own information, but at the same time, we may also need to consider the hidden danger of config unlimited expansion. Maybe we can extract the tool or tool-web item of the configuration file and make it into a separate configuration file? |
|
++ Having this would be great - lots of people using picoclaw would also be self hosting searxng. |
|
Update: fix merge conflict |
|
@truongvinht I did try this out with my locally hosted instance (built your branch) |
|
okay nvm found it
|
nikolasdehor
left a comment
There was a problem hiding this comment.
Well-motivated feature. DuckDuckGo blocking datacenter IPs and Brave removing their free tier are real problems for self-hosted deployments. SearXNG fills the gap nicely.
The implementation is clean and follows the existing search provider pattern. A few minor observations:
-
New HTTP client per request: Same pattern as other providers in this file, but
http.Client{Timeout: 10 * time.Second}should be created once and reused. Not a blocker since the other providers do the same thing. -
No proxy support: Unlike Brave and Perplexity providers, SearXNG does not use the
opts.Proxysetting. Since SearXNG is typically self-hosted on the same network, this is usually fine, but for consistency it should either support proxy or document why it does not. -
README changes are extensive: The PR updates the README in multiple places with search provider comparisons and configuration examples. The content is accurate and helpful. The Brave Search description was updated from "Free tier available" to "Paid ($5/1000 queries)" which is consistent with the stated motivation.
-
Priority order: SearXNG is placed after Brave but before Tavily in the priority chain. This makes sense since SearXNG is self-hosted (presumably more reliable than Tavily for the user) but does not require an API key like Brave.
-
Perplexity config added to README: The PR also adds Perplexity configuration examples to the README sections where only Brave/DuckDuckGo/Tavily were listed. This is a good completeness fix but unrelated to SearXNG.
LGTM.
|
LGTM |
|
could you fix lint pls? |
Resolve merge conflicts to keep both SearXNG and GLM Search providers. Updated search priority order to: Perplexity > Brave > SearXNG > Tavily > DuckDuckGo > GLM Search Co-Authored-By: Claude Opus 4.6 <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>
I just fixed merge conflict and lint issue |
|
sorry, again 😞 |
Resolved conflicts in 3 files: - config/config.example.json: keep both searxng and glm_search configs - pkg/agent/loop.go: adopt upstream's IsToolEnabled guard + keep searxng fields - pkg/config/config.go: adopt upstream's ToolConfig embed + keep SearXNG field Co-Authored-By: Claude Opus 4.6 <[email protected]>
|
@afjcjsbx dont worry just fixed conflict |
|
@truongvinht Cảm ơn bạn đã thêm SearXNG làm backend tìm kiếm web! Phân tích vấn đề rất rõ ràng, DuckDuckGo chặn IP datacenter còn Brave đã bỏ gói miễn phí, SearXNG tự host được là giải pháp rất hợp lý. Chúng mình đang xây dựng PicoClaw Dev Group trên Discord để các contributors trao đổi và hợp tác. Nếu bạn quan tâm, hãy gửi email đến |
feat: Add SearXNG as web search provider
feat: Add SearXNG as web search provider
📝 Description
Adds SearXNG as a third web search backend alongside DuckDuckGo and Brave.
Motivation:
Configurable via
config.json. Fully backwards compatible — existing configs without thesearxngblock work unchanged.🗣️ Type of Change
🤖 AI Code Generation
🔗 Related Issue
No existing issue. DuckDuckGo IP blocking and Brave free tier removal affect all users deploying PicoClaw on cloud VMs.
📚 Technical Context (Skip for Docs)
/search?q=query&format=json), making it the only free and reliable web search option for datacenter-hosted PicoClaw instances.🧪 Test Environment
📸 Evidence (Optional)
Click to view Logs/Screenshots
Feb 20 13:56:05 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:05 [2026-02-20T13:56:05Z] [INFO] agent: LLM requested tool calls {tools=[web_search], count=1, iteration=1, agent_id=main}
Feb 20 13:56:05 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:05 [2026-02-20T13:56:05Z] [INFO] agent: Tool call: web_search({"count":10,"query":"OpenClaw AI agent framework news 2026"}) {iteration=1, agent_id=main, tool=web_search}
Feb 20 13:56:05 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:05 [2026-02-20T13:56:05Z] [INFO] tool: Tool execution started {tool=web_search, args=map[count:10 query:OpenClaw AI agent framework news 2026]}
Feb 20 13:56:06 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:06 [2026-02-20T13:56:06Z] [INFO] tool: Tool execution completed {tool=web_search, duration_ms=1163, result_length=3152}
Feb 20 13:56:20 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:20 [2026-02-20T13:56:20Z] [INFO] agent: Response: Based on the search results, here's the latest news about OpenClaw:....
☑️ Checklist