Skip to content

feat: Add SearXNG as web search provider#534

Merged
afjcjsbx merged 8 commits intosipeed:mainfrom
truongvinht:feat/searxng
Mar 5, 2026
Merged

feat: Add SearXNG as web search provider#534
afjcjsbx merged 8 commits intosipeed:mainfrom
truongvinht:feat/searxng

Conversation

@truongvinht
Copy link
Contributor

@truongvinht truongvinht commented Feb 20, 2026

📝 Description

Adds SearXNG as a third web search backend alongside DuckDuckGo and Brave.

Motivation:

  • DuckDuckGo blocks datacenter IPs (Oracle Cloud, AWS, etc.), returning empty/blocked responses (138 bytes)
  • Brave Search eliminated their free API tier (Feb 12, 2026), now requiring $5/month minimum
  • SearXNG is self-hosted, free, and aggregates 70+ search engines with a native JSON API

Configurable via config.json. Fully backwards compatible — existing configs without the searxng block work unchanged.

🗣️ Type of Change

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 📖 Documentation update
  • ⚡ Code refactoring (no functional changes, no api changes)

🤖 AI Code Generation

  • 🤖 Fully AI-generated (100% AI, 0% Human)
  • 🛠️ Mostly AI-generated (AI draft, Human verified/modified)
  • 👨‍💻 Mostly Human-written (Human lead, AI assisted or none)

🔗 Related Issue

No existing issue. DuckDuckGo IP blocking and Brave free tier removal affect all users deploying PicoClaw on cloud VMs.

📚 Technical Context (Skip for Docs)

🧪 Test Environment

  • Hardware: Oracle Cloud E2.1.Micro (1 OCPU, 1GB RAM)
  • OS: Ubuntu 22.04
  • Model/Provider: DeepSeek V3
  • Channels: Discord

📸 Evidence (Optional)

Click to view Logs/Screenshots

Feb 20 13:56:05 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:05 [2026-02-20T13:56:05Z] [INFO] agent: LLM requested tool calls {tools=[web_search], count=1, iteration=1, agent_id=main}
Feb 20 13:56:05 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:05 [2026-02-20T13:56:05Z] [INFO] agent: Tool call: web_search({"count":10,"query":"OpenClaw AI agent framework news 2026"}) {iteration=1, agent_id=main, tool=web_search}
Feb 20 13:56:05 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:05 [2026-02-20T13:56:05Z] [INFO] tool: Tool execution started {tool=web_search, args=map[count:10 query:OpenClaw AI agent framework news 2026]}
Feb 20 13:56:06 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:06 [2026-02-20T13:56:06Z] [INFO] tool: Tool execution completed {tool=web_search, duration_ms=1163, result_length=3152}
Feb 20 13:56:20 instance-20260215-1129 picoclaw[2394]: 2026/02/20 13:56:20 [2026-02-20T13:56:20Z] [INFO] agent: Response: Based on the search results, here's the latest news about OpenClaw:....

☑️ Checklist

  • My code/docs follow the style of this project.
  • I have performed a self-review of my own changes.
  • I have updated the documentation accordingly.

truongvinht and others added 4 commits February 20, 2026 12:02
Implements SearXNG as a third web search provider to address Oracle Cloud
datacenter IP blocking issues and provide a cost-free, self-hosted alternative
to commercial search APIs.

Changes:
- Add SearXNGConfig struct with Enabled, BaseURL, and MaxResults fields
- Implement SearXNGSearchProvider with JSON API integration
- Update provider priority: Perplexity > Brave > SearXNG > DuckDuckGo
- Wire SearXNG configuration through agent tool registration
- Add default config values (disabled by default, empty BaseURL)

Benefits:
- Solves DuckDuckGo datacenter IP blocking (138 bytes redirect responses)
- Zero-cost alternative to Brave Search API ($5/1000 queries)
- Self-hosted solution with 70+ aggregated search engines
- Privacy-focused with no rate limits or API keys required
- Ideal for Oracle Cloud, GCP, AWS, and Azure VM deployments

The implementation follows the existing provider interface pattern and
maintains backward compatibility with all existing search providers.

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Update README to document the new SearXNG search provider option alongside
existing Brave, DuckDuckGo, and Perplexity providers.

Changes:
- Document provider priority order: Perplexity > Brave > SearXNG > DuckDuckGo
- Add SearXNG configuration examples in Quick Start and Full Config sections
- Expand "Get API Keys" section with all 4 search provider options
- Enhance troubleshooting section with detailed setup instructions for each provider
- Add SearXNG to API Key Comparison table (unlimited/self-hosted)

SearXNG benefits documented:
- Zero cost with no API fees or rate limits
- Privacy-focused self-hosted solution
- Aggregates 70+ search engines for comprehensive results
- Solves datacenter IP blocking issues on Oracle Cloud, GCP, AWS, Azure
- No API key required, just deploy and configure base URL

This documentation complements the code implementation in commit e7d8975.

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Update config.example.json to include SearXNG web search provider
configuration alongside existing Brave, DuckDuckGo, and Perplexity options.

This ensures users have a complete reference for all available search
providers when setting up their PicoClaw instance.

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
  Brave Search discontinued free tier on Feb 12, 2026.
  Updated all README references to reflect paid pricing.
  Emphasized SearXNG as free alternative.

  Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
@sky5454
Copy link
Contributor

sky5454 commented Feb 22, 2026

I think adding this will be convenient for users to aggregate their own information, but at the same time, we may also need to consider the hidden danger of config unlimited expansion. Maybe we can extract the tool or tool-web item of the configuration file and make it into a separate configuration file?

@IceWreck
Copy link

++ Having this would be great - lots of people using picoclaw would also be self hosting searxng.

@truongvinht
Copy link
Contributor Author

Update: fix merge conflict

@airpods69
Copy link

@truongvinht I did try this out with my locally hosted instance (built your branch)
but getting a 403 error :(

@airpods69
Copy link

okay nvm found it
https://stackoverflow.com/a/79056789

The problem is that by default the SearXNG seems to only support format=html. I don't know what is the best way to resolve this but you can get format=json working by editing /etc/searxng/settings.yml and adding json to the enabled search formats. Would be nice to have environment variable for this for the docker image.

Copy link

@nikolasdehor nikolasdehor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-motivated feature. DuckDuckGo blocking datacenter IPs and Brave removing their free tier are real problems for self-hosted deployments. SearXNG fills the gap nicely.

The implementation is clean and follows the existing search provider pattern. A few minor observations:

  1. New HTTP client per request: Same pattern as other providers in this file, but http.Client{Timeout: 10 * time.Second} should be created once and reused. Not a blocker since the other providers do the same thing.

  2. No proxy support: Unlike Brave and Perplexity providers, SearXNG does not use the opts.Proxy setting. Since SearXNG is typically self-hosted on the same network, this is usually fine, but for consistency it should either support proxy or document why it does not.

  3. README changes are extensive: The PR updates the README in multiple places with search provider comparisons and configuration examples. The content is accurate and helpful. The Brave Search description was updated from "Free tier available" to "Paid ($5/1000 queries)" which is consistent with the stated motivation.

  4. Priority order: SearXNG is placed after Brave but before Tavily in the priority chain. This makes sense since SearXNG is self-hosted (presumably more reliable than Tavily for the user) but does not require an API key like Brave.

  5. Perplexity config added to README: The PR also adds Perplexity configuration examples to the README sections where only Brave/DuckDuckGo/Tavily were listed. This is a good completeness fix but unrelated to SearXNG.

LGTM.

@afjcjsbx
Copy link
Collaborator

afjcjsbx commented Mar 2, 2026

LGTM

@afjcjsbx
Copy link
Collaborator

afjcjsbx commented Mar 3, 2026

could you fix lint pls?

truongvinht and others added 2 commits March 4, 2026 21:42
Resolve merge conflicts to keep both SearXNG and GLM Search
providers. Updated search priority order to:
Perplexity > Brave > SearXNG > Tavily > DuckDuckGo > GLM Search

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@truongvinht
Copy link
Contributor Author

could you fix lint pls?

I just fixed merge conflict and lint issue

@afjcjsbx
Copy link
Collaborator

afjcjsbx commented Mar 5, 2026

sorry, again 😞

@CLAassistant
Copy link

CLAassistant commented Mar 5, 2026

CLA assistant check
All committers have signed the CLA.

Resolved conflicts in 3 files:
- config/config.example.json: keep both searxng and glm_search configs
- pkg/agent/loop.go: adopt upstream's IsToolEnabled guard + keep searxng fields
- pkg/config/config.go: adopt upstream's ToolConfig embed + keep SearXNG field

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@truongvinht
Copy link
Contributor Author

@afjcjsbx dont worry just fixed conflict

@afjcjsbx afjcjsbx merged commit a3648ae into sipeed:main Mar 5, 2026
4 checks passed
@Orgmar
Copy link
Contributor

Orgmar commented Mar 6, 2026

@truongvinht Cảm ơn bạn đã thêm SearXNG làm backend tìm kiếm web! Phân tích vấn đề rất rõ ràng, DuckDuckGo chặn IP datacenter còn Brave đã bỏ gói miễn phí, SearXNG tự host được là giải pháp rất hợp lý.

Chúng mình đang xây dựng PicoClaw Dev Group trên Discord để các contributors trao đổi và hợp tác. Nếu bạn quan tâm, hãy gửi email đến [email protected] với tiêu đề [Join PicoClaw Dev Group] truongvinht, mình sẽ gửi link mời Discord cho bạn!

dj-oyu pushed a commit to dj-oyu/picoclaw that referenced this pull request Mar 8, 2026
feat: Add SearXNG as web search provider
fishtrees pushed a commit to fishtrees/picoclaw that referenced this pull request Mar 12, 2026
feat: Add SearXNG as web search provider
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants