Web Search MCP Server
A production-ready Model Context Protocol (MCP) server that provides web search capabilities through SearXNG. Designed for seamless integration with LLM agents, featuring HTTP/SSE transport and LLM-optimized response formatting.
Features
- Multi-Engine Search - Aggregates results from Brave, DuckDuckGo, Startpage, and more
- LLM-Optimized - Three specialized tools with formatted responses for agent consumption
- Knowledge Extraction - Automatic extraction of structured facts from Wikidata
- Real-Time Streaming - MCP over HTTP/SSE for live agent integration
- Docker Ready - Fully containerized with docker-compose
- Production Features - Health checks, metrics, structured logging, request tracking
Architecture```
┌─────────────────────┐
│ LLM Agent │
│ (MCP Client) │
└──────────┬──────────┘
│ MCP over HTTP/SSE
↓
┌─────────────────────┐
│ Web Search MCP │ Port 8003
│ Server │
│ - 3 Search Tools │
│ - REST API │
│ - Metrics │
└──────────┬──────────┘
│ HTTP
↓
┌─────────────────────┐
│ SearXNG │ Port 8888
│ (Meta-Search) │
│ - Brave │
│ - DuckDuckGo │
│ - Startpage │
│ - Wikipedia │
└─────────────────────┘
## Quick Start
### Prerequisites
- Docker & Docker Compose
- (Optional) Python 3.11+ for local development
### 1. Clone and Start```bash
git clone https://github.com/slinusc/web-search-mcp-server.git
cd web-search-mcp-server
docker compose up -d
This starts:
- SearXNG on port 8888 (search backend)
- MCP Server on port 8003 (agent interface)
2. Test the API```bash
Quick search (LLM-optimized)
curl -X POST http://localhost:8003/api/quick_search \
-H "Content-Type: application/json" \
-d '{"query": "latest developments in AI"}'
Structured search (with facts)
curl -X POST http://localhost:8003/api/structured_search \
-H "Content-Type: application/json" \
-d '{"query": "climate change 2024"}'
Health check
curl http://localhost:8003/healthz
### 3. Connect Your MCP Client```json
{
"mcpServers": {
"websearch": {
"url": "http://localhost:8003/sse"
}
}
}
MCP Tools
1. web_search
Comprehensive search with full metadata```python
{
"query": "machine learning frameworks",
"categories": "general", # or "news", "images", "videos"
"pageno": 1
}
Returns: Full SearXNG response with all engines, suggestions, and infoboxes.
### 2. `quick_search`
**Fast, LLM-optimized results**```python
{
"query": "who invented the internet",
"categories": "general"
}
Returns: Top 10 ranked results with knowledge base, relevance scores, and source attribution.
3. structured_search
Facts-focused with context```python
{
"query": "Python programming language",
"categories": "general"
}
Returns: Ranked results + Wikidata attributes + context snippets + suggestions.
## Response Format
### Quick Search Example```json
{
"ok": true,
"request_id": "a1b2c3d4e5f6",
"query": "Python programming",
"total_results": 28,
"knowledge_base": {
"entity": "Python (programming language)",
"description": "Python is a high-level, interpreted programming language...",
"attributes": {
"Developer": "Python Software Foundation",
"First appeared": "1991",
"Influenced by": "ABC, Modula-3, C"
},
"image": "https://upload.wikimedia.org/..."
},
"top_results": [
{
"rank": 1,
"title": "Python.org",
"url": "https://www.python.org/",
"snippet": "Python is a programming language that lets you work quickly...",
"score": 9.0,
"engines": ["brave", "duckduckgo", "startpage"]
}
],
"engines_used": ["brave", "duckduckgo", "startpage", "wikipedia"]
}
REST API Reference
| Endpoint | Method | Description |
|---|---|---|
/sse |
GET | MCP SSE stream (for agents) |
/messages |
POST | MCP client messages |
/api/search |
POST | General web search |
/api/quick_search |
POST | Quick LLM-optimized search |
/api/structured_search |
POST | Structured facts search |
/healthz |
GET | Health check |
/metrics |
GET | Performance metrics |
/tools |
GET | List available MCP tools |
Configuration
Environment Variables```bash
docker-compose.yml or .env
SEARXNG_URL=http://searxng:8080 # SearXNG backend URL
SEARXNG_TIMEOUT=10.0 # Request timeout (seconds)
### SearXNG Configuration
Edit `config/settings.yml` to:
- Enable/disable search engines
- Configure result formats
- Set rate limiting
- Customize search behavior
Key settings:```yaml
search:
formats:
- html
- json # Required for MCP server
server:
limiter: false # Adjust for production
Development
Local Setup```bash
Create virtual environment
python -m venv venv
source venv/bin/activate # or venv\Scripts\activate on Windows
Install dependencies
pip install -r requirements.txt
Run server
python server.py
Server runs on `http://localhost:8003`
### Testing```bash
# Test quick search
curl -X POST http://localhost:8003/api/quick_search \
-H "Content-Type: application/json" \
-d '{"query": "test query"}'
# Check metrics
curl http://localhost:8003/metrics
# List available tools
curl http://localhost:8003/tools | jq
Docker
Build```bash
docker build -t mcp-server-websearch .
### Run Standalone```bash
docker run -d \
--name mcp-websearch \
-p 8003:8003 \
-e SEARXNG_URL=http://searxng:8080 \
mcp-server-websearch
Compose (Recommended)```bash
Start all services
docker compose up -d
View logs
docker compose logs -f mcp-websearch
Stop services
docker compose down
## Monitoring
### Health Check```bash
curl http://localhost:8003/healthz
# Response: ok
Metrics```bash
curl http://localhost:8003/metrics
Response:```json
{
"ok": true,
"metrics": {
"api_search_calls": 42,
"api_quick_search_calls": 156,
"api_structured_search_calls": 23,
"mcp_call_tool_calls": 221,
"searxng_requests": 221,
"errors_total": 2
}
}
Agent Integration
Generic MCP Client
The server exposes three MCP tools that are automatically available to any MCP-compatible agent:
web_search- For comprehensive search with full detailsquick_search- For fast answers with top resultsstructured_search- For research with structured facts
LangChain Integration```python
from langchain.tools import Tool
Wrap the MCP tools
tools = [
Tool(
name="web_search",
func=call_mcp_tool, # Your MCP client wrapper
description="Search the web for current information"
)
]
## Use Cases
- **Question Answering** - Quick answers with source attribution
- **Research Assistants** - Deep research with structured facts
- **RAG Systems** - Retrieve current information for context
- **News Monitoring** - Track recent events and developments
- **Fact Checking** - Verify information across multiple sources
## Security Notes
- **Rate Limiting**: Configure SearXNG's `limiter.toml` for production
- **API Keys**: Not required (SearXNG is self-hosted)
- **CORS**: Currently allows all origins - restrict in production
- **Network**: Use Docker networks to isolate services
## Troubleshooting
### SearXNG returns 403 Forbidden```bash
# Check if JSON format is enabled in config/settings.yml
docker exec searxng grep -A 3 "formats:" /etc/searxng/settings.yml
# Should show:
# formats:
# - html
# - json
Connection refused errors```bash
Ensure services are running
docker compose ps
Check logs
docker compose logs searxng
docker compose logs mcp-websearch
### No search results```bash
# Test SearXNG directly
curl -X POST http://localhost:8888/search \
-H "X-Forwarded-For: 127.0.0.1" \
--data "q=test&format=json"
License
MIT License - see LICENSE file for details.
Acknowledgments
- SearXNG - Privacy-respecting metasearch engine
- Model Context Protocol - Anthropic's MCP specification
- Starlette - Lightweight ASGI framework
Contributing
Contributions welcome! Please:
- Fork the repository
- Create a feature branch
- Commit your changes
- Push to the branch
- Open a Pull Request
Support
- Issues: GitHub Issues
- Discussions: GitHub Discussions
