Web Search

Multi-engine web search with advanced filtering, ranking, and result aggregation capabilities.

Search multiple search engines simultaneously with advanced filtering, ranking algorithms, and comprehensive result processing for research and data collection.

Features

Multi-engine search support (Google, Bing, DuckDuckGo, Academic)
Advanced query filtering and ranking
Result deduplication and relevance scoring
Content type and domain filtering
Date range and language restrictions

Connector Options

The node uses reusable connector configuration that applies to all search operations:

Parameter	Type	Required	Description
`apiKeys`	Object	Yes	API keys for search engines (google, bing, etc.)
`defaultEngine`	TEXT	No	Default search engine when not specified
`maxResults`	INT	No	Default maximum results per search (default: 50)
`rateLimit`	INT	No	Requests per minute limit (default: 60)

Methods

webSearch

Execute multi-engine web searches with advanced filtering and ranking.

Parameter	Type	Required	Description
`query`	TEXT	Yes	Search query string
`engines`	Array	No	Search engines to use: google, bing, duckduckgo, academic
`maxResults`	INT	No	Maximum number of results to return
`filters`	Object	No	Search filters and restrictions
`ranking`	TEXT	No	Result ranking method: relevance, date, authority

{
  "query": "machine learning python tutorials",
  "engines": ["google", "bing", "duckduckgo"],
  "maxResults": 50,
  "filters": {
    "dateRange": "past_year",
    "contentType": ["article", "tutorial"],
    "language": "en",
    "domain": ["github.com", "stackoverflow.com"]
  },
  "ranking": "relevance"
}

Output:

results (Array) - Search results with metadata
totalResults (Number) - Total number of results found
searchTime (Number) - Search execution time in milliseconds
engines (Array) - Engines used and their response status
query (Object) - Processed query information

academicSearch

Search academic databases and research repositories for scholarly content.

Parameter	Type	Required	Description
`query`	TEXT	Yes	Academic search query
`publicationType`	Array	No	Publication types: journal, conference, book, thesis
`dateRange`	TEXT	No	Publication date range
`citationThreshold`	INT	No	Minimum citation count
`subject`	Array	No	Subject areas to search within

{
  "query": "natural language processing transformers",
  "publicationType": ["journal", "conference"],
  "dateRange": "past_5_years",
  "citationThreshold": 10,
  "subject": ["computer_science", "artificial_intelligence"]
}

newsSearch

Search news sources and current events with real-time filtering.

Parameter	Type	Required	Description
`query`	TEXT	Yes	News search query
`sources`	Array	No	Specific news sources to search
`sentiment`	TEXT	No	Filter by sentiment: positive, negative, neutral
`freshness`	TEXT	No	Content freshness: hour, day, week, month

{
  "query": "sustainable energy technology",
  "sources": ["reuters", "bloomberg", "techcrunch"],
  "sentiment": "neutral",
  "freshness": "week"
}

Content Filtering

Filter	Type	Description	Example Values
`dateRange`	TEXT	Time period for results	"past_day", "past_week", "past_month", "past_year"
`contentType`	Array	Type of content to find	["article", "blog", "news", "tutorial", "research"]
`language`	TEXT	Content language	"en", "es", "fr", "de", "zh"
`domain`	Array	Specific domains to include	["github.com", "stackoverflow.com", "arxiv.org"]
`excludeDomains`	Array	Domains to exclude	["pinterest.com", "facebook.com"]

Quality Filters

Filter	Type	Description	Example Values
`minWordCount`	INT	Minimum content length	500
`authorityScore`	FLOAT	Domain authority threshold	0.7
`readabilityLevel`	TEXT	Content reading level	"basic", "intermediate", "advanced"
`contentQuality`	TEXT	Quality assessment	"high", "medium", "any"

Result Processing

Ranking Algorithms

Relevance Ranking:

Query term frequency and positioning
Title and header matching weights
Domain authority scoring
Content freshness factors
User engagement signals

Authority Ranking:

Domain reputation scores
Backlink analysis
Content creator credibility
Publication venue quality
Citation counts (academic content)

Date Ranking:

Publication or update timestamps
Content freshness scoring
Trend relevance analysis
Temporal query matching

Deduplication

URL normalization and comparison
Content similarity detection
Title and description matching
Domain clustering analysis
Canonical URL resolution

Performance Optimization

Caching Strategy

{
  "query": "artificial intelligence trends",
  "engines": ["google", "bing"],
  "caching": {
    "enabled": true,
    "ttl": 3600,
    "keyFactors": ["query", "filters", "engines"],
    "compression": true
  }
}

Batch Operations

{
  "batchSearch": {
    "queries": [
      "machine learning applications",
      "deep learning frameworks", 
      "neural network architectures"
    ],
    "engines": ["google", "academic"],
    "maxResults": 30,
    "parallel": true,
    "rateLimiting": {
      "requestsPerMinute": 30,
      "delayBetweenRequests": 2000
    }
  }
}

Error Handling

Common Error Responses

Error Type	Cause	Resolution
`QUOTA_EXCEEDED`	API limit reached	Wait for quota reset or use different engine
`INVALID_QUERY`	Malformed search query	Check query syntax and special characters
`ENGINE_UNAVAILABLE`	Search engine down	Use alternative engines or retry later
`RATE_LIMITED`	Too many requests	Implement request throttling

Error Response Format

{
  "success": false,
  "error": {
    "type": "QUOTA_EXCEEDED",
    "message": "Daily search quota exceeded for Google Search API",
    "engine": "google",
    "retryAfter": 3600,
    "suggestions": [
      "Use alternative search engines",
      "Wait for quota reset",
      "Upgrade API plan"
    ]
  }
}

Usage Examples

Basic Web Search

{
  "query": "Python web scraping tutorial",
  "engines": ["google", "bing"],
  "maxResults": 20,
  "filters": {
    "contentType": ["tutorial", "documentation"],
    "language": "en"
  }
}

Academic Research Search

{
  "query": "quantum computing algorithms",
  "engines": ["academic"],
  "maxResults": 25,
  "filters": {
    "publicationType": ["journal", "conference"],
    "dateRange": "past_3_years",
    "citationThreshold": 5
  },
  "ranking": "authority"
}

Competitive Intelligence

{
  "query": "Company XYZ product launches 2024",
  "engines": ["google", "bing"],
  "maxResults": 100,
  "filters": {
    "dateRange": "past_year",
    "contentType": ["news", "press_release"],
    "domain": ["techcrunch.com", "venturebeat.com", "bloomberg.com"]
  },
  "ranking": "date"
}