mirror of
https://github.com/Heretek-AI/opencode-searxng-plugin.git
synced 2026-11-07 10:56:03 -10:00
feat: SearXNG websearch provider + searxng tool for opencode v2
- Effect API provider (id searxng) with user-configured baseUrl, no forced default - Companion searxng tool with per-query categories/engines/language/time_range/safesearch/pageno - SearXNG JSON mapping with tolerant date parsing, 25s timeout, bearer auth passthrough - Publish-ready as @heretek-ai/opencode-searxng (public, provenance)
This commit is contained in:
commit
c60dcda4b9
14 files changed
+4660
No files matched your search
@@ -0,0 +1,24 @@
|
||||
name: publish
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 22
|
||||
registry-url: "https://registry.npmjs.org"
|
||||
- run: npm ci
|
||||
- run: npm run typecheck
|
||||
- run: npm run build
|
||||
- run: npm publish --provenance --access public
|
||||
@@ -0,0 +1,4 @@
|
||||
dist/
|
||||
node_modules/
|
||||
*.tsbuildinfo
|
||||
.npm/
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 heretek-ai
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,96 @@
|
||||
# @heretek-ai/opencode-searxng
|
||||
|
||||
SearXNG web search provider + `searxng` tool for opencode v2. Self-hosted, no API key.
|
||||
|
||||
Registers:
|
||||
|
||||
- `websearch` provider `searxng` — works with the built-in `websearch` tool once selected.
|
||||
- `searxng` tool — same backend with explicit per-call `categories`, `engines`, `language`, `time_range`, `safesearch`, `pageno`, `maxResults`.
|
||||
|
||||
Query strings are forwarded verbatim, so SearXNG syntax keeps working: `!wikipedia !images :de effect hooks`.
|
||||
|
||||
## Requirements
|
||||
|
||||
- opencode v2
|
||||
- A reachable SearXNG instance with JSON enabled (`search.formats` must include `json` in `settings.yml`, otherwise `/search?format=json` returns 403).
|
||||
- Self-host recommended. Public instances often disable `format=json`, rate-limit, or bot-block.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
opencode plugin add @heretek-ai/opencode-searxng
|
||||
```
|
||||
|
||||
Or in `opencode.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"plugins": [
|
||||
{
|
||||
"package": "@heretek-ai/opencode-searxng",
|
||||
"options": {
|
||||
"baseUrl": "http://localhost:8888",
|
||||
"categories": "general",
|
||||
"language": "all",
|
||||
"safesearch": 1,
|
||||
"maxResults": 8
|
||||
}
|
||||
}
|
||||
],
|
||||
"websearch": { "provider": "searxng" }
|
||||
}
|
||||
```
|
||||
|
||||
See `opencode.example.json` for all options.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Option | Env | Default | Notes |
|
||||
|---|---|---|---|
|
||||
| `baseUrl` (required) | `SEARXNG_BASE_URL` | — | e.g. `http://localhost:8888`. Trailing slash trimmed. Must be http(s). |
|
||||
| `categories` | `SEARXNG_CATEGORIES` | `general` | Default `categories` param. |
|
||||
| `engines` | `SEARXNG_ENGINES` | unset | e.g. `wikipedia,duckduckgo`. Unset = server default. |
|
||||
| `language` | `SEARXNG_LANGUAGE` | `all` | e.g. `en`, `de`. |
|
||||
| `safesearch` | `SEARXNG_SAFESEARCH` | `1` | `0\|1\|2`. |
|
||||
| `timeRange` | `SEARXNG_TIME_RANGE` | unset | `day\|week\|month\|year`. |
|
||||
| `maxResults` | `SEARXNG_MAX_RESULTS` | `8` | Clamped 1–20. |
|
||||
| `timeoutMs` | `SEARXNG_TIMEOUT_MS` | `25000` | Clamped 1000–120000. |
|
||||
| `bearerToken` | `SEARXNG_BEARER_TOKEN` | unset | For reverse-proxy fronted instances (`Authorization: Bearer`). |
|
||||
|
||||
No default provider is forced. Opt in with `"websearch": { "provider": "searxng" }`.
|
||||
|
||||
## Usage
|
||||
|
||||
Standard tool (uses plugin defaults):
|
||||
|
||||
```
|
||||
websearch(query="NixOS SearXNG settings.yml search.formats")
|
||||
```
|
||||
|
||||
Structured tool (per-call overrides):
|
||||
|
||||
```
|
||||
searxng(query="effect hooks", engines="wikipedia,duckduckgo", language="en", maxResults=5)
|
||||
searxng(query="!images :de fennec fox", categories="images")
|
||||
```
|
||||
|
||||
## SearXNG server notes
|
||||
|
||||
- `GET {base}/search?q=...&format=json` must return `{"results": [...]}`. Test with:
|
||||
```bash
|
||||
curl "$SEARXNG_BASE_URL/search?q=test&format=json" | jq .results[0]
|
||||
```
|
||||
- `403` almost always means `json` not in `search.formats`.
|
||||
- `429` means server limiter. `401` means proxy auth.
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run typecheck
|
||||
npm run build
|
||||
npm test
|
||||
npm publish --dry-run
|
||||
```
|
||||
|
||||
Publish is via npm Trusted Publishing (see `.github/workflows/publish.yml`). First manual publish needs `npm publish --access public`.
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"plugins": [
|
||||
{
|
||||
"package": "@heretek-ai/opencode-searxng",
|
||||
"options": {
|
||||
"baseUrl": "http://localhost:8888",
|
||||
"categories": "general",
|
||||
"engines": "",
|
||||
"language": "all",
|
||||
"safesearch": 1,
|
||||
"maxResults": 8,
|
||||
"timeoutMs": 25000
|
||||
}
|
||||
}
|
||||
],
|
||||
"websearch": {
|
||||
"provider": "searxng"
|
||||
}
|
||||
}
|
||||
Generated
+3891
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/package.json",
|
||||
"name": "@heretek-ai/opencode-searxng",
|
||||
"version": "0.1.0",
|
||||
"description": "SearXNG web search provider + searxng tool for opencode v2 (self-hosted, no API key)",
|
||||
"type": "module",
|
||||
"license": "MIT",
|
||||
"author": "heretek-ai",
|
||||
"homepage": "https://github.com/Heretek-AI/opencode-searxng-plugin#readme",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/Heretek-AI/opencode-searxng-plugin.git"
|
||||
},
|
||||
"bugs": {
|
||||
"url": "https://github.com/Heretek-AI/opencode-searxng-plugin/issues"
|
||||
},
|
||||
"keywords": [
|
||||
"opencode",
|
||||
"opencode-plugin",
|
||||
"websearch",
|
||||
"searxng",
|
||||
"search",
|
||||
"self-hosted",
|
||||
"heretek-ai"
|
||||
],
|
||||
"main": "./dist/index.js",
|
||||
"types": "./dist/index.d.ts",
|
||||
"exports": {
|
||||
".": "./dist/index.js",
|
||||
"./server": "./dist/server.js"
|
||||
},
|
||||
"files": [
|
||||
"dist",
|
||||
"README.md",
|
||||
"LICENSE"
|
||||
],
|
||||
"publishConfig": {
|
||||
"access": "public",
|
||||
"provenance": true
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=22"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc -p tsconfig.json",
|
||||
"typecheck": "tsc --noEmit -p tsconfig.json",
|
||||
"test": "npm run build && node --test test/",
|
||||
"prepublishOnly": "npm run build"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"effect": "^4.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@opencode/plugin": "^2.0.30",
|
||||
"@opencode/schema": "^2.0.30",
|
||||
"@types/node": "^22.7.0",
|
||||
"effect": "4.0.1",
|
||||
"typescript": "^5.6.3"
|
||||
}
|
||||
}
|
||||
+127
@@ -0,0 +1,127 @@
|
||||
import { Schema } from "effect"
|
||||
|
||||
/** Plugin options as supplied via opencode.json `plugins: [{package, options}]`. */
|
||||
export interface RawOptions {
|
||||
readonly baseUrl?: unknown
|
||||
readonly categories?: unknown
|
||||
readonly engines?: unknown
|
||||
readonly language?: unknown
|
||||
readonly safesearch?: unknown
|
||||
readonly timeRange?: unknown
|
||||
readonly maxResults?: unknown
|
||||
readonly timeoutMs?: unknown
|
||||
readonly bearerToken?: unknown
|
||||
}
|
||||
|
||||
export interface ResolvedConfig {
|
||||
readonly baseUrl: string
|
||||
readonly categories: string
|
||||
readonly engines: string | undefined
|
||||
readonly language: string
|
||||
readonly safesearch: 0 | 1 | 2
|
||||
readonly timeRange: string | undefined
|
||||
readonly maxResults: number
|
||||
readonly timeoutMs: number
|
||||
readonly bearerToken: string | undefined
|
||||
}
|
||||
|
||||
const TIME_RANGES = new Set(["", "day", "week", "month", "year"])
|
||||
|
||||
function env(name: string): string | undefined {
|
||||
const value = process.env[name]?.trim()
|
||||
return value ? value : undefined
|
||||
}
|
||||
|
||||
function asString(value: unknown): string | undefined {
|
||||
return typeof value === "string" && value.trim() !== "" ? value.trim() : undefined
|
||||
}
|
||||
|
||||
function asInt(value: unknown): number | undefined {
|
||||
if (typeof value === "number" && Number.isInteger(value)) return value
|
||||
if (typeof value === "string" && value.trim() !== "") {
|
||||
const parsed = Number(value)
|
||||
if (Number.isInteger(parsed)) return parsed
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
export function normalizeBaseUrl(raw: string): string {
|
||||
const trimmed = raw.trim().replace(/\/+$/, "")
|
||||
let url: URL
|
||||
try {
|
||||
url = new URL(trimmed)
|
||||
} catch {
|
||||
throw new Error(`Invalid SearXNG baseUrl: ${JSON.stringify(raw)} (expected http(s)://host[:port][/subpath])`)
|
||||
}
|
||||
if (url.protocol !== "http:" && url.protocol !== "https:") {
|
||||
throw new Error(`Invalid SearXNG baseUrl protocol: ${url.protocol} (expected http: or https:)`)
|
||||
}
|
||||
return trimmed
|
||||
}
|
||||
|
||||
export const ConfigError = (message: string) => new Error(`[opencode-searxng] ${message}`)
|
||||
|
||||
/** Resolve plugin options + env. Throws on invalid required config. */
|
||||
export function resolveConfig(options: Record<string, unknown> = {}): ResolvedConfig {
|
||||
const raw = options as RawOptions
|
||||
|
||||
const baseUrlRaw =
|
||||
asString(raw.baseUrl) ?? env("SEARXNG_BASE_URL") ?? asString((options as { url?: unknown }).url)
|
||||
if (!baseUrlRaw) {
|
||||
throw ConfigError(
|
||||
`missing baseUrl. Set plugin options { "baseUrl": "http://localhost:8888" } or env SEARXNG_BASE_URL.`,
|
||||
)
|
||||
}
|
||||
const baseUrl = normalizeBaseUrl(baseUrlRaw)
|
||||
|
||||
const categories = asString(raw.categories) ?? env("SEARXNG_CATEGORIES") ?? "general"
|
||||
const engines = asString(raw.engines) ?? env("SEARXNG_ENGINES") ?? undefined
|
||||
const language = asString(raw.language) ?? env("SEARXNG_LANGUAGE") ?? "all"
|
||||
|
||||
const safesearchRaw = asInt(raw.safesearch) ?? (env("SEARXNG_SAFESEARCH") ? asInt(env("SEARXNG_SAFESEARCH")) : undefined) ?? 1
|
||||
const safesearch = safesearchRaw === 0 || safesearchRaw === 1 || safesearchRaw === 2 ? safesearchRaw : 1
|
||||
|
||||
const timeRangeRaw = asString(raw.timeRange) ?? env("SEARXNG_TIME_RANGE") ?? undefined
|
||||
if (timeRangeRaw !== undefined && !TIME_RANGES.has(timeRangeRaw)) {
|
||||
throw ConfigError(`invalid timeRange ${JSON.stringify(timeRangeRaw)} (expected one of: day, week, month, year)`)
|
||||
}
|
||||
const timeRange = timeRangeRaw === "" ? undefined : timeRangeRaw
|
||||
|
||||
const maxResultsRaw =
|
||||
asInt(raw.maxResults) ?? (env("SEARXNG_MAX_RESULTS") ? asInt(env("SEARXNG_MAX_RESULTS")) : undefined) ?? 8
|
||||
const maxResults = Math.min(20, Math.max(1, maxResultsRaw))
|
||||
|
||||
const timeoutRaw =
|
||||
asInt(raw.timeoutMs) ?? (env("SEARXNG_TIMEOUT_MS") ? asInt(env("SEARXNG_TIMEOUT_MS")) : undefined) ?? 25_000
|
||||
const timeoutMs = Math.min(120_000, Math.max(1_000, timeoutRaw))
|
||||
|
||||
const bearerToken =
|
||||
asString(raw.bearerToken) ?? env("SEARXNG_BEARER_TOKEN") ?? env("SEARXNG_AUTH_BEARER") ?? undefined
|
||||
|
||||
return { baseUrl, categories, engines, language, safesearch, timeRange, maxResults, timeoutMs, bearerToken }
|
||||
}
|
||||
|
||||
/** Effect Schema for the companion `searxng` tool input (structured per-query overrides). */
|
||||
export const ToolInput = Schema.Struct({
|
||||
query: Schema.String.annotate({ description: "Search query. Supports SearXNG syntax: !engine !category :language (e.g. '!wikipedia :de effect hooks')" }),
|
||||
categories: Schema.optional(Schema.String).annotate({
|
||||
description: "Comma-separated SearXNG categories (e.g. 'general,images,news'). Defaults to plugin config.",
|
||||
}),
|
||||
engines: Schema.optional(Schema.String).annotate({
|
||||
description: "Comma-separated SearXNG engines (e.g. 'wikipedia,duckduckgo'). Defaults to plugin config / server default.",
|
||||
}),
|
||||
language: Schema.optional(Schema.String).annotate({ description: "Result language (e.g. 'all', 'en', 'de'). Defaults to plugin config." }),
|
||||
time_range: Schema.optional(Schema.Literals(["day", "week", "month", "year"])).annotate({
|
||||
description: "Limit results by age. Defaults to plugin config.",
|
||||
}),
|
||||
safesearch: Schema.optional(Schema.Literals([0, 1, 2])).annotate({
|
||||
description: "Safe search level. Defaults to plugin config.",
|
||||
}),
|
||||
pageno: Schema.optional(Schema.Int.pipe(Schema.check(Schema.isGreaterThanOrEqualTo(1), Schema.isLessThanOrEqualTo(10)))).annotate({
|
||||
description: "Result page (1-10). Default 1.",
|
||||
}),
|
||||
maxResults: Schema.optional(Schema.Int.pipe(Schema.check(Schema.isGreaterThanOrEqualTo(1), Schema.isLessThanOrEqualTo(20)))).annotate({
|
||||
description: "Max results to return. Defaults to plugin config.",
|
||||
}),
|
||||
})
|
||||
export type ToolInput = typeof ToolInput.Type
|
||||
+111
@@ -0,0 +1,111 @@
|
||||
import { define } from "@opencode/plugin/effect/plugin"
|
||||
import { Tool } from "@opencode/schema/tool"
|
||||
import { Duration, Effect, Schema, Scope } from "effect"
|
||||
import { HttpClient } from "effect/http"
|
||||
import { resolveConfig, ToolInput } from "./config.js"
|
||||
import { PROVIDER_ID, PROVIDER_NAME, search } from "./searxng.js"
|
||||
|
||||
const ToolOutput = Schema.Struct({
|
||||
results: Schema.Array(
|
||||
Schema.Struct({
|
||||
url: Schema.String,
|
||||
title: Schema.String.pipe(Schema.optional),
|
||||
content: Schema.String.pipe(Schema.optional),
|
||||
}),
|
||||
),
|
||||
})
|
||||
|
||||
const TOOL_DESCRIPTION = [
|
||||
"Search the web via self-hosted SearXNG with explicit per-query control.",
|
||||
"Supports SearXNG syntax in `query` (!engine !category :language, e.g. '!wikipedia :de effect hooks').",
|
||||
"Explicit `engines`/`categories`/`language` fields override plugin defaults for this call only.",
|
||||
].join(" ")
|
||||
|
||||
export const Plugin = define<HttpClient.HttpClient | Scope.Scope>({
|
||||
id: "heretek-ai.websearch.searxng",
|
||||
effect: Effect.fn("SearxngPlugin")(function* (ctx) {
|
||||
const http = yield* HttpClient.HttpClient
|
||||
let config: ReturnType<typeof resolveConfig>
|
||||
try {
|
||||
config = resolveConfig(ctx.options ?? {})
|
||||
} catch (error) {
|
||||
yield* Effect.logWarning("opencode-searxng disabled: invalid configuration", {
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
yield* ctx.websearch.transform((editor) => {
|
||||
editor.add({
|
||||
id: PROVIDER_ID,
|
||||
name: PROVIDER_NAME,
|
||||
execute: (input) =>
|
||||
search(http, ctx.app, config, { query: input.query }).pipe(
|
||||
Effect.timeoutOrElse({
|
||||
duration: Duration.millis(config.timeoutMs + 5_000),
|
||||
orElse: () => Effect.fail(new Error(`SearXNG request timed out after ${config.timeoutMs}ms`)),
|
||||
}),
|
||||
),
|
||||
})
|
||||
// Intentionally no editor.default.set(): user opts in via
|
||||
// opencode.json { "websearch": { "provider": "searxng" } }.
|
||||
})
|
||||
|
||||
yield* ctx.tool
|
||||
.transform((editor) =>
|
||||
editor.add({
|
||||
name: PROVIDER_ID,
|
||||
options: { codemode: false },
|
||||
description: TOOL_DESCRIPTION,
|
||||
input: ToolInput,
|
||||
output: ToolOutput,
|
||||
execute: (input) =>
|
||||
Effect.gen(function* () {
|
||||
const results = yield* search(
|
||||
http,
|
||||
ctx.app,
|
||||
config,
|
||||
{
|
||||
query: input.query,
|
||||
categories: input.categories,
|
||||
engines: input.engines,
|
||||
language: input.language,
|
||||
time_range: input.time_range,
|
||||
safesearch: input.safesearch,
|
||||
pageno: input.pageno ?? 1,
|
||||
},
|
||||
input.maxResults ?? config.maxResults,
|
||||
)
|
||||
const output = {
|
||||
results: results.map((item) => ({
|
||||
url: item.url,
|
||||
...(item.title ? { title: item.title } : {}),
|
||||
...(item.content ? { content: item.content } : {}),
|
||||
})),
|
||||
}
|
||||
const content =
|
||||
output.results.length === 0
|
||||
? "No search results found. Please try a different query."
|
||||
: output.results
|
||||
.map((item) => {
|
||||
const title = item.title ?? item.url
|
||||
return `## [${title}](${item.url})${item.content ? `\n\n${item.content}` : ""}`
|
||||
})
|
||||
.join("\n\n")
|
||||
return { output, content, metadata: { provider: PROVIDER_ID } }
|
||||
}).pipe(
|
||||
Effect.mapError(
|
||||
(error) =>
|
||||
new Tool.Error({
|
||||
message: `Unable to search SearXNG for ${input.query}`,
|
||||
error,
|
||||
}),
|
||||
),
|
||||
),
|
||||
}),
|
||||
)
|
||||
.pipe(Effect.orDie)
|
||||
}),
|
||||
})
|
||||
|
||||
export default Plugin
|
||||
+146
@@ -0,0 +1,146 @@
|
||||
import { Duration, Effect, Schema } from "effect"
|
||||
import { HttpClient, HttpClientRequest, HttpClientResponse } from "effect/http"
|
||||
import type { ResolvedConfig } from "./config.js"
|
||||
|
||||
export const VERSION = "0.1.0"
|
||||
export const PROVIDER_ID = "searxng"
|
||||
export const PROVIDER_NAME = "SearXNG"
|
||||
|
||||
/** Tolerant SearXNG `/search?format=json` item shape. Extra fields are ignored. */
|
||||
export const SearxngItem = Schema.Struct({
|
||||
url: Schema.String,
|
||||
title: Schema.NullOr(Schema.String).pipe(Schema.optional),
|
||||
content: Schema.NullOr(Schema.String).pipe(Schema.optional),
|
||||
publishedDate: Schema.NullOr(Schema.String).pipe(Schema.optional),
|
||||
pubdate: Schema.NullOr(Schema.String).pipe(Schema.optional),
|
||||
engine: Schema.NullOr(Schema.String).pipe(Schema.optional),
|
||||
})
|
||||
export type SearxngItem = typeof SearxngItem.Type
|
||||
|
||||
export const SearxngResponse = Schema.Struct({
|
||||
query: Schema.String.pipe(Schema.optional),
|
||||
results: Schema.Array(SearxngItem),
|
||||
})
|
||||
export type SearxngResponse = typeof SearxngResponse.Type
|
||||
|
||||
export interface SearchParams {
|
||||
readonly query: string
|
||||
readonly categories?: string | undefined
|
||||
readonly engines?: string | undefined
|
||||
readonly language?: string | undefined
|
||||
readonly time_range?: string | undefined
|
||||
readonly safesearch?: number | undefined
|
||||
readonly pageno?: number | undefined
|
||||
}
|
||||
|
||||
export interface WebSearchResult {
|
||||
readonly url: string
|
||||
readonly title?: string | undefined
|
||||
readonly content?: string | undefined
|
||||
readonly time: { readonly published?: number | undefined }
|
||||
}
|
||||
|
||||
export function parsePublished(value: string | null | undefined): number | undefined {
|
||||
if (!value) return undefined
|
||||
const parsed = Date.parse(value)
|
||||
return Number.isFinite(parsed) ? parsed : undefined
|
||||
}
|
||||
|
||||
/** Map one SearXNG result row to opencode `WebSearch.Result`. Returns undefined when unusable. */
|
||||
export function mapItem(item: SearxngItem): WebSearchResult | undefined {
|
||||
const url = item.url?.trim()
|
||||
if (!url) return undefined
|
||||
const title = item.title?.trim() ? item.title.trim() : undefined
|
||||
const content = item.content?.trim() ? item.content.trim() : undefined
|
||||
const published = parsePublished(item.publishedDate ?? item.pubdate ?? undefined)
|
||||
return {
|
||||
url,
|
||||
...(title ? { title } : {}),
|
||||
...(content ? { content } : {}),
|
||||
time: { ...(published !== undefined ? { published } : {}) },
|
||||
}
|
||||
}
|
||||
|
||||
export function mapResults(items: readonly SearxngItem[], maxResults: number): readonly WebSearchResult[] {
|
||||
const out: WebSearchResult[] = []
|
||||
for (const item of items) {
|
||||
if (out.length >= maxResults) break
|
||||
const mapped = mapItem(item)
|
||||
if (mapped) out.push(mapped)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function buildSearchUrl(baseUrl: string, params: SearchParams, defaults: ResolvedConfig): string {
|
||||
const url = new URL(`${baseUrl.replace(/\/+$/, "")}/search`)
|
||||
url.searchParams.set("q", params.query)
|
||||
url.searchParams.set("format", "json")
|
||||
const categories = params.categories ?? defaults.categories
|
||||
if (categories) url.searchParams.set("categories", categories)
|
||||
const engines = params.engines ?? defaults.engines
|
||||
if (engines) url.searchParams.set("engines", engines)
|
||||
const language = params.language ?? defaults.language
|
||||
if (language) url.searchParams.set("language", language)
|
||||
const timeRange = params.time_range ?? defaults.timeRange
|
||||
if (timeRange) url.searchParams.set("time_range", timeRange)
|
||||
const safesearch = params.safesearch ?? defaults.safesearch
|
||||
url.searchParams.set("safesearch", String(safesearch))
|
||||
url.searchParams.set("pageno", String(params.pageno ?? 1))
|
||||
return url.toString()
|
||||
}
|
||||
|
||||
export function userAgent(app: { readonly name: string; readonly version: string; readonly channel: string }): string {
|
||||
return `opencode/${app.channel}/${app.version}/${app.name} opencode-searxng/${VERSION}`
|
||||
}
|
||||
|
||||
export function hintForStatus(status: number): string {
|
||||
if (status === 403)
|
||||
return "SearXNG returned 403. Enable JSON output on the server: search.formats must include 'json' in settings.yml."
|
||||
if (status === 429) return "SearXNG rate limited (HTTP 429). Retry later or raise server limiter settings."
|
||||
if (status === 401 || status === 403)
|
||||
return `SearXNG authentication failed (HTTP ${status}). Check reverse-proxy auth / bearer token.`
|
||||
return `SearXNG request failed (HTTP ${status})`
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a SearXNG search and return opencode-shaped results.
|
||||
* Query string is forwarded verbatim so `!engine !category :lang` syntax keeps working per-query.
|
||||
*/
|
||||
export const search = (
|
||||
http: HttpClient.HttpClient,
|
||||
app: { readonly name: string; readonly version: string; readonly channel: string },
|
||||
config: ResolvedConfig,
|
||||
params: SearchParams,
|
||||
maxResults?: number,
|
||||
): Effect.Effect<readonly WebSearchResult[], unknown> =>
|
||||
Effect.gen(function* () {
|
||||
const url = buildSearchUrl(config.baseUrl, params, config)
|
||||
const limit = Math.min(20, Math.max(1, maxResults ?? config.maxResults))
|
||||
const request = HttpClientRequest.get(url).pipe(
|
||||
HttpClientRequest.acceptJson,
|
||||
HttpClientRequest.setHeaders({
|
||||
"User-Agent": userAgent(app),
|
||||
Accept: "application/json",
|
||||
...(config.bearerToken ? { Authorization: `Bearer ${config.bearerToken}` } : {}),
|
||||
}),
|
||||
)
|
||||
const response = yield* HttpClient.withScope(HttpClient.filterStatusOk(http))
|
||||
.execute(request)
|
||||
.pipe(
|
||||
Effect.flatMap(HttpClientResponse.schemaBodyJson(SearxngResponse)),
|
||||
Effect.scoped,
|
||||
Effect.timeoutOrElse({
|
||||
duration: Duration.millis(config.timeoutMs),
|
||||
orElse: () => Effect.fail(new Error(`SearXNG request timed out after ${config.timeoutMs}ms`)),
|
||||
}),
|
||||
Effect.mapError((cause) => {
|
||||
const status =
|
||||
typeof cause === "object" && cause !== null && "response" in cause
|
||||
? (cause as { response?: { status?: number } }).response?.status
|
||||
: undefined
|
||||
if (status !== undefined) return new Error(hintForStatus(status))
|
||||
return cause
|
||||
}),
|
||||
)
|
||||
return mapResults(response.results, limit)
|
||||
})
|
||||
@@ -0,0 +1,3 @@
|
||||
export { Plugin, default } from "./index.js"
|
||||
export { resolveConfig, ToolInput } from "./config.js"
|
||||
export { buildSearchUrl, mapItem, mapResults, parsePublished, search } from "./searxng.js"
|
||||
@@ -0,0 +1,51 @@
|
||||
import { describe, it } from "node:test"
|
||||
import assert from "node:assert/strict"
|
||||
import { normalizeBaseUrl, resolveConfig } from "../dist/config.js"
|
||||
|
||||
describe("normalizeBaseUrl", () => {
|
||||
it("trims trailing slashes", () => {
|
||||
assert.equal(normalizeBaseUrl("http://localhost:8888///"), "http://localhost:8888")
|
||||
})
|
||||
it("keeps subpaths", () => {
|
||||
assert.equal(normalizeBaseUrl("https://example.com/searxng/"), "https://example.com/searxng")
|
||||
})
|
||||
it("rejects non-http", () => {
|
||||
assert.throws(() => normalizeBaseUrl("ftp://example.com"), /protocol/)
|
||||
})
|
||||
it("rejects garbage", () => {
|
||||
assert.throws(() => normalizeBaseUrl("not a url"), /Invalid SearXNG baseUrl/)
|
||||
})
|
||||
})
|
||||
|
||||
describe("resolveConfig", () => {
|
||||
it("throws without baseUrl", () => {
|
||||
const prev = process.env.SEARXNG_BASE_URL
|
||||
delete process.env.SEARXNG_BASE_URL
|
||||
try {
|
||||
assert.throws(() => resolveConfig({}), /missing baseUrl/)
|
||||
} finally {
|
||||
if (prev !== undefined) process.env.SEARXNG_BASE_URL = prev
|
||||
}
|
||||
})
|
||||
it("applies defaults and clamps", () => {
|
||||
const cfg = resolveConfig({ baseUrl: "http://localhost:8888/", maxResults: 99, timeoutMs: 1 })
|
||||
assert.equal(cfg.baseUrl, "http://localhost:8888")
|
||||
assert.equal(cfg.categories, "general")
|
||||
assert.equal(cfg.language, "all")
|
||||
assert.equal(cfg.safesearch, 1)
|
||||
assert.equal(cfg.maxResults, 20)
|
||||
assert.equal(cfg.timeoutMs, 1000)
|
||||
})
|
||||
it("reads env fallback", () => {
|
||||
process.env.SEARXNG_BASE_URL = "http://searxng:8888"
|
||||
try {
|
||||
const cfg = resolveConfig({})
|
||||
assert.equal(cfg.baseUrl, "http://searxng:8888")
|
||||
} finally {
|
||||
delete process.env.SEARXNG_BASE_URL
|
||||
}
|
||||
})
|
||||
it("rejects bad timeRange", () => {
|
||||
assert.throws(() => resolveConfig({ baseUrl: "http://x", timeRange: "hour" }), /timeRange/)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,86 @@
|
||||
import { describe, it } from "node:test"
|
||||
import assert from "node:assert/strict"
|
||||
import { buildSearchUrl, mapItem, mapResults, parsePublished } from "../dist/searxng.js"
|
||||
import { resolveConfig } from "../dist/config.js"
|
||||
|
||||
const defaults = resolveConfig({ baseUrl: "http://localhost:8888" })
|
||||
|
||||
describe("parsePublished", () => {
|
||||
it("parses ISO dates", () => {
|
||||
assert.equal(parsePublished("2024-05-01T12:00:00+00:00"), Date.parse("2024-05-01T12:00:00+00:00"))
|
||||
})
|
||||
it("returns undefined for missing/garbage", () => {
|
||||
assert.equal(parsePublished(undefined), undefined)
|
||||
assert.equal(parsePublished(null), undefined)
|
||||
assert.equal(parsePublished("not a date"), undefined)
|
||||
})
|
||||
})
|
||||
|
||||
describe("mapItem", () => {
|
||||
it("maps full row", () => {
|
||||
const out = mapItem({
|
||||
url: "https://example.com/a",
|
||||
title: " Title ",
|
||||
content: " Body ",
|
||||
publishedDate: "2024-05-01T00:00:00Z",
|
||||
})
|
||||
assert.deepEqual(out, {
|
||||
url: "https://example.com/a",
|
||||
title: "Title",
|
||||
content: "Body",
|
||||
time: { published: Date.parse("2024-05-01T00:00:00Z") },
|
||||
})
|
||||
})
|
||||
it("drops empty url", () => {
|
||||
assert.equal(mapItem({ url: " " }), undefined)
|
||||
})
|
||||
it("omits blank title/content and bad date", () => {
|
||||
const out = mapItem({ url: "https://example.com", title: " ", content: "", publishedDate: "xx" })
|
||||
assert.deepEqual(out, { url: "https://example.com", time: {} })
|
||||
})
|
||||
it("falls back to pubdate", () => {
|
||||
const out = mapItem({ url: "https://example.com", pubdate: "2023-01-02" })
|
||||
assert.deepEqual(out, { url: "https://example.com", time: { published: Date.parse("2023-01-02") } })
|
||||
})
|
||||
})
|
||||
|
||||
describe("mapResults", () => {
|
||||
it("filters bad rows and caps maxResults", () => {
|
||||
const out = mapResults(
|
||||
[{ url: "https://a.example" }, { url: "" }, { url: "https://b.example" }],
|
||||
1,
|
||||
)
|
||||
assert.equal(out.length, 1)
|
||||
assert.equal(out[0]?.url, "https://a.example")
|
||||
})
|
||||
})
|
||||
|
||||
describe("buildSearchUrl", () => {
|
||||
it("builds json url with defaults", () => {
|
||||
const url = new URL(buildSearchUrl("http://localhost:8888/", { query: "hello world" }, defaults))
|
||||
assert.equal(url.origin + url.pathname, "http://localhost:8888/search")
|
||||
assert.equal(url.searchParams.get("q"), "hello world")
|
||||
assert.equal(url.searchParams.get("format"), "json")
|
||||
assert.equal(url.searchParams.get("categories"), "general")
|
||||
assert.equal(url.searchParams.get("language"), "all")
|
||||
assert.equal(url.searchParams.get("safesearch"), "1")
|
||||
assert.equal(url.searchParams.get("pageno"), "1")
|
||||
})
|
||||
it("per-query overrides win and bang syntax survives", () => {
|
||||
const url = new URL(
|
||||
buildSearchUrl(
|
||||
"http://localhost:8888",
|
||||
{ query: "!wikipedia :de effect", engines: "wikipedia", language: "de", pageno: 2 },
|
||||
defaults,
|
||||
),
|
||||
)
|
||||
assert.equal(url.searchParams.get("q"), "!wikipedia :de effect")
|
||||
assert.equal(url.searchParams.get("engines"), "wikipedia")
|
||||
assert.equal(url.searchParams.get("language"), "de")
|
||||
assert.equal(url.searchParams.get("pageno"), "2")
|
||||
})
|
||||
it("preserves base subpath", () => {
|
||||
const url = new URL(buildSearchUrl("https://example.com/searxng", { query: "x" }, defaults))
|
||||
assert.equal(url.origin + url.pathname, "https://example.com/searxng/search")
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"lib": ["ES2022"],
|
||||
"strict": true,
|
||||
"skipLibCheck": true,
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"sourceMap": true,
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"exactOptionalPropertyTypes": true,
|
||||
"verbatimModuleSyntax": true
|
||||
},
|
||||
"include": ["src"]
|
||||
}
|
||||
Reference in new issue
Block a user