|
Update ddgs to 9.16.0 and keep backend="auto" while preserving the
LLM-facing query/max_results inputs and title/href/body result schema.
Implementation:
- Add OLLAMA_WEB_SEARCH_REGION with auto/us-en/jp-jp values, normalized
configuration input, and a clear startup error for unsupported values.
- In auto mode, detect kana after NFKC normalization and exclusion of URLs
and site:/filetype: tokens. Select jp-jp for kana and us-en otherwise,
without modifying the original query or sharing per-call region state.
- Preserve href/url and body/snippet fallbacks and normalize None values.
- Reject malformed HTTP(S) URLs without DNS lookups or extra requests.
- Unwrap known Yahoo redirect URLs before excluding Bing aclick ads.
- Represent missing and URL-only titles as empty strings rather than
inventing titles from snippets. Preserve remaining provider order and
allow fewer results after filtering, without additional searches.
- Preserve provider exception propagation and identify ddgs in error logs.
Coverage and documentation:
- Add 72 regression cases for configuration, region routing, concurrency,
public schemas, title/URL/ad boundaries, limits, and search_and_fetch
continuation. Update existing provider mocks and log assertions.
- Document the region environment setting and result semantics in both
READMEs and tools-schema.md; update the changelog.
- Keep only a short backend availability caveat in tools-schema.md and
remove the temporary backend comparison, design, and validation docs.
Validation:
- pytest: 222 passed; ruff check . and mypy src/ passed.
- uv lock --check and git diff --check passed.
- Initial live acceptance: 56 searches plus 3 registry smoke checks;
default-region auto succeeded 8/18 times and real ad filtering removed
two ads. Stable relevance/title quality targets remain unmet because
provider acquisition is unreliable.
- Follow-up backend study: 234 comparative searches plus 3 Yahoo response
diagnostics. Yahoo led individual backends at 9/18; restricted multi-
backend combinations did not outperform the contemporaneous auto
control. Authentication, access restrictions, temporary upstream
failures, and Yahoo HTML/parser mismatches were observed.
Do not claim search reliability is fixed or add backend configuration,
custom retries, ranking changes, or challenge bypasses. Running app
processes were not restarted; restart is needed to load the changes.
|
|
Store server.log inside OLLAMA_WEB_DATA_DIR and manage the ollama-web file handler through the Starlette lifespan. Close and detach the handler on shutdown to prevent duplicate records and Windows file locks when applications are recreated.
Stop converting DuckDuckGo provider failures into empty search results. Record a query-safe traceback, re-raise the original exception, and let ToolRegistry expose the exception type and message as a tool error without adding or spoofing Referer headers.
Remove the unused non-streaming chat_with_tools implementation and its private helpers. Move capability, options, standard tool, MCP tool, and final tools=[] regression coverage onto the production streaming path.
Add integration coverage for data/server.log placement, handler cleanup, repeated lifespans, and provider 403 diagnostics. Update the Unreleased changelog entry.
Validated with 118 tests, Ruff, mypy, diff checks, real DuckDuckGo searches, successful result-page fetches, and observable 401/403 responses from sites that reject automated retrieval.
|
