Give Google the boot by building your own search engine
- ID
- 14103
- Status
- summarized
- Published
- 14 Aug 2026, 4:48 AM
- Fetched
- 14 Aug 2026, 2:09 PM
- Provider
- The Register
- Category
- technology
- Original URL
- https://www.theregister.com/software/2026/08/13/give-google-the-boot-by-building-your-own-search-engine/5287535
- Source URL
- https://www.theregister.com/headlines.atom
Summary
- Score
- 7.0
- Created
- 14 Aug 2026, 2:10 PM
- Tags
- Audience
- developersvibe_codersai_agent_users
What happened
UK-based developer Alex Morley-Finch built Marlin, an open-source personal search engine that indexed ~560,000 homepages for ~$10 in cloud GPU time using under 1GB of storage. It uses a small OpenAI-compatible local LLM to generate summaries, categories, and tags for each page, with four components: a fetcher, a worker, a steward for filtering bad pages, and an API with web UI. His first crawl went wrong—90% corporate sites and docs—so he added a weighting system to prioritize pages he actually cared about, like portfolios and indie projects.
Why it matters
If you want a search index scoped to your own interests—e.g., Malaysian indie dev portfolios, local startup pages, or niche technical blogs—this shows it's feasible for ~$10 and a weekend, with a crawl-weighting approach to avoid drowning in generic corporate content. The main unresolved pain point is LLM-generated tagging quality, which is worth discussing before you replicate the architecture.
Discussion angle
The crawl-weighting problem is the real lesson: his first run pulled 90% corporate/docs. Discuss how you'd seed and tune a weighting queue for a Malaysia- or SEA-focused index, and whether LLM auto-tagging is reliable enough to skip manual curation.