Creating a search engine for fun and because Google sucks
A software engineer is developing a free, fast, and privacy-focused search engine using Rust, sourcing results from DuckDuckGo and images from Google, while addressing challenges with pagination and HTML parsing.
Read original article
A software engineer is developing a personal search engine due to dissatisfaction with current options, particularly Google, which has become overloaded with ads and AI-generated content. The engineer aims to create a free, fast, and safe search engine using Rust, avoiding reliance on paid APIs and ensuring user privacy by not tracking searches. The initial plan involves scraping results from DuckDuckGo (DDG) and images from Google, leveraging DDG Lite for its simplicity. The engineer has encountered challenges with pagination and parsing HTML responses, leading to issues with retrieving additional search results. Despite these hurdles, the project is ongoing, with the engineer exploring alternative parsing methods and considering the use of Servo, a Rust-based browser engine. The search engine is currently in a rudimentary state, and the engineer is open to job opportunities in Rust development.
- The engineer is building a personal search engine to address dissatisfaction with existing options.
- The project focuses on being free, fast, and safe, prioritizing user privacy.
- Initial results will be sourced from DuckDuckGo and images from Google.
- Challenges include pagination issues and HTML parsing errors.
- The engineer is exploring alternative methods and is open to job opportunities in Rust development.
Related
Google rejected me and now I'm building a search engine
The article recounts a rejection from Google during an interview, prompting the individual to create a non-profit, community-driven search engine emphasizing ethical values over profit, welcoming contributions for development.
Non-Google search engines blocked from showing Reddit results
Reddit's updated Content Policy blocks non-Google search engines from displaying recent results, limiting access to its content for AI training, and raising concerns about content ownership and search engine dominance.
Building a Local Perplexity Alternative with Perplexica, Ollama, and SearXNG
The article outlines how to create a self-hosted alternative to the AI search engine Perplexity using open-source tools, emphasizing cost-effectiveness and customization through a step-by-step installation guide.
This month in Servo: tabbed browsing, Windows buffs, devtools, and more
In August 2024, Servo introduced new features like WebXR support, a redesigned browser interface, performance enhancements for Windows, improved development tools, and increased monthly donations for ongoing development.
Build a quick Local code intelligence using Ollama with Rust
Bosun developed Swiftide, a Rust-based tool for efficient code indexing and querying, utilizing Qdrant and FastEmbed. It enhances performance with OpenTelemetry, integrating various language models for improved response times.
12 commentsAlternatives while probably fine in America, suck in the Nordics. I think people forget just how much search traffic happens in this category.
[1] - https://docs.searxng.org [2] - https://github.com/ItzCrazyKns/Perplexica
It's a metasearch engine that can query multiple search providers at once, including google, so you're not missing out on the good results you expect. Pick an instance at https://searx.space/ and tell your friends!
I imagine I could do it on consumer hardware for less than $10-20k.
Perhaps common crawl has done much of the heavy lifting already and I just have an indexing task.
I visited the page on mobile and I have to scroll horizontally back and forth before I can read?
That is not fun at all.
It'd be so easy to filter I wonder why they/Microsoft don't bother. Oh, wait...
Related
Google rejected me and now I'm building a search engine
The article recounts a rejection from Google during an interview, prompting the individual to create a non-profit, community-driven search engine emphasizing ethical values over profit, welcoming contributions for development.
Non-Google search engines blocked from showing Reddit results
Reddit's updated Content Policy blocks non-Google search engines from displaying recent results, limiting access to its content for AI training, and raising concerns about content ownership and search engine dominance.
Building a Local Perplexity Alternative with Perplexica, Ollama, and SearXNG
The article outlines how to create a self-hosted alternative to the AI search engine Perplexity using open-source tools, emphasizing cost-effectiveness and customization through a step-by-step installation guide.
This month in Servo: tabbed browsing, Windows buffs, devtools, and more
In August 2024, Servo introduced new features like WebXR support, a redesigned browser interface, performance enhancements for Windows, improved development tools, and increased monthly donations for ongoing development.
Build a quick Local code intelligence using Ollama with Rust
Bosun developed Swiftide, a Rust-based tool for efficient code indexing and querying, utilizing Qdrant and FastEmbed. It enhances performance with OpenTelemetry, integrating various language models for improved response times.