Dawn of a new era in Search: Balancing innovation, competition, and public good

I've been using Kagi for a while (almost two years now!) and it's been nothing but excellent!

Lenses are very useful (Reddit lens is on every second search), and I personally really like the AI features they are working on.

The quick assist triggered by a question mark at the end of a search query which makes a quick ai-generated summary of the few top results is something I use constantly.

The new more advanced assistant which is able to do searches, which can also be constrained to lenses, and lets you pick an arbitrary model, is also excellent, and basically means I don't need a chatgpt/claude subscription, as Kagi covers it very well.

All in all, great product which I'm happy to pay for.

> The Google Search Index is a unique and irreplaceable resource within the digital ecosystem. Mandating fair access to it or treating it as an essential facility could address the core issues...

The article estimates the Google Search Index at 12.5PB. If Kagi thinks that is a big enough moat to be the primary target then, well, I suppose they should know. But I'm also skeptical. You could fit that on about 50 Hetzner SX295, so about $20k/month. Plus the cost of gathering the data. It is surely a huge resource.

But weighed against the combination of Google Search + AdWords + Android + YouTube + Chrome, all in a single company? To me a 12.5PB search index feels like small change in comparison.

NB: Happy Kagi-paying customer here.

I've been using (and occasionally paying) Kagi on and off for a couple of years now. I truly think they're building something interesting and valuable! While I haven't agreed with every product decision they've made, the founder is very good at both understanding his business and also explaining their decisions. This is a well crafted explainer of the search business and the monopoly case-- much better for sharing with less tech-savvy peers than most mainstream media explainers on this subject!

Edit: wording

Edit 2: Can you imagine a world where Google's Internet Search Index is legally considered an "Essential Facility"!? https://law.stanford.edu/publications/essential-platforms/

> Google has built a massive index of the internet that covers close to 100% of the accessible web.

While their index (of other peoples stuff) is enormous it far from includes everything. It is easy to disqualify and people would be screaming if content farms would be included. What even is a content farm nowadays? One can return a reasonable article for any query with llms rich in links to other pages that don't exist but could be indexed and are part of the accessible web

If you make a new website with a few thousand pages and a few thousand images it takes quite a while for google to pick up the entire thing, if it even bothers to.

google tries to fill the result page with a small subset of websites. A good thing for users most of the time and the easiest ad money but horrible for new players.

it use to be quite common for bloggers (and others) to follow everything written about them or of interest. google (blog search) and technorati were very useful for that kind of discovery.

The average user might never have noticed that but when it was killed off the www stopped being a community.

We can pretend the index is still there. If you cant get to it it's s much like the llm content.

I’ve been a happy Kagi customer for probably 2 years(?) now. Highlights for me, as a professional:

- I can blacklist low-value domains (such as geeksforgeeks) that dominate the top of many programming searches.

- I can increase/decrease the priority of domains or pin domains to the top of searches, such as official documentation for languages/libraries.

- I can use “Lenses” to filter results for programming/academic/forum results.

This essentially advocates for the same thing defined in Cory Doctorow's The Internet Con: How to Seize the Means of Computation. That point being, requiring open protocols from big tech will enable competition and innovation. This will return the creative inspiration to the technologists. I completely agree with it, and I hope we are reaching an inflection point where walled data gardens are cracked open.

I switched from Google to DDG for default search a few years ago, and then to Kagi maybe 18 months ago. Kagi's simply excellent.

Who knows what websites and pages are even on the web?

There's no index to the web that I know of apart from Google and DuckDuckGo and maybe this Kagi thing.

I want to explore the web - surely search isn't the only way to use the web?

I imagine it could be fun to explore the web, lists and graphs of interest where I can hop from here to there via list of links or graphs or nodes or something?

Does anyone know of anything like this?

I think Kagi is correct and that the way we explore information on the internet will look very different in X years with all the changes LLMs will bring. I think the real question will be what will it look like.

I don't think it looks like search today. Google got where they were because they were 10x better than everything else and had an experience focusing on what mattered at the time. I don't think the 10x experience will look like ten blue links. I don't know what that next experience is, but I'll know it when I see it.

I wish Kagi was cheaper. It's expensive in many countries and it doesn't help that we are bombarded with subscriptions everywhere.

> Apple has stated that Bing does not match Google’s search result quality, and they are unwilling to compromise on user experience by offering subpar results.

I wouldn't take this statement at face value. This is most likely a BS PR excuse for Apple to maintain their current deal with Google. I wouldn't expect anything less from any large corporation looking to protect $20 billion in annual revenue.

Paying kagi user. All day every day

I've used Kagi for a year and it was better than Google for most of the searches.

However I think the model will be changed to something more like Perplexity.ai

I've switched to Perplexity and for most of the searches it works better than Kagi.

They'd need to add something like this to survive in the long run, because for exploratory searches tools like Perplexity are really good.

Here’s a 30min audio version of TFA:

https://www.audiowaveai.com/p/2626-dawn-of-a-new-era-in-sear...

As someone who hasn’t used paid search services, what sort of problems has this solved for other HN users that make it worth it? How does it compare to “AI” based search tools like Perplexity?

A smaller competitor wants and advocates for itself. Makes sense but is it really surprising? It would be strange otherwise.

I do wonder how far one can get charging for search.

Now it dawns on me that in this new era I can write an article for the public good and by balancing the wording I am able to place it before the 3852 competing public good articles on the search results page. lets innovate

This is an extremely disappointing post. In the past, I’ve enthusiastically supported and advocated for Kagi.

But Kagi advocating for using force to destroy its competitors is completely unacceptable to me and an admission that they do not believe they have a viable product.

Antitrust law is arbitrary and evil. If you make more money than your competitors, you have undue market power. If you price below your competitors, you are dumping. If you price the same as your competitors, you are colluding. The whole thing is a naked power grab by politicians and inferior companies.

This is a sad day. Kagi is the best thing that’s happened to the internet in the past decade. And now I have to stop my auto renew.

i feel bad for kagi/ddg

they are still trying to fight the google by building pretty much the same product

while perplexity is obviously in the lead by being ai-first