Elasticsearch is open source, again

Here’s the initial AWS response to the license change that they made in 2018, which I helped write. At the time we didn’t think a new license made sense, as AGPL is sufficient to block AWS from using the code, but the core of the issue was that AWS wanted to contribute security features to the open source project and Elastic wanted to keep security as an enterprise feature, so rejected all the approaches AWS made at the time. https://aws.amazon.com/blogs/opensource/keeping-open-source-...

I'm pretty happy with this, since they are keeping the option to use the Elastic License. Now everyone can be happy. To me, it's weird that the AGPL is any more "open source" than the Elastic License. The AGPL requires you to publish all of your source code if you make any changes to the product; the Elastic License just says, "don't use our code to make a direct competitor to Elasticsearch". I find the former to be much more restrictive in most practical ways since the majority of companies don't want to open source their code, but very few of them plan to sell hosted search.

Personally, I do wish that there was more broad acceptance of the Elastic License. Who wants to put in years building a business and then have a competitor with better distribution take your code and compete directly with you? For me, the reasons to want open-source code are:

* If a vendor goes under, I can self-host

* If a vendor raises prices too much, I can self-host

* If there's a bug in the code that affects me too much, I can fix it

* If there's a feature I really need, I can add it

The Elastic License allows for all of the above. Seems fair to me.

> The good news is that while it was painful, it worked. 3 years later, Amazon is fully invested in their fork, the market confusion has been (mostly) resolved, and our partnership with AWS is stronger than ever. We were even named AWS partner of the year.

I don't entirely understand this bit.

> For example, MongoDB used to be AGPL and Grafana is AGPL. It shows that AGPL doesn’t affect usage or popularity.

I take some issue with this characterization. Let's look at Grafana in particular. Grafana was not always AGPL, and much of its popularity came before the license change. I've been in multiple organizations who only purchased a license for Grafana to avoid the AGPL terms because it had gained traction already in the organization and switching away would have been more costly, and AGPL software is still outright banned.

That Grafana is still popular does not show that the AGPL doesn't impact usage or popularity, only that Grafana is still popular.

Nice, maybe they can hire some humans to write more explanatory website text, documentation, pricing information—better yet, anything on their website.

Elastic.co is the #1 example we use with our clients when we want to show how 'vague websites make you lose clients.' We show them the website and ask, 'What do you think of this company?' and 'What do you think they are providing as a service?' Not a single client, including tech-savvy ones, has been able to answer.

Elastic.co is probably one of the worst websites that somehow gained popularity despite its crappy pricing model and support. Their documentation assumes you already know everything about their weirdly vague services and have in-depth knowledge of server infrastructures.

To anyone who works for them: If you're reading this, know that your website is so terrible that it became our first example of a crappy company.

Too late. We just deployed a new project with OpenSearch after learning from an Elastic salesperson that their licensing fees would eat up significant portions of the projects profits. Also, since it's not spelled out clearly, I'm supposing huge parts of vital functionality are still subject to a paid license, even if the core functionality is technically open source now.

For use by businesses, the AGPL is a nightmare from my perspective. What does it actually require on behalf of a company using AGPL components? If I write my own library and link statically I need to release that? What about if I link it dynamically? What if the library is running on a separate machine and is separated by the network?

I'm sure there will be people commenting in this thread that they understand exactly what the AGPL requires, and it's not that bad, but their opinion matters much less than the opinion of lawyers.

I've never been able to get lawyers in a business setting comfortable with us using AGPL components, for fear that it will be interpreted at some point to require us to release our application source code.

As a result, we've never been able to use anything licensed AGPL in a corporate setting.

Since Elasticsearch changed their license, Loki has also appeared as a competitor and the Grafana machine have released a suite of tools that cover the observability categories. It may also be that the license change encouraged users to look for alternatives and there are more now than just being graylog and Elasticsearch.

Clickhouse has proven to also be a very capable database for logs and there are stacks that use it for log storage.

There's already an open source Elasticsearch, it's called OpenSearch and it is pretty much all you need right now.

Users that had to pay the price to migrate to OpenSearch do not have a reason to migrate back to Elasticsearch.

If Amazon's OpenSearch plateaus, and everyone wants to host ElasticSearch, but on their existing AWS infra/sales business inertia, this issue will fundamentally reoccur. Nothing actually changed except Elastic thinks Amazon will commit to their own fork. If Amazon doesn't, we're back to square one: Amazon hosts open source ElasticSearch, Elastic changes their license, another fork.

"Amazon is fully invested in their fork." Amazon is a cutthroat business that will change strategy if their investment isn't paying off.

That this scenario isn't addressed at the very top of your "addressing the trolls" doesn't bode well at all.

> [D.N.A.]

> [LOVE.]

I don't understand what these are supposed to mean. Is this a new writing style, song references, or just a quirk of the author?

Lots of negativity in this thread. Can we take just a bit of time to appreciate this. Thank you Elastic folks!

Hasn't everyone already moved on to Open search? At this point it's more stable, which is preferable.

Too little too late. Cannot trust.

Great, how long until they change the license again?

That is great news but I have prepared my stack for OpenSearch which seems even if less feature rich but is more trustworthy.

At least there are some security features built in (OIDC/OAuth/JWT/Proxy etc.) which are dead critical to operate any software stack be it internal or otherwise.

As for centralised logging or building a search functionality, OpenSearch was already good enough back in the day at the start of the fork.

I think both ES and OS would continue to flourish in their own ways.

Could be a defensive move of Elasticsearch to win the mindshare of the community. OpenSearch has been gaining some momentum, albeit far from being comparable to Elasticsearch. See OSS Insight: https://ossinsight.io/analyze/elastic/elasticsearch?vs=opens....

I'm not sure Amazon is willing to go all in to win the war of search services, for that means they need to to handsomely reward the best coders that contribute the most to the OpenSearch project to produce insane amount of high-quality code for better and new features. See the great article Code Hard Or Go Home: https://hypercritical.co/2013/04/12/code-hard-or-go-home. Nonetheless, there's a chance that Amazon may be determined to make OpenSearch catch up with ES, just like Apple has made Apple Maps comparable, even not better than, Google Maps. Therefore, open sourcing ES with AGPL is not a bad choice to retain the talent in the ES community.

And AGPL is kinda restrictive to many large customers too, as the customers do not want to risk being forced to open source their business-critical code. In fact, many companies simply ban the use of licensed software. Therefore, AGPL reflects quite the spirit of OSS while in the meantime will not undermine Elastic's business model.

The software engineers who care enough to have an opinion probably likely don't contribute to the ELK stack, and those who are impacted by this license change are no better off. Ever since Elastic went on the warpath with their serverless cloud and Gen AI the entire "open source" pitch is moot, regardless of the license.

Regardless of the openness of their code - their observability product is grossly bloated and unimpressive, the security product is sideways, fleet is broken by design, the entire database sector is coming after their analytics use cases at much better perf + much lower costs (and winning), management look incompetent, RAG is a big bet - but unlikely to be the saving grace the stack + company needs. It's truly a product on fire. Elasticsearch was interesting 10 years ago - nowadays not so much. This just seems like a "hope for the best" distraction for scarier things to come for Elastic.

Congrats on releasing under a free software license again.

We are there to complain when something becomes proprietary, there's no reason we'd not be there when the opposite happens.

> our partnership with AWS is stronger than ever. We were even named AWS partner of the year.

This detail in the post made me chuckle. Oftentimes big vendors give out these kinds of marketing awards strategically.

One big firm I know makes it a point to have its CEO present on-stage awards at its annual user conference to customer that have indicated they might not review.

"Changing the license was a mistake, and Elastic now backtracks from it"

This could have been the entire article and it would make more sense than whatever this is.

This is huge. We wrote "Why we picked AGPL" only a few weeks ago, discussing why ParadeDB (an Elastic competitor!) chose the AGPL. Glad to see Elastic is joining forces on the AGPL front.

Stock is down almost 25% afterhours (not sure if due to this or their earnings call)

They've probably been losing market share to all the opensource vector search solutions. But anyways, this is still good.

Is it still open core? Which parts are only available on the enterprise version? Couldn't easily find in their website (after this announcement).

I'm increasingly of the opinion that the definition of "open source" that narrowly defines open source is going to be the thing that contributes to the reduction of open source software.

Open source communities are essentially anarchist syndicates, collectively working towards common good. Groups like Amazon coming in and taking their work and selling it, profiting to the tune of millions, and contributing nothing back will fundamentally alter the dynamics of the system.

IMO, we need a definition of open source that permits me to limit users to those who aren't actively exploiting me. GPL flavors get close, but not sufficient.

I believe, genuinely, in wellbeing for all, and that we should be working towards the common good of all. Open source is one such effort, but I'm tired of seeing people say, "it's only open source if the code is also permitted to be used against the common good and for the enrichment of a very select few already wealthy people."

what.... are those random words... in brackets??

The change of Elastic license resulted (or perhaps coincided) with the massive growth of other tools like Meilisearch, Quickwit, Loki, typesense, and more.

In a way, their original open source license was suppressing innovation. I know this is not a popular opinion in some circles of pure OSS aficionados but it seems the evidence is to the contrary

Too late, there are so many competitors that are both faster and provide more features. Same will happen with Redis in 3-4 years.

Okay, cool. No notes, that's neat.

Are there any SMEs that have worked with both OpenSearch (the fork) and ElasticSearch? Are there significant differences?

I know the AWS fork had the big difference back then of having RBAC built into their Kibana portion.

Why are they using Kendrick Lamar songs for each title?

Open source, again … for now.

This is their somewhat muddy response to the “trolls” who might say

“Changing the license was a mistake, and Elastic now backtracks from it”.

We removed a lot of market confusion when we changed our license 3 years ago. And because of our actions, a lot has changed. It’s an entirely different landscape now. We aren’t living in the past. We want to build a better future for our users. It’s because we took action then, that we are in a position to take action now.

Elastic trying to reclaim the lost ground in their original territory by re-embracing open source is a bold move. They might be hoping to attract developers and businesses back by offering more transparency and community involvement. Could this have anything to do with vector databases starting to eat into some of their market share, pushing Elastic to innovate and re-establish their relevance?

What license are contributions to Elastic given as? I'm very confused how this works with all the licenses, are they just all compatible with AGPL magically? Or are all contributions under their primary private license, I think the last time I saw someone change a license similar to how Elastic did (maybe MongoDB?) I suggested AGPL, I don't really like the license, but it is designed for this type of thing.

I'm guessing these license models are all because they want to just plainly sell their own instances without cloud providers competing with them directly (who typically have unlimited resources to do so!) or keeping any changes or fixes to themselves (I think that was the reasoning with Mongo?).

Offtopic / Meta to the article itself:

What is with the format of this article, like if they meant for the stuff in brackets to be headings, but chose this format instead of making them headings.

Yes, awesome news.

I made a career with Solr, but building my product, Wide Angle Analytics, on top of Elastic search, made me realize how much more robust and polished ES actually is.

With restoration of Open Source alignment I am confident we will continue building with ES as we are very happy with it.

Honestly at this point I just see little reason to invest in Elastic Search.

I have to wonder how much what is happening with terraform and tofu is related to this.

While I can understand why they went down this path, it burned a bridge and I just don’t know why I would even bother instead of using opensearch.

Confused? Alternatives to Elasticsearch are

- https://www.meilisearch.com/

- https://typesense.org/

- https://manticoresearch.com/

and maybe

- https://github.com/Sygil-Dev/whoosh-reloaded

Nice! Time to switch back real quick.

Despite what they are claiming, they are clearly changing back the license because they felt the pain of not being Open Source anymore.

The proof that companies like that publicly under value the success that they owe to being Open Source.

   But being able to use the term Open Source, by using AGPL, an OSI approved license, removes any questions, or fud, people might have.

Also it makes me laugh so much the guy is trying to victimize himself pretending that they are unjustly targeted by FUD when it is not FUD but a real existing problem.

Does anyone involved with MediaWiki here know how this could affect CirrusSearch? Once ES became non-OS it kinda went to hell. https://www.mediawiki.org/wiki/Extension:CirrusSearch

Looks like Elastic is bowing to OpenSearch threat. This is great news to have more Open Source choices, yet I think for those who care about Open Source the trust have been broken and there is little assurance there will not be a license change one again, serving needs of the moment

Does anyone else find the wording of this is really strange and mildly unprofessional?

I think they still have a contributor licence agreement, meaning every contributor transfers the copyright of his contribution to the owner of Elasticsearch.

To super-simplify things: using the AGPL software, can I offer a competing service to Elastic’s?

If yes, then it is opensource, otherwise it is not.

I'm glad the AGPL is getting some attention; it is the ideal open source license in the age of big tech

Too late, the community all moved over to OpenSearch.

I'll never forgive Elastic for locking basic security features behind their paid licence. Over the years probably millions of people had their data compromised due to that (due to people inadvertently leaving instances on the public internet - having auth enabled by default would have helped a lot)

IMHO, this is just the beginning of the trend for reverting source-available licenses back to open-source licenses. The reason is that source-available licenses don't help increasing revenue growth rate. https://news.ycombinator.com/item?id=41266819

A bit aside but anyone have experience using elastic managed on Azure with a many terabyte+ cluster for prod? Just curious if having less control etc has ever been problem.

But es has fallen behind in vector search

Thank you, ElasticSearch!

I know the AGPL may be a terrible license and all, but it is allowed by Free Software purists. I hope more companies follow suit.

It's an interesting move for sure but I don't think it's enough. A move back to the Apache license would open up the landscape for a reunification with the now dominant fork, which is Opensearch. But that's not going to happen with AGPL + copyright transfers. AGPL + copyright transfers vs. SSPL is a choice between getting stabbed or shot from a legal point of view. It's a hard no either way for a lot of corporate legal departments.

I also don't think this will inspire a lot of companies or developers to start contributing changes to the Elasticsearch code base again; which is something that ground to a halt earlier. I saw my modest contributions under the Apache license being locked up behind this bullshit license and I learned my lesson: I'm never signing another contributor license again. My trust was violated. Not lifting a finger to help them.

Elastic suffered a self inflicted fork of their developer community three years ago and Opensearch has become the default search solution for a lot of developers and companies. Opensearch replaced Elasticsearch as a neutral ground for open source researchers to rally around. I don't see that changing in any material way because of this license change.

It's interesting that they are doing this though because clearly they are feeling the pressure and basically people using the opensource argument was cutting off their stream of new users. I consult in this space and Opensearch has become the default choice for new users. It isn't even close. Why would you pick Elastic as a first time user? They don't even consider Elasticsearch because it's all closed source and proprietary and Opensearch does the job. I don't think this change is enough to change that.

IMHO their next logical step is embracing/acknowledging Opensearch and moving their efforts to join Opensearch and supporting that. That's a huge community of users, developers and companies that's just sitting there without delivering any revenue to Elastic. It's stupid; they are competing with their own product and leaving a lot of money on the table. Elastic has all the core skills to support that community but they are just sitting on their hands now pretending it doesn't exist. They must be starting to feel the pressure to just toss in the towel and grab a chunk of that market. This our way or the highway position sure has resulted in a lot of people choosing the highway.

Petition to make all the official blog post written like this, I love the rap songs reference

I guess it's true. Except you cannot contribute code under AGPL, so it's only open source in name, there is no room for a community to form around the software, Elastic co remains in sole control