July 11th, 2024

AWS Secrets Manager Agent

The GitHub repository provides an HTTP service for simplifying access to AWS Secrets Manager in various environments. It offers guidance on building, deploying, and using the Secrets Manager Agent, including configuration, logging, and security details.

Read original articleLink Icon
AWS Secrets Manager Agent

The GitHub repository for the AWS Secrets Manager Agent hosts a client-side HTTP service designed to simplify accessing secrets from AWS Secrets Manager in various environments like AWS Lambda, Amazon ECS, Amazon EKS, and Amazon EC2. This service allows applications to retrieve and store secrets in memory, reducing the need for frequent calls to Secrets Manager. The repository offers comprehensive guidance on building, installing, and utilizing the Secrets Manager Agent. It outlines steps for creating the agent binary, deploying it on different platforms (including Amazon EKS, Amazon ECS, Docker, and AWS Lambda), and fetching secrets through the agent. Moreover, the repository addresses configuration settings, logging mechanisms, and security considerations associated with the Secrets Manager Agent. For individuals seeking more information or specific assistance regarding this tool, the repository serves as a valuable resource.

Link Icon 15 comments
By @WatchDog - 4 months
So the point of this is just to cache secrets, to avoid caching them in your app memory?

Seems like kinda a niche threat model, if your app is already compromised to the point where it's secret cache can be read, it seems likely that the attacker could also pivot to just read from the cache, or use the instance credentials to read from secrets manager itself.

By @derefr - 4 months
Why are all the various "secrets vault" approaches so splintered and proprietary, anyway? Why is there a separate tool I have to install for:

• AWS secrets, GCP secrets, Azure secrets... each has its own API

• secrets in a HashiCorp Vault install

• secrets from whatever cloud password manager

• "ambient" secrets from env-vars, or the local .netrc, or the local macOS Keychain

• k8s Secrets resources (when you're a k8s CRD controller)

• secrets stored in SOPS files, in turn encrypted by keys held in any of the above

Why haven't we seen a generic "secrets client" library, with pluggable adapters for handling all of these cases through the same library API / CLI tooling?

Or better yet, why not a generic stub secrets client, that speaks to an also-generic "caching middleware proxy" like this AWS one — where the proxy has the pluggable backend adapters + connection config for them?

By @slaughtr - 4 months
This seems like quite a lot of setup and hassle for what could be handled some other way with less fuss, like chamber[0] or Doppler[1]. Heck, even the classic .env seems like a better choice in every way.

What are the advantages to a configuration like this? Seems the HTTP interface with non-encrypted cache and separate agent situation isn’t something secure enough to satisfy most companies these days.

[0] https://github.com/segmentio/chamber

[1] https://www.doppler.com/

By @thedougd - 4 months
What I really want is a consul-template for AWS Secrets Manager. As I wrote this I googled and found a plugin:

https://github.com/chrissav/consul-template-plugin-secretsma...

I didn't realize consul-template supported plugins.

By @SunnyW - 4 months
For senior developers who are ready to write code, integrating the appropriate AWS SDK library for your programming language and writing a few lines of code might seem straightforward, and may not take more than half a day. However, consider a large company with thousands of applications—like in my case—where this effort is multiplied a thousandfold. Moreover, these applications are developed in over 10 different languages, some of which may not even have an available AWS SDK. Therefore, using an agent that simplifies these operations into a single HTTP call to a sidecar service truly adds value.

Another consideration is operation; imagine that there are 10 different libraries maintained for this purpose, and if there is a new feature, say, you need all logs going to one place, making sure it is available in all languages would require a team with different programming skills to do so. Secrets agent, being language agnostic, you only need to change at one place, and someone else may have already done it for it or ready to do it, as it is open source project.

When it comes to cost saving, imagine scenarios where a junior developer improperly implements secret retrieval in a Lambda function, with retrieval occurring at every function invocation and each function handling 100 transactions per second. Such a single oversight can cost $1,000 a month, and if left unnoticed for a year—a common occurrence when the function appears to work—people often overlook further scrutiny as long as it functions.

By @wrs - 4 months
FYI, there is an AWS-provided Lambda layer similar in principle to this, also including access to Parameter Store.

https://aws.amazon.com/blogs/compute/using-the-aws-parameter...

By @perryizgr8 - 4 months
How is this different from calling Secrets Manager directly? The only benefit I can think of is caching. So your secrets can be fetched a bit faster. But that is such a niche use-case, and you can easily cache it yourself if you need to.
By @micahbule - 4 months
One particular use case that I might try this for is for (very) restrictive environments. One such case was with my previous work where we had to develop services for the client but we can only do it in a remote desktop with certain network and application restrictions. Instead of having conditions for the environment to load certain config, we can simply retrieve the secrets stored in AWS (ex. RDS credentials) via the agent.
By @lijok - 4 months
I'm going to say this as nicely as I can. Secrets Manager can fuck right off with their $.50/mo/secret pricing.

Moved all our secrets to S3 a long time ago and haven't looked back.

By @420official - 4 months
This is really cool, I've been running something similar to simplify rotating database credentials for legacy projects.
By @webprofusion - 4 months
So a bit like Hashicorp Vault (in that it has a locally accessed secrets store) but backed by AWS Secrets Manager.
By @symlinkk - 4 months
Who cares? People are only upvoting this because it’s written in Rust. The actual tool seems useless
By @gtirloni - 4 months
This should come in handy with SOPS and git log.
By @Sparkyte - 4 months
I got to use secrets manager a while back it was a breath of fresh air as it was all of those things you seeking in vault without all of the problems of it being hashicorp. No offense hashicorp. I rather blame AWS than a self-managed solution.
By @shironandonon_ - 4 months
this feels more like Azure Secrets which has been a superior product.