Saving $10k/Month on Analytics – Snowplow Serverless Alternative
Agon Data has implemented a serverless analytics solution, saving $10,000 monthly, ensuring data ownership, and utilizing tools like Buz and AWS Kinesis, with plans for optimization and potential open-sourcing.
Read original articleAgon Data has successfully implemented a serverless alternative to traditional analytics solutions like Snowplow, resulting in significant cost savings of approximately $10,000 per month. The shift to a cloud-native, serverless architecture has provided a scalable and flexible analytics platform while ensuring full data ownership. The team noted that many existing analytics providers have increased their prices due to organizational bloat and high operational costs, making them less viable for smaller organizations. Their new infrastructure leverages tools such as Buz for data collection, AWS Kinesis for streaming, and Benthos for data movement, all while maintaining compatibility with the Snowplow data model. The deployment process has been streamlined using SST, allowing for automated infrastructure management and schema updates. Initial costs for the new setup are projected to be under $100 per month, a stark contrast to previous expenses. The team expresses gratitude for the open-source community, which has facilitated the development of their data pipeline. Future plans include optimizing the system further and potentially open-sourcing their solution for broader use.
- Agon Data has reduced analytics costs by approximately $10,000 per month with a serverless solution.
- The new infrastructure ensures data ownership and scalability while maintaining compatibility with the Snowplow data model.
- Tools like Buz, AWS Kinesis, and Benthos are utilized for data collection and movement.
- The deployment process is automated using SST, enhancing efficiency and reliability.
- Future plans include further optimization and potential open-sourcing of their data stack.
Related
Our great database migration
Shepherd, an insurance pricing company, migrated from SQLite to Postgres to boost performance and scalability for their pricing engine, "Alchemist." The process involved code changes, adopting Neon database, and optimizing performance post-migration.
Datadog Is the New Oracle
Datadog faces criticism for high costs and limited access to observability features. Open Source tools like Prometheus and Grafana are gaining popularity, challenging proprietary platforms. Startups aim to offer affordable alternatives, indicating a shift towards mature Open Source observability platforms.
How to save $13.27 on your SaaS bill
The author discusses managing costs with Vercel's analytics, converting images to reduce charges, and building a custom API using SQLite. They faced deployment challenges but plan future enhancements.
Agnost is an open source GitOps platform running on Kubernetes clusters
Agnost is an open-source platform for cloud-agnostic deployments, simplifying application management and infrastructure setup while avoiding vendor lock-in. It integrates with Git providers and supports easy self-hosting.
Reclaim the Stack
A team at mynewsdesk.com developed a Kubernetes-based platform to replace Heroku, achieving 90% cost reduction and 30% performance improvement, while enhancing developer experience and open-sourcing the entire stack.
The flow is:
https request to fastly
Synthetic request that logs to fastlylogs
Ship logfile to s3
Process with lambda
This is basically freeish except for the RDS instance.
Related
Our great database migration
Shepherd, an insurance pricing company, migrated from SQLite to Postgres to boost performance and scalability for their pricing engine, "Alchemist." The process involved code changes, adopting Neon database, and optimizing performance post-migration.
Datadog Is the New Oracle
Datadog faces criticism for high costs and limited access to observability features. Open Source tools like Prometheus and Grafana are gaining popularity, challenging proprietary platforms. Startups aim to offer affordable alternatives, indicating a shift towards mature Open Source observability platforms.
How to save $13.27 on your SaaS bill
The author discusses managing costs with Vercel's analytics, converting images to reduce charges, and building a custom API using SQLite. They faced deployment challenges but plan future enhancements.
Agnost is an open source GitOps platform running on Kubernetes clusters
Agnost is an open-source platform for cloud-agnostic deployments, simplifying application management and infrastructure setup while avoiding vendor lock-in. It integrates with Git providers and supports easy self-hosting.
Reclaim the Stack
A team at mynewsdesk.com developed a Kubernetes-based platform to replace Heroku, achieving 90% cost reduction and 30% performance improvement, while enhancing developer experience and open-sourcing the entire stack.