We’re thrilled to announce that Atlas Stream Processing—the MongoDB-native way to process streaming data—is now generally available, empowering developers to quickly build responsive, event-driven applications!
Our team spent the last two years defining a vision and building a product that leans into MongoDB’s strengths to overcome the hard challenges in stream processing. After a decade of building stream processing products outside of MongoDB, we are using everything that makes MongoDB unique and differentiated—the Query API and powerful aggregation framework, as well as the document model and its schema flexibility—to create an awesome developer experience.
It’s a new approach to stream processing, and based on the feedback of so many of you in our community, it’s the best way for most developers using MongoDB to do it.
Let’s get into what’s new.
What’s new in general availability?
Production Readiness
Ready to support your production workloads, ensuring reliable and scalable stream processing for your mission-critical applications.
Time Series Collection Support
Emit processor results into Time Series Collections. Pre-process data continuously while saving it for historical access later in a collection type available in MongoDB Atlas built to efficiently store and query time series data.
Development and Production Tiers
Besides the SP30 cluster tier available during the public preview, we’re introducing an SP10 tier to provide flexibility and a cost-effective option for exploratory use cases and low-traffic stream processing workloads.
Improved Kafka Support
Added support for Kafka headers allows applications to provide additional metadata alongside event data. They are helpful for various stream processing use cases (e.g., routing messages, conditional processing, and more).
Least Privilege Access
Atlas Database Users can grant access to Stream Processing Instances and enable access to only those who need it. Read our tutorial for more information.
Stream Processor Alerting
Gain insight and visibility into the health of your stream processors by creating alerts for when a failure occurs. Supported methods for alerting include email, SMS, monitoring platforms like Datadog, and more.
Why Atlas Stream Processing?
Atlas Stream Processing brings the power and flexibility of MongoDB’s document model and Query API to the challenging stream processing space. With Atlas Stream Processing, developers can:
Effortlessly handle complex and rapidly changing data structures
Use the familiar MongoDB Query API for processing streaming data
Seamlessly integrate with MongoDB Atlas
Benefit from a fully managed service that eliminates operational overhead
Customer highlights
Read what developers are saying about Atlas Stream Processing:
At Acoustic, our key focus is to empower brands with behavioral insights that enable them to create engaging, personalized customer experiences. To do so, our Acoustic Connect platform must be able to efficiently process and manage millions of marketing, behavioral, and customer signals as they occur. With Atlas Stream Processing, our engineers can leverage the skills they already have from working with data in Atlas to process new data continuously, ensuring our customers have access to real-time customer insights.
John Riewerts, EVP, Engineering at Acoustic
Atlas Stream Processing enables us to process, validate, and transform data before sending it to our messaging architecture in AWS powering event-driven updates throughout our platform. The reliability and performance of Atlas Stream Processing has increased our productivity, improved developer experience, and reduced infrastructure cost.
Cody Perry, Software Engineer, Meltwater
What’s ahead for Atlas Stream Processing?
We’re rapidly introducing new features and functionality to ensure MongoDB delivers a world-class stream processing experience for all development teams. Over the next few months, you can expect to see:
Advanced Networking Support
Support for VPC Peering to Kafka Clusters for teams requiring additional networking capabilities
Expanded Cloud Region Support
Support for all cloud regions available in Atlas Data Federation
Expanded Cloud Provider Support
Support for Microsoft Azure
Expanded Data Source and Sink Support
We have plans to expand beyond Kafka and Atlas databases in the coming months. Let us know which sources and sinks you need, and we will factor that into our planning
Richer Metrics & Observability
Support for expanded visibility into your stream processors to help simplify monitoring and troubleshooting
Expanded Deployment Flexibility
Support for deploying stream processors with Terraform. This integration will help to enable a seamless CI/CD pipeline, enhancing operational efficiency with infrastructure as code. Look out for a dedicated blog in the near future on how to get started with Atlas Stream Processing and Terraform.
So whether you’re looking to process high-velocity sensor data, continuously analyze customer data to deliver personalized experiences, or perform predictive maintenance to increase yields and reduce costs, Atlas Stream Processing has you covered. Join the hundreds of development teams already building with Atlas Stream Processing. Stay tuned to hear more from us soon, and good luck building!
Login today or check out our introductory tutorial to get started.
Source: Read More
