Posted On: Dec 7, 2021
AWS Glue streaming extract, transform, and load (ETL) jobs can now read from AWS Glue Data Catalog tables created using the AWS Glue Schema Registry. With streaming ETL in AWS Glue, you can set up continuous ingestion pipelines to prepare streaming data on the fly and make it available for analysis in seconds. The AWS Glue Schema Registry allows you to centrally discover, control, and evolve data stream schemas. This integration streamlines the job setup process and simplifies schema enforcement.
Customers can use this feature to manage and enforce the schema of streaming event data like IoT event streams, clickstreams, and network logs, then quickly set up ETL jobs to process the streams. They can clean and transform those data streams in-flight, and continuously load the results into data stores for analytics.
This feature is available in the same AWS Regions as AWS Glue.
To learn more, visit our documentation.