Automated data quality suggestions and analysis with Deequ on AWS Glue
-
Updated
Dec 29, 2022 - Scala
Automated data quality suggestions and analysis with Deequ on AWS Glue
SparkConnect Server plugin and protobuf messages for the Amazon Deequ Data Quality Engine.
deequ.NET is a port of the awslabs/deequ library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
This project is an end-to-end ETL solution for processing and analyzing US flight data with Apache Spark and Iceberg. It features pipeline unit tests, data quality checks, and an interactive dashboard. Explore it below! ↓
Deequ incremental metrics implementation in python
"Ingest Data" and start cleaning it up as well as delving a little into Deequ
To associate your repository with the deequ topic, visit your repo's landing page and select "manage topics."