Automating Data Quality Monitoring: Going Deeper Than Data Observability
Author: Jeremy Stanley (Author), Paige Schwartz (Author)
Publisher finelybook 出版社: Oreilly & Associates Inc
Edition 版次: 1st
Publication Date 出版日期: 2024-02-13
Language 语言: English
Print Length 页数: 170 pages
ISBN-10: 1098145933
ISBN-13: 9781098145934
Book Description
The world’s businesses ingest a combined 2.5 quintillion bytes of data every day. But how much of this vast amount of data–used to build products, power AI systems, and drive business decisions–is poor quality or just plain bad? This practical book shows you how to ensure that the data your organization relies on contains only high-quality records.
Most data engineers, data analysts, and data scientists genuinely care about data quality, but they often don’t have the time, resources, or understanding to create a data quality monitoring solution that succeeds at scale. In this book, Jeremy Stanley and Paige Schwartz from Anomalo explain how you can use automated data quality monitoring to cover all your tables efficiently, proactively alert on every category of issue, and resolve problems immediately.
This book will help you:
- Learn why data quality is a business imperative
- Understand and assess unsupervised learning models for detecting data issues
- Implement notifications that reduce alert fatigue and let you triage and resolve issues quickly
- Integrate automated data quality monitoring with data catalogs, orchestration layers, and BI and ML systems
- Understand the limits of automated data quality monitoring and how to overcome them
- Learn how to deploy and manage your monitoring solution at scale
- Maintain automated data quality monitoring for the long term
About the Author
Paige Schwartz is a professional technical writer at Anomalo who has written for clients such as Airbnb, Grammarly, and OpenAI. She specializes in communicating complex software engineering topics to a general audience and has spent her career working with machine learning and data systems, including 5 years as a product manager on Google Search. She holds a joint BA in Computer Science and English from UC Berkeley.
