Skip to content

Welcome to UniQual

UniQual is a unified platform for Data Quality testing, Observability and Reporting built using several OSS tools.

Ideology

We see data tests are often performed in silos (at pipeline/notebook level) and getting answers for below is difficult

  1. Who did the test?
  2. When was the last test performed?
  3. What was the data quality status before?
  4. Data Quality report on Project\Asset group level
  5. How does the downstream pipeline/tool know the status of its preceding pipeline?
  6. Two or more tools required to understand/perform data quality
  7. Data quality results/reports are not query friendly (eg: great expectations results are saved as HTML)
  8. Custom coding required for notifications or depends on platforms capabilities
  9. Data Quality tools dont use open table formats(Delta/Iceberg/Hudi etc,.)
  10. Separate tools required for monitoring Data Platform

Solution approach

  1. Asset Centric - Treat tables, pipelines etc., as assets which enables business to deliver/decide/execute
  2. Simplified Results- Establish framework/modules to make data quality tests easier and report friendly using open table formats

Helps in

  1. Observability from Day 1
  2. Report friendly results
  3. Data Quality & Documentation as a Code

Tools used

---
config:
  theme: mc
---
graph TD
    A[UniQual] --> B[Kestra]
    B -->|Orchestration| C[Data Workflow Management]
    A --> D[Cuallee]
    D -->|Data Quality| E[Checks & Tests]
    A --> F[Mkdocs]
    F -->|Documentation| G[Project & Technical Docs]
    style A fill:#4a90e2
    style B fill:#2ecc71
    style D fill:#e74c3c
    style F fill:#f39c12

Features by Platform

Platform Freshness Volume
Snowflake Yes Yes