Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-6685

Add ThresholdCheckOperator for Data Quality Checking

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 2.0.0
    • Fix Version/s: 1.10.10
    • Component/s: operators
    • Labels:
      None

      Description

      This PR includes a new operator in CheckOperator that allows users to perform a threshold data quality check.

      ThresholdCheckOperator will check a single value, sql result against a threshold range, and will fail a task if it is outside this range. The lower and upper bound of the threshold can be defined as either a numeric values, or sql-statements that returns a numeric value.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                alue alex l
                Reporter:
                alue alex l
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: