Uploaded image for project: 'OODT'
  1. OODT
  2. OODT-215 Workflow2 Architecture
  3. OODT-202

NonBlockWorkflow double processing and not pausing

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: None
    • Fix Version/s: 0.5
    • Component/s: workflow manager
    • Labels:
      None
    • Environment:

      from JPL's last internal WM release JIRA

      Description

      Here is the case i ran:

      4 Workflows:

      MoaOrbits : consists of 1 task

      {MoaOrbits}

      MoaIasiGPolygonMap : consists of 2 task

      {MoaIasiGPolygon, MoaIasiMap}

      MoaAmsuaGPolygonMap : consists of 2 task

      {MoaAmsuaGPolygon, MoaAmsuaMap}

      MoaMhsGPolygonMap : consists of 2 task

      {MoaMhsGPolygon, MoaMhsMap}

      I summit MoaOrbits workflow. On ingest of its primary product MOA_ORBITS, it summits (via a cas-crawler action) all 3 GPolygonMap workflows in the order given above.
      None of the GPolygonMap workflows are related to each other in any other way.

      Required Task Input:

      MoaOrbit task: a days worth of IASI files
      GPolygon tasks: the Orbit file created by MoaOrbits task and a days worth of the appropriate METOPA L1 files (IASI | AMSUA | MHS)
      Map tasks: the GPolygon file created by the GPolgon task in the same workflow

      This is how it is expected to run and currently runs with standard workflow engine:

      MoaOrbits run successfully and ingests a MOA_ORBITS product
      MoaIasiGPolygonMap runs successfully ingesting a MOA_IASI_GPOLYGON product and a MOA_IASI_MAP product
      MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and thus the MoaAmsuaMap task fails to ever pass conditions to run
      MoaMhsGPolygonMap runs successfully ingesting a MOA_MHS_GPOLYGON product and a MOA_MHS_MAP product

      This is how it runs with non-blocking workflow engine:

      MoaOrbits run successfully and ingests a MOA_ORBITS product
      MoaIasiGPolygonMap runs TWICE successfully ingesting 2 MOA_IASI_GPOLYGON products and 2 MOA_IASI_MAP products
      MoaAmsuaGPolygonMap fails when generating a MOA_AMSUA_GPOLYGON product (which is not ingested) and the MoaAmsuaMap task fails to ever pass preconditions to run and
      instead of waiting a few seconds between checking conditions it dumps to the logger this message many times a second: INFO: Task: [MoaAmsuaMap] has no required metadata fields
      MoaMhsGPolygonMap runs TWICE successfully ingesting 2 MOA_MHS_GPOLYGON products and 2 MOA_MHS_MAP products

      The short of it seems like workflows with > 1 task get run twice and workflows with task that block have no wait time between rechecking conditions

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                chrismattmann Chris A. Mattmann
                Reporter:
                bfoster Brian Foster
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: