Uploaded image for project: 'Apache Airflow'
  1. Apache Airflow
  2. AIRFLOW-19

How can I have an Operator B iterate over a list returned from upstream by Operator A?

VotersWatch issueWatchersLinkUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Not A Bug
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: operators
    • Labels:

      Description

      Here is what I am trying to do exactly: https://gist.github.com/praveev/7b93b50746f8e965f7139ecba028490a

      the python operator log just returns the following

      [2016-04-28 11:56:22,296]

      {models.py:1041}

      INFO - Executing <Task(PythonOperator): run_distcp_on_each> on 2016-04-28 11:56:12
      [2016-04-28 11:56:22,350]

      {python_operator.py:66}

      INFO - Done. Returned value was: None

      it didn't even print my kwargs and to_process data

      To simplify this. Lets say t1 returns 3 elements. I want to iterate over the list and run t2 -> t3 for each element.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              praveev Praveenkumar Venkatesan

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment