Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-690 Tez API Ease of Use
  3. TEZ-757

Allow the DAG configuration to be available as a payload for user code

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      A number of user pluggable code is/will use standard hadoop configuration as their means of configuration. If the DAG's (not AM) configuration is available as a byte[] payload in the appContext then it may be used as payload for user code whenever user payload is not specified. This will significantly reduce the configuration overhead and amount of code needed to write Tez jobs. Users can configure everything in their job using their local site.xml.
      We could also look at creating a UserPayload object that as 2 members. 1) actual user payload byte[] if set 2) DAG configuration.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              bikassaha Bikas Saha
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: