Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-684 Uber/Local modes for Tez
  3. TEZ-1538

Tez Local Mode fails if "fs.defaultFS" is not set to be "file:///' in core-site.xml

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 0.5.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      When users submit their DAG, TezClient will create a submission context. It
      also create a tez-conf.pb file in staging directory. Once DAGAppMaster starts,
      it reads user's configuration from tez-conf.pb file. However, tez-conf.pb file
      is created by TezClientUtils.createFinalTezConfForApp() which will filter
      configurations that are not starting with "tez.".

      Then "fs.defaultFS" will be filtered even user sets it in tez-site.xml and user source code. And DAGAppMaster can not pick this configuration from tez-conf.pb file. Then it always contacts localhost:8020 and fails.

        Attachments

        1. TEZ-1538.patch
          0.8 kB
          Chen He

          Issue Links

            Activity

              People

              • Assignee:
                airbots Chen He
                Reporter:
                airbots Chen He
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: