Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1104

TezProcessor.setupMRLegacyConfigs() creates task IDs which are not valid Hadoop TaskAttemptID strings

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: 0.4.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      Hadoop 2.2.0

      Description

      When executing MR jobs via Tez, the values assigned to mapred.task.id and mapreduce.task.id cause exceptions to be thrown if these values are passed to TaskAttempID.forName():

      	java.lang.IllegalArgumentException: TaskAttemptId string : task1396892688715_80817_m_000076_3 is not properly formed
      		at org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:201)
      		at org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.<init>(ParquetRecordWriterWrapper.java:49)
      

      Perhaps TezProcessor should start with:

      StringBuilder taskAttemptIdBuilder = new StringBuilder("attempt_");
      

      instead of

      StringBuilder taskAttemptIdBuilder = new StringBuilder("task");
      

      ?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ccondit Craig Condit
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: