Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1878

Task-specific log level override not working in certain conditions

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5.4
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      When trying to use the feature from TEZ-1775:

      tez.task.log.level is set to "DEBUG;org.apache.hadoop.ipc=INFO;org.apache.hadoop.security=INFO;"

      Running a job with overrides:

      $

      {HADOOP_COMMON_HOME}

      /bin/hadoop jar tez/tez-tests-0.6.0-SNAPSHOT.jar testorderedwordcount -DUSE_MR_CONFIGS=false -DINTER_JOB_SLEEP_INTERVAL=10 -Dtez.task-specific.launch.cmd-opts.list="finalreduce[]" -Dtez.task-specific.log.level="INFO" /tmp/in1 /tmp/out1

      In the above case, I see TEZ_CONTAINER_LOG_PARAMS setup incorrectly for the container in which the final reducer tasks were run.

      1. TEZ-1878.1.txt
        5 kB
        Siddharth Seth

        Activity

        Hide
        hitesh Hitesh Shah added a comment -

        However, it may be helpful to note that when setting "-Dtez.task-specific.log.level="INFO;org.apache.hadoop.ipc=WARN;" - this works correctly.

        Show
        hitesh Hitesh Shah added a comment - However, it may be helpful to note that when setting "-Dtez.task-specific.log.level="INFO;org.apache.hadoop.ipc=WARN;" - this works correctly.
        Hide
        sseth Siddharth Seth added a comment -

        Good catch. Patch to fix this. Hitesh Shah - please review

        Show
        sseth Siddharth Seth added a comment - Good catch. Patch to fix this. Hitesh Shah - please review
        Hide
        hitesh Hitesh Shah added a comment -

        +1

        Show
        hitesh Hitesh Shah added a comment - +1
        Hide
        sseth Siddharth Seth added a comment -

        Thanks for the review. Committed to master and branch-0.6

        Show
        sseth Siddharth Seth added a comment - Thanks for the review. Committed to master and branch-0.6
        Hide
        sseth Siddharth Seth added a comment -

        Committed to branch-0.5.

        Show
        sseth Siddharth Seth added a comment - Committed to branch-0.5.
        Hide
        hitesh Hitesh Shah added a comment -

        Closing issue as 0.5.4, 0.6.1 and 0.7.0 have been released.

        Show
        hitesh Hitesh Shah added a comment - Closing issue as 0.5.4, 0.6.1 and 0.7.0 have been released.

          People

          • Assignee:
            sseth Siddharth Seth
            Reporter:
            hitesh Hitesh Shah
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development