Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-863

NullPointerException in shuffle phase

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.3.0
    • 0.3.0
    • None
    • None
    • Hadoop 2.3.0, Hive 0.13, Tez 0.3.0

    Description

      When running amplab-benchmark (query 2a, 2b, 2c), randomly the following exception is thrown. It is not very consistent though.

      Amplab query reference: https://amplab.cs.berkeley.edu/benchmark/
      >>>
      SELECT SUBSTR(sourceIP, 1, 10), SUM(adRevenue) FROM uservisits GROUP BY SUBSTR(sourceIP, 1, 10)
      >>>

      Vertex re-running, vertexName=Map 1, vertexId=vertex_1392875986731_0052_2_01
      Vertex failed, vertexName=Reducer 2, vertexId=vertex_1392875986731_0052_2_00, diagnostics=[Task failed, taskId=task_1392875986731_0052_2_00_000030, diagnostics=[AttemptID:attempt_1392875986731_0052_2_00_000030_0 Info:Error: java.lang.NullPointerException
      at org.apache.tez.runtime.library.common.shuffle.impl.Shuffle.handleEvents(Shuffle.java:175)
      at org.apache.tez.runtime.library.input.ShuffledMergedInput.handleEvents(ShuffledMergedInput.java:195)
      at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:566)
      at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$1100(LogicalIOProcessorRuntimeTask.java:81)
      at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.run(LogicalIOProcessorRuntimeTask.java:628)
      at java.lang.Thread.run(Thread.java:744)

      Attachments

        1. TEZ-863.addon-1.txt
          2 kB
          Rajesh Balamohan
        2. TEZ-863.addon.txt
          1 kB
          Rajesh Balamohan
        3. TEZ-863.1.txt
          5 kB
          Siddharth Seth
        4. syslog_attempt_1392875986731_0052_2_00_000030_0.log
          15 kB
          Gopal Vijayaraghavan

        Activity

          People

            rajesh.balamohan Rajesh Balamohan
            rajesh.balamohan Rajesh Balamohan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: