Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5715

ProcfsBasedProcessTree#constructProcessInfo() can still throw NumberFormatException

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.2.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Ubuntu 13.04 (OS Kernel 3.9.0), Armv71Exynos5440

      Description

      For long running jobs I have hit an issue that seems to be to be similar to the bug reported in https://issues.apache.org/jira/browse/MAPREDUCE-3583

      Unfortunately I do not have the OS logs for this issue, but the utime for the application was read by Hadoop as "184467440737095551615" which does not fit into a Long. In MAPREDUCE-3583 a change was made to ProcfsBasedProcessTree.java
      in order to support larger values for stime. Perhaps we need to support larger values for utime (although this could increase the complexity of the math that is being performed on those numbers)

      1. constructprocessfailing.jpg
        155 kB
        German Florez-Larrahondo

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          It looks like that the exception was from parsing the utime below.

                  // Set (name) (ppid) (pgrpId) (session) (utime) (stime) (vsize) (rss)
                  pinfo.updateProcessInfo(m.group(2), m.group(3),
                          Integer.parseInt(m.group(4)), Integer.parseInt(m.group(5)),
                          Long.parseLong(m.group(7)), new BigInteger(m.group(8)),
                          Long.parseLong(m.group(10)), Long.parseLong(m.group(11)));
          
          Show
          Tsz Wo Nicholas Sze added a comment - It looks like that the exception was from parsing the utime below. // Set (name) (ppid) (pgrpId) (session) (utime) (stime) (vsize) (rss) pinfo.updateProcessInfo(m.group(2), m.group(3), Integer .parseInt(m.group(4)), Integer .parseInt(m.group(5)), Long .parseLong(m.group(7)), new BigInteger(m.group(8)), Long .parseLong(m.group(10)), Long .parseLong(m.group(11)));

            People

            • Assignee:
              Unassigned
              Reporter:
              German Florez-Larrahondo
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development