Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3344

Fix warning - procfs stat file is not in the expected format

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Although this doesn't appear to be causing any functional issues, it is spamming our log files quite a bit.

      It appears that the regex in ProcfsBasedProcessTree doesn't work for all /proc/<pid>/stat files.

      Here's the error I'm seeing:

          "source_host": "asdf",
          "method": "constructProcessInfo",
          "level": "WARN",
          "message": "Unexpected: procfs stat file is not in the expected format for process with pid 6953"
          "file": "ProcfsBasedProcessTree.java",
          "line_number": "514",
          "class": "org.apache.hadoop.yarn.util.ProcfsBasedProcessTree",
      

      And here's the basic info on process with pid 6953:

      [asdf ~]$ cat /proc/6953/stat
      6953 (python2.6 /expo) S 1871 1871 1871 0 -1 4202496 9364 1080 0 0 25 3 0 0 20 0 1 0 144918696 205295616 5856 18446744073709551615 1 1 0 0 0 0 0 16781312 2 18446744073709551615 0 0 17 13 0 0 0 0 0
      [asdf ~]$ ps aux|grep 6953
      root      6953  0.0  0.0 200484 23424 ?        S    21:44   0:00 python2.6 /export/apps/salt/minion-scripts/module-sync.py
      jbringhu 13481  0.0  0.0 105312   872 pts/0    S+   22:13   0:00 grep -i 6953
      [asdf ~]$ 
      

      This is using 2.6.32-431.11.2.el6.x86_64 in RHEL 6.5.

        Attachments

        1. YARN-3344.06.patch
          5 kB
          Akira Ajisaka
        2. YARN-3344.07.patch
          5 kB
          Akira Ajisaka
        3. YARN-3344.08.patch
          5 kB
          Akira Ajisaka
        4. YARN-3344.09.patch
          8 kB
          Akira Ajisaka
        5. YARN-3344-trunk.005.patch
          5 kB
          Ravindra Kumar Naik

          Activity

            People

            • Assignee:
              ravindra.naik Ravindra Kumar Naik
              Reporter:
              jonbringhurst Jon Bringhurst
            • Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: