Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2915

LinuxTaskController does not work when JniBasedUnixGroupsNetgroupMapping or JniBasedUnixGroupsMapping is enabled

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0
    • Fix Version/s: 0.20.205.0
    • Component/s: task-controller
    • Labels:
      None

      Description

      When a job is submitted, LinuxTaskController launches the native task-controller binary for job initialization. The native program does a series of prep work and call execv() to run JobLocalizer. It was observed that JobLocalizer does fails to run when JniBasedUnixGroupsNetgroupMapping or JniBasedUnixGroupsMapping is enabled, resulting in 100% job failures.

      JobLocalizer normally does not need the native library (libhadoop) for its functioning, but enabling a JNI user-to-group mapping function cause it to load the library. However, JobLocalizer cannot locate the library since "java.library.path" is not set.

      The proposed solution is to pass the java.library.path property through task-controller. LinuxTaskController already does it when launching the task log truncater.

      1. linux_tc_fix.patch.txt
        0.7 kB
        Kihwal Lee
      2. mr-2915.patch.txt
        7 kB
        Kihwal Lee

        Activity

          People

          • Assignee:
            Kihwal Lee
            Reporter:
            Kihwal Lee
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Due:
              Created:
              Updated:
              Resolved:

              Development