Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2915

LinuxTaskController does not work when JniBasedUnixGroupsNetgroupMapping or JniBasedUnixGroupsMapping is enabled

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.205.0
    • 0.20.205.0
    • task-controller
    • None

    Description

      When a job is submitted, LinuxTaskController launches the native task-controller binary for job initialization. The native program does a series of prep work and call execv() to run JobLocalizer. It was observed that JobLocalizer does fails to run when JniBasedUnixGroupsNetgroupMapping or JniBasedUnixGroupsMapping is enabled, resulting in 100% job failures.

      JobLocalizer normally does not need the native library (libhadoop) for its functioning, but enabling a JNI user-to-group mapping function cause it to load the library. However, JobLocalizer cannot locate the library since "java.library.path" is not set.

      The proposed solution is to pass the java.library.path property through task-controller. LinuxTaskController already does it when launching the task log truncater.

      Attachments

        1. mr-2915.patch.txt
          7 kB
          Kihwal Lee
        2. linux_tc_fix.patch.txt
          0.7 kB
          Kihwal Lee

        Activity

          People

            kihwal Kihwal Lee
            kihwal Kihwal Lee
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: