Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12406

AbstractMapWritable.readFields throws ClassNotFoundException with custom writables

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.7.1
    • 2.8.0, 2.7.3, 3.0.0-alpha1
    • io
    • None
    • Ubuntu Linux 14.04 LTS amd64

    • Reviewed
    • Patch

    Description

      Note: I am not an expert at JAVA, Class loaders, or Hadoop. I am just a hacker. My solution might be entirely wrong.

      AbstractMapWritable.readFields throws a ClassNotFoundException when reading custom writables. Debugging the job using remote debugging in IntelliJ revealed that the class loader being used in Class.forName() is different than that used by the Thread's current context (Thread.currentThread().getContextClassLoader()). The class path for the system class loader does not include the libraries of the job jar. However, the class path for the context class loader does. The proposed patch changes the class loading mechanism in readFields to use the Thread's context class loader instead of the system's default class loader.

      Attachments

        1. HADOOP-12406.1.patch
          2 kB
          Vinod Kumar Vavilapalli
        2. HADOOP-12406.patch
          1 kB
          Nadeem Douba

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ndouba Nadeem Douba
            ndouba Nadeem Douba
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment