Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-440

Custom processor classes not detected unless added to chukwa-core jar

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Data Processors
    • Labels:
      None

      Description

      http://wiki.apache.org/hadoop/DemuxModification

      After implementing a custom parser as shown in the wiki link above and mapping it to a data type in chukwa-demux.xml, there's no easy way to register the class so it gets included in the job that's submitted to the hadoop cluster. I've added my jar containing the class to the lib/ directory of the chukwa data processor install and verified that it's in the classapath, but it's not submitted with the Hadoop job. On the Hadoop cluster ClassNotFoundExceptions are thrown in my mapper.

      The only way I've been able to make this work is to do the following:
      1. put my class in the package org.apache.hadoop.chukwa.extraction.demux.processor.mapper
      2. then manually add that class to the chukwa-core-0.3.0.jar that is on my data processor

      Instead, the class should be detected from whatever jar it lives in that's in the demux class path, regardless of the package it lives in.

        Attachments

        1. CHUKWA-440-2.patch
          4 kB
          Eric Yang
        2. CHUKWA-440-1.patch
          4 kB
          Eric Yang
        3. CHUKWA-440.patch
          2 kB
          Eric Yang

          Activity

            People

            • Assignee:
              eyang Eric Yang
              Reporter:
              billgraham Bill Graham
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: