Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-465

Misconfiguration leads to NullPointerException in PostProcessorManager

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: None
    • Component/s: Data Processors
    • Labels:
      None
    • Environment:

      Description

      I didn't know to copy chukwa-demux-conf.xml.template to chukwa-demux-conf.xml and got this error in postprocess.log:

      ----------------------------------------------

      2010-03-16 16:46:18,150 INFO main PostProcessorManager - PostProcess Start, directory:/chukwa/postProcess/demuxOutputDir_1268783176828
      2010-03-16 16:46:18,151 ERROR main PostProcessorManager - java.lang.NullPointerException
      at org.apache.hadoop.chukwa.extraction.demux.PostProcessorManager.processDemuxPigOutput(PostProcessorManager.java:173)
      at org.apache.hadoop.chukwa.extraction.demux.PostProcessorManager.start(PostProcessorManager.java:145)
      at org.apache.hadoop.chukwa.extraction.demux.PostProcessorManager.main(PostProcessorManager.java:80)

      2010-03-16 16:46:18,151 WARN main PostProcessorManager - Error in processDemuxOutput for :/chukwa/postProcess/demuxOutputDir_1268783176828
      2010-03-16 16:46:18,175 WARN main PostProcessorManager - Error in postProcess :/chukwa/postProcess/demuxOutputDir_1268783176828 has been moved to:/chukwa/postProcessInError//demuxOutputDir_1268783176828_1268783178154

      ----------------------------------------------

      The problem is here:

      String[] classes = conf.get(POST_DEMUX_DATA_LOADER).split(",");

      In my case (again, due to bad configuration) that configuration file is missing and so we end up trying to split through a null reference.

      Note: I got this code from trunk, revision 924074.

        Activity

        Hide
        asrabkin Ari Rabkin added a comment -

        Good catch. The right fix, I think, is to add a default value for that field.

        Show
        asrabkin Ari Rabkin added a comment - Good catch. The right fix, I think, is to add a default value for that field.
        Hide
        afathalla Ahmed Fathalla added a comment -

        Fixed the problem by changing line

        String[] classes = conf.get(POST_DEMUX_DATA_LOADER).split(",");

        to

        String[] classes = conf.get(POST_DEMUX_DATA_LOADER,"org.apache.hadoop.chukwa.dataloader.MetricDataLoaderPool,org.apache.hadoop.chukwa.dataloader.FSMDataLoader").split(",");

        Show
        afathalla Ahmed Fathalla added a comment - Fixed the problem by changing line String[] classes = conf.get(POST_DEMUX_DATA_LOADER).split(","); to String[] classes = conf.get(POST_DEMUX_DATA_LOADER,"org.apache.hadoop.chukwa.dataloader.MetricDataLoaderPool,org.apache.hadoop.chukwa.dataloader.FSMDataLoader").split(",");
        Hide
        afathalla Ahmed Fathalla added a comment -

        Here is the patch file.

        Show
        afathalla Ahmed Fathalla added a comment - Here is the patch file.
        Hide
        eyang Eric Yang added a comment -

        +1 looks good.

        Show
        eyang Eric Yang added a comment - +1 looks good.
        Hide
        eyang Eric Yang added a comment -

        I just committed this, thanks Ahmed.

        Show
        eyang Eric Yang added a comment - I just committed this, thanks Ahmed.
        Hide
        hudson Hudson added a comment -

        Integrated in Chukwa-trunk #343 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/343/)
        CHUKWA-465. Setup default value for Post Demux Data Loader. (Ahmed Fathalla via Eric Yang)
        CHUKWA-465. Setup default value for Post Demux Data Loader. (Ahmed Fathalla via Eric Yang)

        Show
        hudson Hudson added a comment - Integrated in Chukwa-trunk #343 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/343/ ) CHUKWA-465 . Setup default value for Post Demux Data Loader. (Ahmed Fathalla via Eric Yang) CHUKWA-465 . Setup default value for Post Demux Data Loader. (Ahmed Fathalla via Eric Yang)

          People

          • Assignee:
            afathalla Ahmed Fathalla
            Reporter:
            kirktrue Kirk True
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development