Chukwa
  1. Chukwa
  2. CHUKWA-565

HbaseWriter not compatible with TsProcessor

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The TsProcessor class used Demux.jobConf to obtain it's configurations, which HBaseWriter does not initialize when processors are used in the collector. HBseWriter should initialize Demux.jobConf with the configuration object that it uses.

      1. CHUKWA-565-1.patch
        6 kB
        Bill Graham
      2. CHUKWA-565_2.patch
        9 kB
        Bill Graham
      3. CHUKWA-565_3.patch
        8 kB
        Bill Graham
      4. CHUKWA-565_4.patch
        9 kB
        Eric Yang

        Activity

        Hide
        Hudson added a comment -

        Integrated in Chukwa-trunk #463 (See https://builds.apache.org/job/Chukwa-trunk/463/)
        CHUKWA-565. Added support HBaseWriter support for TsProcessor. (Bill Graham via Eric Yang) (Revision 1424228)

        Result = SUCCESS
        eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424228
        Files :

        • /incubator/chukwa/trunk/CHANGES.txt
        • /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/hbase/HBaseWriter.java
        • /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/Demux.java
        • /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TsProcessor.java
        Show
        Hudson added a comment - Integrated in Chukwa-trunk #463 (See https://builds.apache.org/job/Chukwa-trunk/463/ ) CHUKWA-565 . Added support HBaseWriter support for TsProcessor. (Bill Graham via Eric Yang) (Revision 1424228) Result = SUCCESS eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1424228 Files : /incubator/chukwa/trunk/CHANGES.txt /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/hbase/HBaseWriter.java /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/Demux.java /incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TsProcessor.java
        Hide
        Eric Yang added a comment -

        I just committed this. Thanks Bill.

        Show
        Eric Yang added a comment - I just committed this. Thanks Bill.
        Hide
        Eric Yang added a comment -

        I have a updated version locally to fix the initialization problem, which I can submit for review. The test case passes fine.

        Show
        Eric Yang added a comment - I have a updated version locally to fix the initialization problem, which I can submit for review. The test case passes fine.
        Hide
        Bill Graham added a comment -

        Merge problems, good catch. I'm glad to update the patch but the more I think about it I'm uneasy with this entire change, mainly because of how old it is and because I can't test it. For that reason and the fact that no one seems to be bothered by this issue I suggest we cancel this JIRA.

        If others are more confident in this chance though and have a way to regress it, I'll gladly submit a rev to this patch.

        Show
        Bill Graham added a comment - Merge problems, good catch. I'm glad to update the patch but the more I think about it I'm uneasy with this entire change, mainly because of how old it is and because I can't test it. For that reason and the fact that no one seems to be bothered by this issue I suggest we cancel this JIRA. If others are more confident in this chance though and have a way to regress it, I'll gladly submit a rev to this patch.
        Hide
        Eric Yang added a comment -

        Bill, HBaseWriter private variable ChukwaConfiguration is not initialized before conf.get is called in the next line.

        Show
        Eric Yang added a comment - Bill, HBaseWriter private variable ChukwaConfiguration is not initialized before conf.get is called in the next line.
        Hide
        Bill Graham added a comment -

        Here's an updated patch.

        Show
        Bill Graham added a comment - Here's an updated patch.
        Hide
        Bill Graham added a comment -

        Ugh, indeed. I can take a look and work on an updated patch, sure. I don't think I'll be able to test it though unfortunately since I no longer have a running Chukwa environment.

        Show
        Bill Graham added a comment - Ugh, indeed. I can take a look and work on an updated patch, sure. I don't think I'll be able to test it though unfortunately since I no longer have a running Chukwa environment.
        Hide
        Ari Rabkin added a comment -

        Ugh. This patch has rotted just a bit. I think it's just cosmetic, but I don't trust myself to revise it. Bill, I feel bad that we let this slip a year [!], but can you regenerate?

        Show
        Ari Rabkin added a comment - Ugh. This patch has rotted just a bit. I think it's just cosmetic, but I don't trust myself to revise it. Bill, I feel bad that we let this slip a year [!] , but can you regenerate?
        Hide
        Bill Graham added a comment -

        Sure, here's a patch that works on the trunk.

        Show
        Bill Graham added a comment - Sure, here's a patch that works on the trunk.
        Hide
        Eric Yang added a comment -

        Bill, the patch is staled. The proposed changes looks good, could you generate the patch for the current trunk? Thanks

        Show
        Eric Yang added a comment - Bill, the patch is staled. The proposed changes looks good, could you generate the patch for the current trunk? Thanks
        Hide
        Bill Graham added a comment -

        Attaching patch 1.

        Show
        Bill Graham added a comment - Attaching patch 1.

          People

          • Assignee:
            Bill Graham
            Reporter:
            Bill Graham
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development