Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-741

Update to Hadoop 2.6.0 and HBase 1.0.0

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.0
    • Component/s: Build and Test Code
    • Labels:
      None

      Description

      Update Chukwa to support Hadoop 2.6.0 and HBase 1.0.0. We use some internal Hadoop 1.x internal structure, such as JobHistory log, and parser class associated with it. Those parts are not going to work with Hadoop 2.x. We will disable JobHistory parsing until better analytics can be done for YARN generated log files.

        Activity

        Hide
        eyang Eric Yang added a comment -

        Update to Hadoop 2.6.0 and HBase 1.0.0.
        Disabled TestDemuxManager, it will be replaced by Oozie workflow.
        Deleted JobLog class because Hadoop 2.x doesn't have JobHistory parser class.
        Deleted ChukwaJobTrackerInstrumentation class because there is no JobTracker any more.

        Show
        eyang Eric Yang added a comment - Update to Hadoop 2.6.0 and HBase 1.0.0. Disabled TestDemuxManager, it will be replaced by Oozie workflow. Deleted JobLog class because Hadoop 2.x doesn't have JobHistory parser class. Deleted ChukwaJobTrackerInstrumentation class because there is no JobTracker any more.
        Hide
        eyang Eric Yang added a comment -

        I just committed this.

        Show
        eyang Eric Yang added a comment - I just committed this.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1665125 from Eric Yang in branch 'chukwa/trunk'
        [ https://svn.apache.org/r1665125 ]

        CHUKWA-741. Updated to Hadoop 2.6.0 and HBase 1.0.0. (Eric Yang)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1665125 from Eric Yang in branch 'chukwa/trunk' [ https://svn.apache.org/r1665125 ] CHUKWA-741 . Updated to Hadoop 2.6.0 and HBase 1.0.0. (Eric Yang)
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-trunk #525 (See https://builds.apache.org/job/Chukwa-trunk/525/)
        CHUKWA-741. Updated to Hadoop 2.6.0 and HBase 1.0.0. (Eric Yang) (eyang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1665125)

        • /chukwa/trunk/CHANGES.txt
        • /chukwa/trunk/pom.xml
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/test/FilePerPostWriter.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/SeqFileWriter.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/localfs/LocalToRemoteHdfsMover.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/UserStore.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/ViewStore.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/WidgetStore.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/CHUKWA_CONSTANT.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/Demux.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLog.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/hicc/HiccWebServer.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/inputtools/log4j/Log4jMetricsSink.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/mapred/ChukwaJobTrackerInstrumentation.java
        • /chukwa/trunk/src/main/web/hicc/index.jsp
        • /chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/extraction/archive/TestArchive.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-trunk #525 (See https://builds.apache.org/job/Chukwa-trunk/525/ ) CHUKWA-741 . Updated to Hadoop 2.6.0 and HBase 1.0.0. (Eric Yang) (eyang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1665125 ) /chukwa/trunk/CHANGES.txt /chukwa/trunk/pom.xml /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/test/FilePerPostWriter.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/SeqFileWriter.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/localfs/LocalToRemoteHdfsMover.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/UserStore.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/ViewStore.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/WidgetStore.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/CHUKWA_CONSTANT.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/Demux.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLog.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/hicc/HiccWebServer.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/inputtools/log4j/Log4jMetricsSink.java /chukwa/trunk/src/main/java/org/apache/hadoop/mapred/ChukwaJobTrackerInstrumentation.java /chukwa/trunk/src/main/web/hicc/index.jsp /chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/extraction/archive/TestArchive.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Chukwa-trunk #527 (See https://builds.apache.org/job/Chukwa-trunk/527/)
        CHUKWA-741. Updated to Hadoop 2.6.0 and HBase 1.0.0. (Eric Yang) (eyang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1665125)

        • /chukwa/trunk/CHANGES.txt
        • /chukwa/trunk/pom.xml
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/test/FilePerPostWriter.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/SeqFileWriter.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/localfs/LocalToRemoteHdfsMover.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/UserStore.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/ViewStore.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/WidgetStore.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/CHUKWA_CONSTANT.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/Demux.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLog.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/hicc/HiccWebServer.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/inputtools/log4j/Log4jMetricsSink.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/mapred/ChukwaJobTrackerInstrumentation.java
        • /chukwa/trunk/src/main/web/hicc/index.jsp
        • /chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/extraction/archive/TestArchive.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Chukwa-trunk #527 (See https://builds.apache.org/job/Chukwa-trunk/527/ ) CHUKWA-741 . Updated to Hadoop 2.6.0 and HBase 1.0.0. (Eric Yang) (eyang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1665125 ) /chukwa/trunk/CHANGES.txt /chukwa/trunk/pom.xml /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/test/FilePerPostWriter.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/SeqFileWriter.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/writer/localfs/LocalToRemoteHdfsMover.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/UserStore.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/ViewStore.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datastore/WidgetStore.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/CHUKWA_CONSTANT.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/Demux.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLog.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/hicc/HiccWebServer.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/inputtools/log4j/Log4jMetricsSink.java /chukwa/trunk/src/main/java/org/apache/hadoop/mapred/ChukwaJobTrackerInstrumentation.java /chukwa/trunk/src/main/web/hicc/index.jsp /chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/extraction/archive/TestArchive.java

          People

          • Assignee:
            eyang Eric Yang
            Reporter:
            eyang Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development