Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.2.0
    • Component/s: None
    • Labels:
      None
    1. CHUKWA-195.patch
      2 kB
      Jerome Boulon

      Activity

      Hide
      macyang Mac Yang added a comment -

      Some more details here would be helpful.

      Show
      macyang Mac Yang added a comment - Some more details here would be helpful.
      Hide
      jboulon Jerome Boulon added a comment -

      Most M/R are not compatible with Hadoop 0.20, for example the hdfs package has been renamed to dfs and there's a lot of deprecated methods/class that we are using.
      This Jira is to upgrade the source code remove all warnings and be compatible with Hadoop 0.20.

      Show
      jboulon Jerome Boulon added a comment - Most M/R are not compatible with Hadoop 0.20, for example the hdfs package has been renamed to dfs and there's a lot of deprecated methods/class that we are using. This Jira is to upgrade the source code remove all warnings and be compatible with Hadoop 0.20.
      Hide
      asrabkin Ari Rabkin added a comment -

      Just talked to Owen, who thought this might be a blocker for a release.

      Thoughts?

      Show
      asrabkin Ari Rabkin added a comment - Just talked to Owen, who thought this might be a blocker for a release. Thoughts?
      Hide
      jboulon Jerome Boulon added a comment -

      1) Are you saying that if we cannot get data from a .20 cluster then it's a blocker?
      2) or are you saying that the backend should run on .20?

      if 1) it's OK because we are able to collect information from a .20 cluster but it sounds strange that we want to be the first one to move a production application to .20?

      Show
      jboulon Jerome Boulon added a comment - 1) Are you saying that if we cannot get data from a .20 cluster then it's a blocker? 2) or are you saying that the backend should run on .20? if 1) it's OK because we are able to collect information from a .20 cluster but it sounds strange that we want to be the first one to move a production application to .20?
      Hide
      macyang Mac Yang added a comment -

      Chatted with Owen, the concern was not about which version of hadoop that chukwa runs on. Rather, it's about the build process and the bundling of multiple hadoop jars (currently required to build the client jar).

      Show
      macyang Mac Yang added a comment - Chatted with Owen, the concern was not about which version of hadoop that chukwa runs on. Rather, it's about the build process and the bundling of multiple hadoop jars (currently required to build the client jar).
      Hide
      jboulon Jerome Boulon added a comment -

      I agree that moving to H.20 should be the target for C0.2 but not for C0.1.2.
      Pig lib directory also contains 2 hadoop jars, .17 & .18 to deal with an API change and still be compatible.

      Show
      jboulon Jerome Boulon added a comment - I agree that moving to H.20 should be the target for C0.2 but not for C0.1.2. Pig lib directory also contains 2 hadoop jars, .17 & .18 to deal with an API change and still be compatible.
      Hide
      jboulon Jerome Boulon added a comment -

      Chukwa is working out of the box on Hadoop 0.20. This patch remove the H 0.18 jar file.
      If you want to specify a queue then you need to add the queue parameter to chukwa-demux-conf.xml.

      Example:
      <property>
      <name>mapred.job.queue.name</name>
      <value>YourQueueName</value>
      <description>Hadoop Queue </description>
      </property>

      This should not go to the 0.1.2 branch.

      Show
      jboulon Jerome Boulon added a comment - Chukwa is working out of the box on Hadoop 0.20. This patch remove the H 0.18 jar file. If you want to specify a queue then you need to add the queue parameter to chukwa-demux-conf.xml. Example: <property> <name>mapred.job.queue.name</name> <value>YourQueueName</value> <description>Hadoop Queue </description> </property> This should not go to the 0.1.2 branch.
      Hide
      jboulon Jerome Boulon added a comment -

      Another pass will be needed in order to remove all deprecated methods.

      Show
      jboulon Jerome Boulon added a comment - Another pass will be needed in order to remove all deprecated methods.
      Hide
      eyang Eric Yang added a comment -

      Since Hadoop 0.20 deprecated a lot of JobTracker and JobConf classes, the compilation warnings are ok for now. We will update when Hadoop 0.21 has the job tracker refactor in place. I just committed this, thanks Jerome.

      Show
      eyang Eric Yang added a comment - Since Hadoop 0.20 deprecated a lot of JobTracker and JobConf classes, the compilation warnings are ok for now. We will update when Hadoop 0.21 has the job tracker refactor in place. I just committed this, thanks Jerome.
      Hide
      hudson Hudson added a comment -
      Show
      hudson Hudson added a comment - Integrated in Chukwa-trunk #47 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/47/ )

        People

        • Assignee:
          jboulon Jerome Boulon
          Reporter:
          jboulon Jerome Boulon
        • Votes:
          1 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development