Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-901

Make Flume NG build and pass tests against Hadoop 0.23 branch

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: NG alpha 1
    • Fix Version/s: v1.0.0
    • Component/s: None
    • Labels:

      Description

      This task is about adding an experimental profile for Hadoop 0.23

        Issue Links

          Activity

          Hide
          bmahe Bruno Mahé added a comment -

          Please find attached a patch.
          This patch introduces profiles to select which line of Apache Hadoop you wish to build against. By default, it will build against 0.20.205.0.
          Flume will not build yet against Hadoop 0.23 until HADOOP-7937 gets resolved. But this will not require any change on Flume side. Hadoop 0.23 is still in development, hence the -SNAPSHOT version

          Minor changes:

          • HDFSTextFormatter.java has to be updated to work against both Hadoop 0.20.20X and Hadoop 0.23. Text::getBytes() still returns the raw byte array, but now Text is also doing some preallocation. So the byte array would be different from what would be expected
          • TestHDFSEventSink.java had at least one of its assert parameters in the wrong order (got, expected) versus (expected, got)

          Don't hesitate to let me know if you would like me to rework any part

          Show
          bmahe Bruno Mahé added a comment - Please find attached a patch. This patch introduces profiles to select which line of Apache Hadoop you wish to build against. By default, it will build against 0.20.205.0. Flume will not build yet against Hadoop 0.23 until HADOOP-7937 gets resolved. But this will not require any change on Flume side. Hadoop 0.23 is still in development, hence the -SNAPSHOT version Minor changes: HDFSTextFormatter.java has to be updated to work against both Hadoop 0.20.20X and Hadoop 0.23. Text::getBytes() still returns the raw byte array, but now Text is also doing some preallocation. So the byte array would be different from what would be expected TestHDFSEventSink.java had at least one of its assert parameters in the wrong order (got, expected) versus (expected, got) Don't hesitate to let me know if you would like me to rework any part
          Hide
          esammer E. Sammer added a comment -

          Committed to branch flume-728.

          Thanks Bruno!

          Show
          esammer E. Sammer added a comment - Committed to branch flume-728. Thanks Bruno!
          Hide
          hudson Hudson added a comment -

          Integrated in flume-728 #82 (See https://builds.apache.org/job/flume-728/82/)
          FLUME-901: Make Flume NG build and pass tests against Hadoop 0.23 branch

          (Bruno Mahé via Eric Sammer)

          esammer : http://svn.apache.org/viewvc/?view=rev&rev=1222508
          Files :

          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/pom.xml
          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSTextFormatter.java
          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
          • /incubator/flume/branches/flume-728/pom.xml
          Show
          hudson Hudson added a comment - Integrated in flume-728 #82 (See https://builds.apache.org/job/flume-728/82/ ) FLUME-901 : Make Flume NG build and pass tests against Hadoop 0.23 branch (Bruno Mahé via Eric Sammer) esammer : http://svn.apache.org/viewvc/?view=rev&rev=1222508 Files : /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/pom.xml /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSTextFormatter.java /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java /incubator/flume/branches/flume-728/pom.xml

            People

            • Assignee:
              bmahe Bruno Mahé
              Reporter:
              bmahe Bruno Mahé
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development