Flume
  1. Flume
  2. FLUME-901

Make Flume NG build and pass tests against Hadoop 0.23 branch

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: NG alpha 1
    • Fix Version/s: v1.0.0
    • Component/s: None
    • Labels:

      Description

      This task is about adding an experimental profile for Hadoop 0.23

        Issue Links

          Activity

          Hide
          Bruno Mahé added a comment -

          Please find attached a patch.
          This patch introduces profiles to select which line of Apache Hadoop you wish to build against. By default, it will build against 0.20.205.0.
          Flume will not build yet against Hadoop 0.23 until HADOOP-7937 gets resolved. But this will not require any change on Flume side. Hadoop 0.23 is still in development, hence the -SNAPSHOT version

          Minor changes:

          • HDFSTextFormatter.java has to be updated to work against both Hadoop 0.20.20X and Hadoop 0.23. Text::getBytes() still returns the raw byte array, but now Text is also doing some preallocation. So the byte array would be different from what would be expected
          • TestHDFSEventSink.java had at least one of its assert parameters in the wrong order (got, expected) versus (expected, got)

          Don't hesitate to let me know if you would like me to rework any part

          Show
          Bruno Mahé added a comment - Please find attached a patch. This patch introduces profiles to select which line of Apache Hadoop you wish to build against. By default, it will build against 0.20.205.0. Flume will not build yet against Hadoop 0.23 until HADOOP-7937 gets resolved. But this will not require any change on Flume side. Hadoop 0.23 is still in development, hence the -SNAPSHOT version Minor changes: HDFSTextFormatter.java has to be updated to work against both Hadoop 0.20.20X and Hadoop 0.23. Text::getBytes() still returns the raw byte array, but now Text is also doing some preallocation. So the byte array would be different from what would be expected TestHDFSEventSink.java had at least one of its assert parameters in the wrong order (got, expected) versus (expected, got) Don't hesitate to let me know if you would like me to rework any part
          Hide
          E. Sammer added a comment -

          Committed to branch flume-728.

          Thanks Bruno!

          Show
          E. Sammer added a comment - Committed to branch flume-728. Thanks Bruno!
          Hide
          Hudson added a comment -

          Integrated in flume-728 #82 (See https://builds.apache.org/job/flume-728/82/)
          FLUME-901: Make Flume NG build and pass tests against Hadoop 0.23 branch

          (Bruno Mahé via Eric Sammer)

          esammer : http://svn.apache.org/viewvc/?view=rev&rev=1222508
          Files :

          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/pom.xml
          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSTextFormatter.java
          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
          • /incubator/flume/branches/flume-728/pom.xml
          Show
          Hudson added a comment - Integrated in flume-728 #82 (See https://builds.apache.org/job/flume-728/82/ ) FLUME-901 : Make Flume NG build and pass tests against Hadoop 0.23 branch (Bruno Mahé via Eric Sammer) esammer : http://svn.apache.org/viewvc/?view=rev&rev=1222508 Files : /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/pom.xml /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSTextFormatter.java /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java /incubator/flume/branches/flume-728/pom.xml

            People

            • Assignee:
              Bruno Mahé
              Reporter:
              Bruno Mahé
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development