Flume
  1. Flume
  2. FLUME-901

Make Flume NG build and pass tests against Hadoop 0.23 branch

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: NG alpha 1
    • Fix Version/s: v1.0.0
    • Component/s: None
    • Labels:

      Description

      This task is about adding an experimental profile for Hadoop 0.23

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          3h 7m 1 Bruno Mahé 22/Dec/11 21:42
          Patch Available Patch Available Resolved Resolved
          1h 38m 1 E. Sammer 22/Dec/11 23:21
          Resolved Resolved Closed Closed
          7s 1 E. Sammer 22/Dec/11 23:21
          Roshan Naik made changes -
          Link This issue is cloned as FLUME-1618 [ FLUME-1618 ]
          Hide
          Hudson added a comment -

          Integrated in flume-728 #82 (See https://builds.apache.org/job/flume-728/82/)
          FLUME-901: Make Flume NG build and pass tests against Hadoop 0.23 branch

          (Bruno Mahé via Eric Sammer)

          esammer : http://svn.apache.org/viewvc/?view=rev&rev=1222508
          Files :

          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/pom.xml
          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSTextFormatter.java
          • /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
          • /incubator/flume/branches/flume-728/pom.xml
          Show
          Hudson added a comment - Integrated in flume-728 #82 (See https://builds.apache.org/job/flume-728/82/ ) FLUME-901 : Make Flume NG build and pass tests against Hadoop 0.23 branch (Bruno Mahé via Eric Sammer) esammer : http://svn.apache.org/viewvc/?view=rev&rev=1222508 Files : /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/pom.xml /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSTextFormatter.java /incubator/flume/branches/flume-728/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java /incubator/flume/branches/flume-728/pom.xml
          E. Sammer made changes -
          Fix Version/s NG alpha 3 [ 12318896 ]
          E. Sammer made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          E. Sammer made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          E. Sammer added a comment -

          Committed to branch flume-728.

          Thanks Bruno!

          Show
          E. Sammer added a comment - Committed to branch flume-728. Thanks Bruno!
          Bruno Mahé made changes -
          Attachment FLUME-901.PATCH [ 12508451 ]
          Hide
          Bruno Mahé added a comment -

          Please find attached a patch.
          This patch introduces profiles to select which line of Apache Hadoop you wish to build against. By default, it will build against 0.20.205.0.
          Flume will not build yet against Hadoop 0.23 until HADOOP-7937 gets resolved. But this will not require any change on Flume side. Hadoop 0.23 is still in development, hence the -SNAPSHOT version

          Minor changes:

          • HDFSTextFormatter.java has to be updated to work against both Hadoop 0.20.20X and Hadoop 0.23. Text::getBytes() still returns the raw byte array, but now Text is also doing some preallocation. So the byte array would be different from what would be expected
          • TestHDFSEventSink.java had at least one of its assert parameters in the wrong order (got, expected) versus (expected, got)

          Don't hesitate to let me know if you would like me to rework any part

          Show
          Bruno Mahé added a comment - Please find attached a patch. This patch introduces profiles to select which line of Apache Hadoop you wish to build against. By default, it will build against 0.20.205.0. Flume will not build yet against Hadoop 0.23 until HADOOP-7937 gets resolved. But this will not require any change on Flume side. Hadoop 0.23 is still in development, hence the -SNAPSHOT version Minor changes: HDFSTextFormatter.java has to be updated to work against both Hadoop 0.20.20X and Hadoop 0.23. Text::getBytes() still returns the raw byte array, but now Text is also doing some preallocation. So the byte array would be different from what would be expected TestHDFSEventSink.java had at least one of its assert parameters in the wrong order (got, expected) versus (expected, got) Don't hesitate to let me know if you would like me to rework any part
          Bruno Mahé made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          E. Sammer made changes -
          Field Original Value New Value
          Assignee Bruno Mahé [ bmahe ]
          Bruno Mahé created issue -

            People

            • Assignee:
              Bruno Mahé
              Reporter:
              Bruno Mahé
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development