Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2
    • Fix Version/s: 0.22.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Processing of concatenated gzip files formerly stopped (quietly) at the end of the first substream/"member"; now processing will continue to the end of the concatenated stream, like gzip(1) does. (bzip2 support is unaffected by this patch.)

      Description

      When running MapReduce with concatenated gzip files as input only the first part is read, which is confusing, to say the least. Concatenated gzip is described in http://www.gnu.org/software/gzip/manual/gzip.html#Advanced-usage and in http://www.ietf.org/rfc/rfc1952.txt. (See original report at http://www.nabble.com/Problem-with-Hadoop-and-concatenated-gzip-files-to21383097.html)

      1. HADOOP-6835.v9.yahoo-0.20.2xx-branch.patch
        46 kB
        Greg Roelofs
      2. C6835-9.patch
        49 kB
        Chris Douglas
      3. HADOOP-6835.v8.trunk-hadoop-common.patch
        47 kB
        Greg Roelofs
      4. HADOOP-6835.v7.trunk-hadoop-common.patch
        41 kB
        Greg Roelofs
      5. HADOOP-6835.v6.trunk-hadoop-common.patch
        41 kB
        Greg Roelofs
      6. HADOOP-6835.v5.trunk-hadoop-common.patch
        20 kB
        Greg Roelofs
      7. HADOOP-6835.v4.trunk-hadoop-mapreduce.patch
        47 kB
        Greg Roelofs
      8. HADOOP-6835.v4.trunk-hadoop-common.patch
        18 kB
        Greg Roelofs
      9. HADOOP-6835.v4.yahoo-0.20.2xx-branch.patch
        83 kB
        Greg Roelofs
      10. HADOOP-6835.v3.yahoo-0.20.2xx-branch.patch
        82 kB
        Greg Roelofs
      11. MR-469.v2.yahoo-0.20.2xx-branch.patch
        69 kB
        Greg Roelofs
      12. grr-hadoop-mapreduce.dif.20100614c
        25 kB
        Greg Roelofs
      13. grr-hadoop-common.dif.20100614c
        37 kB
        Greg Roelofs

        Issue Links

          Activity

          Hide
          Oscar Gothberg added a comment -

          Thanks Tom for filing the Jira on this issue.

          Does anyone know a possible alternate gzip codec that could just be plugged into Hadoop? I suddenly have terabytes of gzipped data that I don't know if I can trust hadoop with, and it would be a big pain to re-archive everything.

          Show
          Oscar Gothberg added a comment - Thanks Tom for filing the Jira on this issue. Does anyone know a possible alternate gzip codec that could just be plugged into Hadoop? I suddenly have terabytes of gzipped data that I don't know if I can trust hadoop with, and it would be a big pain to re-archive everything.
          Hide
          David Ciemiewicz added a comment -

          bzip2 compression format also supports concatenation of individual bzip2 compressed files into a single file.

          bzcat has absolutely no problem reading all of the data in one of these concatenated files.

          Unfortunately, both Hadoop Streaming and Pig only see about 2% of the data from the original file in my case. That's a 98% effective data loss.

          Show
          David Ciemiewicz added a comment - bzip2 compression format also supports concatenation of individual bzip2 compressed files into a single file. bzcat has absolutely no problem reading all of the data in one of these concatenated files. Unfortunately, both Hadoop Streaming and Pig only see about 2% of the data from the original file in my case. That's a 98% effective data loss.
          Hide
          David Ciemiewicz added a comment -

          Unfortunately I discovered that concatenated bzip2 files did not work in Map-Reduce until AFTER I went and concatenated 3TB and over 250K compressed files.

          A colleague suggested that I "fix" my data using the following approach:

          hadoop dfs -cat X | bunzip2 | bzip2 | hadoop dfs -put - X.new

          I tried this with a 3GB single file concatenation of multiple bzip2 compressed files.

          This process took just over an hour with compression taking 5-6X longer than decompression (as measured in CPU utilization).

          It only took several minutes to concatenate the multiple part files into a single file.

          I think that this points out that decompressing and recompressing data is not really a viable solution for creating large concatenations of smaller files.

          The best performing solution is to create the smaller part files in parallel with a bunch of reducers, then concatenate them later into one (or several) larger files.

          And so fixing Hadoop Map Reduce to be able to read concatenations of files is actually probably the highest return on investment by the community.

          Show
          David Ciemiewicz added a comment - Unfortunately I discovered that concatenated bzip2 files did not work in Map-Reduce until AFTER I went and concatenated 3TB and over 250K compressed files. A colleague suggested that I "fix" my data using the following approach: hadoop dfs -cat X | bunzip2 | bzip2 | hadoop dfs -put - X.new I tried this with a 3GB single file concatenation of multiple bzip2 compressed files. This process took just over an hour with compression taking 5-6X longer than decompression (as measured in CPU utilization). It only took several minutes to concatenate the multiple part files into a single file. I think that this points out that decompressing and recompressing data is not really a viable solution for creating large concatenations of smaller files. The best performing solution is to create the smaller part files in parallel with a bunch of reducers, then concatenate them later into one (or several) larger files. And so fixing Hadoop Map Reduce to be able to read concatenations of files is actually probably the highest return on investment by the community.
          Hide
          Greg Roelofs added a comment -

          The bzip2 part reportedly is fixed on the trunk (HADOOP-4012); I haven't yet verified this for myself, but I have no reason to believe it doesn't work.

          I'm working on half of the gzip half, i.e., the native-libraries portion. I appear to have a working proof of concept, but my testing so far has been extremely minimal. The java.util.zip portion could be addressed with something similar to Duncan Loveday's MultiMemberGZIPInputStream workaround (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4691425), but the license on his actual code is unclear. (On the other hand, he has an Apache account and apparently still works at BT, so it might be possible to get that clarified.)

          Ravi, do you mind if I assign this issue to myself?

          Show
          Greg Roelofs added a comment - The bzip2 part reportedly is fixed on the trunk ( HADOOP-4012 ); I haven't yet verified this for myself, but I have no reason to believe it doesn't work. I'm working on half of the gzip half, i.e., the native-libraries portion. I appear to have a working proof of concept, but my testing so far has been extremely minimal. The java.util.zip portion could be addressed with something similar to Duncan Loveday's MultiMemberGZIPInputStream workaround ( http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4691425 ), but the license on his actual code is unclear. (On the other hand, he has an Apache account and apparently still works at BT, so it might be possible to get that clarified.) Ravi, do you mind if I assign this issue to myself?
          Hide
          Ravi Gummadi added a comment -

          Please assign it to yourself and work on it.

          Show
          Ravi Gummadi added a comment - Please assign it to yourself and work on it.
          Hide
          Greg Roelofs added a comment -

          Almost-final gzip concatenation code (several style-related issues to deal with, but working code, both native and non-native, with no debug statements) and a halfway test case (need to get bzip2 half working).

          Summary: I implemented an Inflater-based Decompressor with "manual" gzip header/trailer parsing and CRC checks, and added new getRemaining() and resetPartially() methods to the interface. I also modified DecompressorStream to support concatenated streams (decompress() and getCompressedData() methods). For backward compatibility, the default behavior is unchanged; one needs to set the new io.compression.gzip.concat config option to true to turn it on. Since bzip2 apparently changed its behavior without such a setting, perhaps this is overkill...

          Anyway, this is against trunk (as of a week or two ago). I still need to check it against Yahoo's tree, deal with the FIXMEs, update my source tree(s), run test-patch, etc. Also, I haven't included the (binary) test files here; I'll do so in one of the next versions of the patch.

          Show
          Greg Roelofs added a comment - Almost-final gzip concatenation code (several style-related issues to deal with, but working code, both native and non-native, with no debug statements) and a halfway test case (need to get bzip2 half working). Summary: I implemented an Inflater-based Decompressor with "manual" gzip header/trailer parsing and CRC checks, and added new getRemaining() and resetPartially() methods to the interface. I also modified DecompressorStream to support concatenated streams (decompress() and getCompressedData() methods). For backward compatibility, the default behavior is unchanged; one needs to set the new io.compression.gzip.concat config option to true to turn it on. Since bzip2 apparently changed its behavior without such a setting, perhaps this is overkill... Anyway, this is against trunk (as of a week or two ago). I still need to check it against Yahoo's tree, deal with the FIXMEs, update my source tree(s), run test-patch, etc. Also, I haven't included the (binary) test files here; I'll do so in one of the next versions of the patch.
          Hide
          David Ciemiewicz added a comment -

          On vacation Mon-Wed Feb 15-17. Offsite Thu-Fri, Feb 18-19.

          Show
          David Ciemiewicz added a comment - On vacation Mon-Wed Feb 15-17. Offsite Thu-Fri, Feb 18-19.
          Hide
          David Ciemiewicz added a comment -

          Greg, I cannot think of any good reasons to keep the current behavior of failing to read concatenated gzip files. So, requiring end users to actively set a flag io.compression.gzip.concat to permit meeting expectations of every user seems ass backwards. Reading concatenated files should be the default behavior.

          Show
          David Ciemiewicz added a comment - Greg, I cannot think of any good reasons to keep the current behavior of failing to read concatenated gzip files. So, requiring end users to actively set a flag io.compression.gzip.concat to permit meeting expectations of every user seems ass backwards. Reading concatenated files should be the default behavior.
          Hide
          Greg Roelofs added a comment -

          Ciemo,

          On Yahoo grids it will be (via site config). For the default code as used
          by everyone else, I'm open to input. I tend to be conservative when it
          comes to things that could be interpreted as user-visible API changes, but
          if everyone agrees that the old behavior is simply a bug, I'm happy to drop
          the config option and just hardcode concatenation support. Or I can leave
          it configurable but make the concat support the default.

          I'll send an e-mail to mapreduce-user and see what others think.

          Show
          Greg Roelofs added a comment - Ciemo, On Yahoo grids it will be (via site config). For the default code as used by everyone else, I'm open to input. I tend to be conservative when it comes to things that could be interpreted as user-visible API changes, but if everyone agrees that the old behavior is simply a bug, I'm happy to drop the config option and just hardcode concatenation support. Or I can leave it configurable but make the concat support the default. I'll send an e-mail to mapreduce-user and see what others think.
          Hide
          David Ciemiewicz added a comment -

          Greg, I have yet to encounter ANYONE who doesn't consider this a bug because all cited reference EXPECT concatenated files to work because the work in ALL OTHER cited instances including gnu tools, web browsers, etc. Can you think of a single instance where it would be the right thing to stop reading a concatenated file after the first part is read, ignoring all other concatenated parts. Forgive me but suggesting that we keep the existing behavior seems absurd because I cannot think of a single case where this would be the right thing to do.

          Show
          David Ciemiewicz added a comment - Greg, I have yet to encounter ANYONE who doesn't consider this a bug because all cited reference EXPECT concatenated files to work because the work in ALL OTHER cited instances including gnu tools, web browsers, etc. Can you think of a single instance where it would be the right thing to stop reading a concatenated file after the first part is read, ignoring all other concatenated parts. Forgive me but suggesting that we keep the existing behavior seems absurd because I cannot think of a single case where this would be the right thing to do.
          Hide
          Greg Roelofs added a comment -

          I can't think of a good use case for it, but a few years of development
          experience have taught me that, planetwide, someone else very well may have.
          Hence my question to mapreduce-user.

          The one thing that did occur to me was financially oriented, e.g., if an
          existing data flow just fits within budget (whether actual dollars or grid
          capacity or time limits or whatever) because it's been reading half of the
          available data and getting "good enough" results. Suddenly doubling its
          usage (or 50x, as in your case) without adequate warning could be quite
          painful. Admittedly, this is a weak example and probably very unlikely,
          but there may be a real case that's somewhat similar.

          Note that I'm perfectly willing to hardcode it always-on just like the
          trunk's bzip2 code; that would simplify the code, eliminate a per-bufferload
          conditional, and just generally be cleaner. I'm also happy to leave it
          configurable but on by default. However, I'd like to give the user community
          a chance to pipe up in case there actually is a problematic use case out
          there that you and I have overlooked.

          Show
          Greg Roelofs added a comment - I can't think of a good use case for it, but a few years of development experience have taught me that, planetwide, someone else very well may have. Hence my question to mapreduce-user. The one thing that did occur to me was financially oriented, e.g., if an existing data flow just fits within budget (whether actual dollars or grid capacity or time limits or whatever) because it's been reading half of the available data and getting "good enough" results. Suddenly doubling its usage (or 50x, as in your case) without adequate warning could be quite painful. Admittedly, this is a weak example and probably very unlikely, but there may be a real case that's somewhat similar. Note that I'm perfectly willing to hardcode it always-on just like the trunk's bzip2 code; that would simplify the code, eliminate a per-bufferload conditional, and just generally be cleaner. I'm also happy to leave it configurable but on by default. However, I'd like to give the user community a chance to pipe up in case there actually is a problematic use case out there that you and I have overlooked.
          Hide
          Ravi Gummadi added a comment -

          +1 for making the behavior of supporting concatenated gzip files as default behavior.

          Show
          Ravi Gummadi added a comment - +1 for making the behavior of supporting concatenated gzip files as default behavior.
          Hide
          Greg Roelofs added a comment -

          Okey dokey, default it is. mapreduce-user feedback was limited but in favor of hardcoding it (i.e., not even configurable), so that's currently my plan for the final version. Later today I'll post a second interim version that merely flips the current config option. (Main purpose will be to incorporate informal review feedback from Chris Douglas and, hopefully, make the unit test much more complete; at that point I'll consider it ready for formal review and checkin.)

          Show
          Greg Roelofs added a comment - Okey dokey, default it is. mapreduce-user feedback was limited but in favor of hardcoding it (i.e., not even configurable), so that's currently my plan for the final version. Later today I'll post a second interim version that merely flips the current config option. (Main purpose will be to incorporate informal review feedback from Chris Douglas and, hopefully, make the unit test much more complete; at that point I'll consider it ready for formal review and checkin.)
          Hide
          Greg Roelofs added a comment -

          Expanded test coverage uncovered a bug on Friday, and trunk update today has breakage, so this version is against Yahoo's 0.20S+ branch.

          Still not quite final; I haven't finished updating the unit test to exercise both native and built-in gzip and built-in bzip2 at multiple buffer sizes, and I've left some (mostly) commented-out debug statements in place in case that turns up anything further.

          Reviewer questions:

          • Currently the new BuiltInGzipDecompressor class inherits directly from JDK Inflater, but I suspect I should extend BuiltInZlibInflater instead.
          • Is it worthwhile to encapsulate the state label and associated variables into a private inner class (BuiltInGzipDecompressor.java, first FIXME comment)?

          The other FIXMEs are either related to the two items above or else are largely unrelated to this issue.

          Show
          Greg Roelofs added a comment - Expanded test coverage uncovered a bug on Friday, and trunk update today has breakage, so this version is against Yahoo's 0.20S+ branch. Still not quite final; I haven't finished updating the unit test to exercise both native and built-in gzip and built-in bzip2 at multiple buffer sizes, and I've left some (mostly) commented-out debug statements in place in case that turns up anything further. Reviewer questions: Currently the new BuiltInGzipDecompressor class inherits directly from JDK Inflater, but I suspect I should extend BuiltInZlibInflater instead. Is it worthwhile to encapsulate the state label and associated variables into a private inner class (BuiltInGzipDecompressor.java, first FIXME comment)? The other FIXMEs are either related to the two items above or else are largely unrelated to this issue.
          Hide
          Greg Roelofs added a comment -

          Oh, two more review questions:

          • DecompressorStream currently supports two concatenation modes via a pseudo-ifdef ("final boolean useResetPartially"): resetPartially(), which avoids any additional buffer copies at a cost of uglifying the Decompressor interface with this new method; or regular reset() + setInput() to recopy any "excess" bytes (that is, from stream N+1) at the end of stream N. The amount of recopying in the latter case is dependent on the buffer sizes (typically 64KB around here) and sizes of the concatenated gzip streams/members, but in general it won't be much. Barring strong disagreement, I'll go with the latter approach and clean up all the resetPartially() stuff in the next (hopefully final) version of the patch.
          • Any last-minute qualms about hardcoding the concatenation behavior? It would simplify the patch slightly and seems to be the preferred approach, so that's my plan for the next version.
          Show
          Greg Roelofs added a comment - Oh, two more review questions: DecompressorStream currently supports two concatenation modes via a pseudo-ifdef ("final boolean useResetPartially"): resetPartially(), which avoids any additional buffer copies at a cost of uglifying the Decompressor interface with this new method; or regular reset() + setInput() to recopy any "excess" bytes (that is, from stream N+1) at the end of stream N. The amount of recopying in the latter case is dependent on the buffer sizes (typically 64KB around here) and sizes of the concatenated gzip streams/members, but in general it won't be much. Barring strong disagreement, I'll go with the latter approach and clean up all the resetPartially() stuff in the next (hopefully final) version of the patch. Any last-minute qualms about hardcoding the concatenation behavior? It would simplify the patch slightly and seems to be the preferred approach, so that's my plan for the next version.
          Hide
          Greg Roelofs added a comment -

          (This was MAPREDUCE-469, but Chris Douglas noted that it affects the core compression code in hadoop-common, not hadoop-mapreduce, so moved here now.)

          Show
          Greg Roelofs added a comment - (This was MAPREDUCE-469 , but Chris Douglas noted that it affects the core compression code in hadoop-common, not hadoop-mapreduce, so moved here now.)
          Hide
          Greg Roelofs added a comment -

          Mildly amusing: http://hg.openjdk.java.net/jdk7/jdk7/jdk/rev/41a5722e6e10

          Looks like Sun/Oracle finally decided to fix their GZIPInputStream bug. (Patch is dated 20100524.)

          Show
          Greg Roelofs added a comment - Mildly amusing: http://hg.openjdk.java.net/jdk7/jdk7/jdk/rev/41a5722e6e10 Looks like Sun/Oracle finally decided to fix their GZIPInputStream bug. (Patch is dated 20100524.)
          Hide
          Chris Douglas added a comment -

          Currently the new BuiltInGzipDecompressor class inherits directly from JDK Inflater, but I suspect I should extend BuiltInZlibInflater instead.

          I'd lean the other way. It's not really a supertype of BuiltInZlibInflater and neither are public types. It's not really an Inflater, either; it may be worth either supporting that interface or using an Inflater member rather than inheritance, since it only calls public methods. Either way, it's an existing confusion in the compression type hierarchy, and if it calls for any additional testing it can be ignored in this issue.

          Is it worthwhile to encapsulate the state label and associated variables into a private inner class (BuiltInGzipDecompressor.java, first FIXME comment)?

          Since the code is already implemented and tested, refactoring it for a slightly cleaner implementation of a user-opaque, RFC-compliant library doesn't seem like a reasonable condition for committing it.

          DecompressorStream currently supports two concatenation modes via a pseudo-ifdef ("final boolean useResetPartially"): resetPartially(), which avoids any additional buffer copies at a cost of uglifying the Decompressor interface with this new method; or regular reset() + setInput() to recopy any "excess" bytes (that is, from stream N+1) at the end of stream N. The amount of recopying in the latter case is dependent on the buffer sizes (typically 64KB around here) and sizes of the concatenated gzip streams/members, but in general it won't be much. Barring strong disagreement, I'll go with the latter approach and clean up all the resetPartially() stuff in the next (hopefully final) version of the patch.

          Agreed; the penalty for re-copying once per stream is light enough that it can be endured for other codecs' API considerations.

          Any last-minute qualms about hardcoding the concatenation behavior? It would simplify the patch slightly and seems to be the preferred approach, so that's my plan for the next version.

          Sounds fine to me. It may cause faults in some containers, but those are probably bugs covered over by this one.

          Show
          Chris Douglas added a comment - Currently the new BuiltInGzipDecompressor class inherits directly from JDK Inflater, but I suspect I should extend BuiltInZlibInflater instead. I'd lean the other way. It's not really a supertype of BuiltInZlibInflater and neither are public types. It's not really an Inflater , either; it may be worth either supporting that interface or using an Inflater member rather than inheritance, since it only calls public methods. Either way, it's an existing confusion in the compression type hierarchy, and if it calls for any additional testing it can be ignored in this issue. Is it worthwhile to encapsulate the state label and associated variables into a private inner class (BuiltInGzipDecompressor.java, first FIXME comment)? Since the code is already implemented and tested, refactoring it for a slightly cleaner implementation of a user-opaque, RFC-compliant library doesn't seem like a reasonable condition for committing it. DecompressorStream currently supports two concatenation modes via a pseudo-ifdef ("final boolean useResetPartially"): resetPartially(), which avoids any additional buffer copies at a cost of uglifying the Decompressor interface with this new method; or regular reset() + setInput() to recopy any "excess" bytes (that is, from stream N+1) at the end of stream N. The amount of recopying in the latter case is dependent on the buffer sizes (typically 64KB around here) and sizes of the concatenated gzip streams/members, but in general it won't be much. Barring strong disagreement, I'll go with the latter approach and clean up all the resetPartially() stuff in the next (hopefully final) version of the patch. Agreed; the penalty for re-copying once per stream is light enough that it can be endured for other codecs' API considerations. Any last-minute qualms about hardcoding the concatenation behavior? It would simplify the patch slightly and seems to be the preferred approach, so that's my plan for the next version. Sounds fine to me. It may cause faults in some containers, but those are probably bugs covered over by this one.
          Hide
          Greg Roelofs added a comment -

          Updated patch with the promised changes from my third and fourth questions (i.e., reset()/setInput() rather than resetPartially(), and hardcoded concat fix); cleaned up debug/FIXME stuff; and full test files (well, the shorter versions).

          This is still against Yahoo's 0.20.2xx branch; I'll port it back to trunk next.

          Show
          Greg Roelofs added a comment - Updated patch with the promised changes from my third and fourth questions (i.e., reset()/setInput() rather than resetPartially(), and hardcoded concat fix); cleaned up debug/FIXME stuff; and full test files (well, the shorter versions). This is still against Yahoo's 0.20.2xx branch; I'll port it back to trunk next.
          Hide
          Greg Roelofs added a comment -

          Whoops, ships passing...

          It's not really an Inflater, either; it may be worth either supporting that interface or using an Inflater member rather than inheritance, since it only calls public methods.

          Good point. OK, I'll look into that first, then port to trunk. Seems like a pretty lightweight change.

          Show
          Greg Roelofs added a comment - Whoops, ships passing... It's not really an Inflater, either; it may be worth either supporting that interface or using an Inflater member rather than inheritance, since it only calls public methods. Good point. OK, I'll look into that first, then port to trunk. Seems like a pretty lightweight change.
          Hide
          Greg Roelofs added a comment -

          Final (I think) version against Yahoo 0.20.2xx branch. Trunk version coming soon.

          Show
          Greg Roelofs added a comment - Final (I think) version against Yahoo 0.20.2xx branch. Trunk version coming soon.
          Hide
          Greg Roelofs added a comment -

          Final patch against trunk. All the good stuff is in the -common half; the mapreduce half has a new unit test and six supporting test files.

          Because the unit test purports to be a generic concatenated-compressed-input test, I had ported both of the main gzip subtests to work with the bzip2 decoder. That turned up a possible issue in the bzip2 decoder (or, equally likely, in the way I wrote the test case); for now, I commented out that part of the test (near line 586). The weird thing is that my own, more rambunctious test case works fine. This shouldn't be a blocker for a gzip-oriented patch, but if a reviewer could take a quick look at the commented-out block and see if I did something obviously wrong, I'd appreciate it. (The problem didn't turn up in the Yahoo branch because 0.20.x doesn't support bzip2 concatenation, as the issue title notes. The breakage appears to occur precisely at the start of the second bzip2 "member.")

          Show
          Greg Roelofs added a comment - Final patch against trunk. All the good stuff is in the -common half; the mapreduce half has a new unit test and six supporting test files. Because the unit test purports to be a generic concatenated-compressed-input test, I had ported both of the main gzip subtests to work with the bzip2 decoder. That turned up a possible issue in the bzip2 decoder (or, equally likely, in the way I wrote the test case); for now, I commented out that part of the test (near line 586). The weird thing is that my own, more rambunctious test case works fine. This shouldn't be a blocker for a gzip-oriented patch, but if a reviewer could take a quick look at the commented-out block and see if I did something obviously wrong, I'd appreciate it. (The problem didn't turn up in the Yahoo branch because 0.20.x doesn't support bzip2 concatenation, as the issue title notes. The breakage appears to occur precisely at the start of the second bzip2 "member.")
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448272/HADOOP-6835.v4.trunk-hadoop-mapreduce.patch
          against trunk revision 957074.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 21 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/596/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448272/HADOOP-6835.v4.trunk-hadoop-mapreduce.patch against trunk revision 957074. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/596/console This message is automatically generated.
          Hide
          Greg Roelofs added a comment -

          OK, so auto-patch doesn't know about the split projects, and hadoop-mapreduce patches don't work in hadoop-common.

          I tried moving the test case to hadoop-common/src/test/core/org/apache/hadoop/io/compress, but it depends on too many MR classes to be workable (JobConf, Reporter, RecordReader, FileSplit, InputSplit, FileInputFormat, TextInputFormat). So I guess the procedure is (1) reupload the hadoop-common patch and mark as patch-available [actually, I missed a fix in TestCodec, so I need to upload a new one anyway]; (2) get that past QA and checked in; (3) open a separate MR JIRA, attach the test case, and mark it patch-available; and (4) get that past QA and checked in, too.

          If there's an easier way, feel free to enlighten me. (On a related note, I was told by Reliable Sources to blame Owen, but he just left town. Coincidence?)

          Show
          Greg Roelofs added a comment - OK, so auto-patch doesn't know about the split projects, and hadoop-mapreduce patches don't work in hadoop-common. I tried moving the test case to hadoop-common/src/test/core/org/apache/hadoop/io/compress, but it depends on too many MR classes to be workable (JobConf, Reporter, RecordReader, FileSplit, InputSplit, FileInputFormat, TextInputFormat). So I guess the procedure is (1) reupload the hadoop-common patch and mark as patch-available [actually, I missed a fix in TestCodec, so I need to upload a new one anyway] ; (2) get that past QA and checked in; (3) open a separate MR JIRA, attach the test case, and mark it patch-available; and (4) get that past QA and checked in, too. If there's an easier way, feel free to enlighten me. (On a related note, I was told by Reliable Sources to blame Owen, but he just left town. Coincidence?)
          Hide
          Greg Roelofs added a comment -

          Main (hadoop-common) patch, version 5: includes TestCodec fix.

          Show
          Greg Roelofs added a comment - Main (hadoop-common) patch, version 5: includes TestCodec fix.
          Hide
          Greg Roelofs added a comment -

          hadoop-common v5 this time.

          Show
          Greg Roelofs added a comment - hadoop-common v5 this time.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448469/HADOOP-6835.v5.trunk-hadoop-common.patch
          against trunk revision 957074.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The patch appears to cause tar ant target to fail.

          -1 findbugs. The patch appears to cause Findbugs to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/598/testReport/
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/598/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/598/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448469/HADOOP-6835.v5.trunk-hadoop-common.patch against trunk revision 957074. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/598/testReport/ Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/598/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/598/console This message is automatically generated.
          Hide
          Greg Roelofs added a comment -

          Cancelling and resubmitting per ArunM: "sometimes Hudson just gets weird" (or words to that effect).

          Show
          Greg Roelofs added a comment - Cancelling and resubmitting per ArunM: "sometimes Hudson just gets weird" (or words to that effect).
          Hide
          Greg Roelofs added a comment -

          No change to patch, just retrying Hudson. Error lines were:

          [javac] /grid/0/hudson/hudson-slave/workspace/Hadoop-Patch-h4.grid.sp2.yahoo.net/trunk/src/java/org/apache/hadoop/io/compress/GzipCodec.java:195: cannot find symbol
          [javac] symbol : class BuiltInGzipDecompressor
          [javac] location: class org.apache.hadoop.io.compress.GzipCodec
          [javac] : new BuiltInGzipDecompressor();
          [javac] ^
          [javac] /grid/0/hudson/hudson-slave/workspace/Hadoop-Patch-h4.grid.sp2.yahoo.net/trunk/src/java/org/apache/hadoop/io/compress/GzipCodec.java:201: cannot find symbol
          [javac] symbol : class BuiltInGzipDecompressor
          [javac] location: class org.apache.hadoop.io.compress.GzipCodec
          [javac] : BuiltInGzipDecompressor.class;
          [javac] ^

          However, BuiltInGzipDecompressor lives in org.apache.hadoop.io.compress.zlib, and GzipCodec.java includes "import org.apache.hadoop.io.compress.zlib.*;", so this failure makes no sense to either of us.

          Show
          Greg Roelofs added a comment - No change to patch, just retrying Hudson. Error lines were: [javac] /grid/0/hudson/hudson-slave/workspace/Hadoop-Patch-h4.grid.sp2.yahoo.net/trunk/src/java/org/apache/hadoop/io/compress/GzipCodec.java:195: cannot find symbol [javac] symbol : class BuiltInGzipDecompressor [javac] location: class org.apache.hadoop.io.compress.GzipCodec [javac] : new BuiltInGzipDecompressor(); [javac] ^ [javac] /grid/0/hudson/hudson-slave/workspace/Hadoop-Patch-h4.grid.sp2.yahoo.net/trunk/src/java/org/apache/hadoop/io/compress/GzipCodec.java:201: cannot find symbol [javac] symbol : class BuiltInGzipDecompressor [javac] location: class org.apache.hadoop.io.compress.GzipCodec [javac] : BuiltInGzipDecompressor.class; [javac] ^ However, BuiltInGzipDecompressor lives in org.apache.hadoop.io.compress.zlib, and GzipCodec.java includes "import org.apache.hadoop.io.compress.zlib.*;", so this failure makes no sense to either of us.
          Hide
          Greg Roelofs added a comment -

          Pilot error, sigh. Patches work better when new files are git added first...

          Show
          Greg Roelofs added a comment - Pilot error, sigh. Patches work better when new files are git added first...
          Hide
          Greg Roelofs added a comment -

          Same as v5 except with BuiltInGzipDecompressor.java included.

          Show
          Greg Roelofs added a comment - Same as v5 except with BuiltInGzipDecompressor.java included.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448841/HADOOP-6835.v6.trunk-hadoop-common.patch
          against trunk revision 960137.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          -1 javac. The applied patch generated 1019 javac compiler warnings (more than the trunk's current 1017 warnings).

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448841/HADOOP-6835.v6.trunk-hadoop-common.patch against trunk revision 960137. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. -1 javac. The applied patch generated 1019 javac compiler warnings (more than the trunk's current 1017 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/console This message is automatically generated.
          Hide
          Greg Roelofs added a comment -

          Fixed extra pair of javac warnings (unnecessary int typecasts in readUShortLE()).

          The javadoc output file (http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/artifact/trunk/patchprocess/patchJavadocWarnings.txt) shows 15 + 6 warnings, all of which are existing warnings in the security code. I have no idea which one it thinks I'm responsible for (particularly since there's no corresponding "trunkJavadocWarnings.txt" link), but I plead not guilty...

          Show
          Greg Roelofs added a comment - Fixed extra pair of javac warnings (unnecessary int typecasts in readUShortLE() ). The javadoc output file ( http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/602/artifact/trunk/patchprocess/patchJavadocWarnings.txt ) shows 15 + 6 warnings, all of which are existing warnings in the security code. I have no idea which one it thinks I'm responsible for (particularly since there's no corresponding "trunkJavadocWarnings.txt" link), but I plead not guilty...
          Hide
          Greg Roelofs added a comment -

          (Not sure this will work... Sorting by "date attached" currently shows 8:59PM coming after 9:59pm. Oooookay, then.)

          Show
          Greg Roelofs added a comment - (Not sure this will work... Sorting by "date attached" currently shows 8:59PM coming after 9:59pm. Oooookay, then.)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448847/HADOOP-6835.v7.trunk-hadoop-common.patch
          against trunk revision 960137.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/603/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/603/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/603/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/603/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448847/HADOOP-6835.v7.trunk-hadoop-common.patch against trunk revision 960137. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/603/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/603/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/603/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/603/console This message is automatically generated.
          Hide
          Greg Roelofs added a comment -

          Same as v7 but with a pair of new TestCodec unit (sub)tests provided by Chris Douglas. (These represent a low-level, hadoop-common complement to the existing ones in the hadoop-mapreduce patch.)

          All uses of the deprecated "hadoop.native.lib" option in TestCodec are replaced by "io.native.lib.available", too.

          Show
          Greg Roelofs added a comment - Same as v7 but with a pair of new TestCodec unit (sub)tests provided by Chris Douglas. (These represent a low-level, hadoop-common complement to the existing ones in the hadoop-mapreduce patch.) All uses of the deprecated "hadoop.native.lib" option in TestCodec are replaced by "io.native.lib.available", too.
          Hide
          Greg Roelofs added a comment -

          Ready for checkin. The javadoc warnings are all preexisting Kerberos/security-related ones (verified with Chris).

          Show
          Greg Roelofs added a comment - Ready for checkin. The javadoc warnings are all preexisting Kerberos/security-related ones (verified with Chris).
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448852/HADOOP-6835.v8.trunk-hadoop-common.patch
          against trunk revision 960137.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/83/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/83/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/83/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/83/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448852/HADOOP-6835.v8.trunk-hadoop-common.patch against trunk revision 960137. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/83/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/83/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/83/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/83/console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12448852/HADOOP-6835.v8.trunk-hadoop-common.patch
          against trunk revision 960137.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/604/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/604/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/604/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/604/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448852/HADOOP-6835.v8.trunk-hadoop-common.patch against trunk revision 960137. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/604/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/604/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/604/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/604/console This message is automatically generated.
          Hide
          Chris Douglas added a comment -

          +1

          I committed this. Thanks, Greg!

          Show
          Chris Douglas added a comment - +1 I committed this. Thanks, Greg!
          Hide
          Chris Douglas added a comment -

          Very minor modifications to v8 prior to commit

          • Use PureJavaCRC32 per comments
          • Removed unused GzipCodec::GzipInputStream
          • Replace config literals with CommonConfigurationKeys constants
          Show
          Chris Douglas added a comment - Very minor modifications to v8 prior to commit Use PureJavaCRC32 per comments Removed unused GzipCodec::GzipInputStream Replace config literals with CommonConfigurationKeys constants
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #320 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/320/)
          HADOOP-6835. Add support for concatenated gzip input. Contributed by Greg Roelofs

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #320 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/320/ ) HADOOP-6835 . Add support for concatenated gzip input. Contributed by Greg Roelofs
          Hide
          Greg Roelofs added a comment -

          Final Yahoo 0.20.2xx version, incorporating all relevant trunk updates through "v9".

          Show
          Greg Roelofs added a comment - Final Yahoo 0.20.2xx version, incorporating all relevant trunk updates through "v9".
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #390 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/390/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #390 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/390/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #616 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/616/)
          MAPREDUCE-1927. Unit test for HADOOP-6835 (concatenated gzip support). Contributed by Greg Roelofs.

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #616 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/616/ ) MAPREDUCE-1927 . Unit test for HADOOP-6835 (concatenated gzip support). Contributed by Greg Roelofs.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/ )

            People

            • Assignee:
              Greg Roelofs
              Reporter:
              Tom White
            • Votes:
              2 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development