|
Chris Douglas made changes - 19/Jan/08 03:22 AM
[
Permlink
| « Hide
]
Chris Douglas added a comment - 19/Jan/08 03:27 AM
This patch adds lzop compatibility as an optional codec. On writes, it adds a generic header to .lzo files; on reads, it respects and confirms any block-checksum data specified in the header. It cannot be used with SequenceFiles.
Chris Douglas made changes - 19/Jan/08 03:27 AM
Chris Douglas made changes - 19/Jan/08 04:02 AM
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12373587/2664-0.patch  against trunk revision r613499. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs -1. The patch appears to introduce 4 new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1662/testReport/ This message is automatically generated. Fixed findbugs warnings, bumped buffer to 256k (the size used by lzop) for the decompressor, changed the decompressor to the "safe" code to avoid crashing the JVM when it's too small, and added some documentation.
I have some reservations about this patch (memory usage, thread safety if pooled, etc), so I'm pushing it to 0.17.
Chris Douglas made changes - 21/Jan/08 10:13 PM
Chris Douglas made changes - 21/Jan/08 10:13 PM
Chris Douglas made changes - 22/Jan/08 08:00 PM
Chris Douglas made changes - 22/Jan/08 08:01 PM
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12373706/2664-1.patch against trunk revision r614413. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1680/testReport/ This message is automatically generated. -1
I'm pulling this back. The writes from the cstr and its (related) silent incompatibility with SequenceFile are sufficient to prevent it from being checked in. It reads and writes lzop-compatible files, but it is inadequate as a general compression codec. SequenceFile explicitly checks for a non-native version of GzipCodec, but surely there's a better way to effect this. That said, it should be noted that one can still write ".lzo" files from LzoCodec that aren't. The incompatible change in this patch- that asserts precedence for the .lzo extension and changes the former to .lzo_deflate - should be considered for 0.17 regardless of what happens with this patch.
Chris Douglas made changes - 13/Feb/08 11:28 PM
Robert Chansler made changes - 25/Mar/08 03:03 AM
I'm making this PA again. The sin for which it was withdrawn- writing out the header in the constructor- is actually a fairly minor one (that java.util.zip.GzipOutputStream is also guilty of). I'm not sure what to do with the SequenceFile incompatibility.
Chris Douglas made changes - 30/Apr/08 08:46 PM
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12373706/2664-1.patch against trunk revision 645773. @author +1. The patch does not contain any @author tags. tests included -1. The patch doesn't appear to include any new or modified tests. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2356/testReport/ This message is automatically generated.
Chris Douglas made changes - 06/May/08 10:53 PM
This really should have unit test.
Owen O'Malley made changes - 22/May/08 04:47 PM
Mukund Madhugiri made changes - 07/Jun/08 01:26 AM
Added a test and an entry to io.compression.codecs.
Chris Douglas made changes - 11/Jun/08 10:58 PM
Chris Douglas made changes - 11/Jun/08 10:58 PM
Chris Douglas made changes - 11/Jun/08 11:01 PM
Chris Douglas made changes - 11/Jun/08 11:03 PM
Chris Douglas made changes - 11/Jun/08 11:04 PM
Chris Douglas made changes - 11/Jun/08 11:04 PM
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12383878/2664-2.patch against trunk revision 666620. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2643/testReport/ This message is automatically generated.
I just committed this. Thanks, Chris!
Owen O'Malley made changes - 30/Jun/08 03:01 PM
Integrated in Hadoop-trunk #535 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/535/
)
Robert Chansler made changes - 21/Oct/08 11:54 PM
Nigel Daley made changes - 20/Nov/08 11:38 PM
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
New Feature
Closed
Major
lzop-compatible CompresionCodec
