Hadoop Common
  1. Hadoop Common
  2. HADOOP-8361

Avoid out-of-memory problems when deserializing strings

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In HDFS, we want to be able to read the edit log without crashing on an OOM condition. Unfortunately, we currently cannot do this, because there are no limits on the length of certain data types we pull from the edit log. We often read strings without setting any upper limit on the length we're prepared to accept.

      It's not that we don't have limits on strings-- for example, HDFS limits the maximum path length to 8000 UCS-2 characters. Linux limits the maximum user name length to either 64 or 128 bytes, depending on what version you are running. It's just that we're not exposing these limits to the deserialization functions that need to be aware of them.

      1. HADOOP-8361.001.patch
        8 kB
        Colin Patrick McCabe
      2. HADOOP-8361.002.patch
        10 kB
        Colin Patrick McCabe
      3. HADOOP-8361.003.patch
        12 kB
        Colin Patrick McCabe
      4. HADOOP-8361.004.patch
        14 kB
        Colin Patrick McCabe
      5. HADOOP-8361.005.patch
        14 kB
        Colin Patrick McCabe
      6. HADOOP-8361.006.patch
        8 kB
        Colin Patrick McCabe
      7. HADOOP-8361.007.patch
        8 kB
        Colin Patrick McCabe

        Issue Links

          Activity

          Hide
          Colin Patrick McCabe added a comment -
          • avoid OOM on deserializing BlockLocation
          Show
          Colin Patrick McCabe added a comment - avoid OOM on deserializing BlockLocation
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12525683/HADOOP-8361.001.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.viewfs.TestViewFsTrash

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/942//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/942//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525683/HADOOP-8361.001.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.viewfs.TestViewFsTrash +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/942//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/942//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Rather than having this "SHORT" constant in Text, I think it makes more sense to have the API be Text.writeLengthLimitedString(str, limit) (and same for read). Otherwise you have to go digging into the code to find out what "short" means. And like you said, different areas of the code may have different requirements.

          Show
          Todd Lipcon added a comment - Rather than having this "SHORT" constant in Text, I think it makes more sense to have the API be Text.writeLengthLimitedString(str, limit) (and same for read). Otherwise you have to go digging into the code to find out what "short" means. And like you said, different areas of the code may have different requirements.
          Hide
          Todd Lipcon added a comment -

          Also, does BlockLocation's Writable implementation get used anywhere anymore? Block locations aren't serialized in the fsimage, so I don't know if we need to change it at this point - and perhaps we could file a separate JIRA to remove it entirely if it's unused.

          Show
          Todd Lipcon added a comment - Also, does BlockLocation's Writable implementation get used anywhere anymore? Block locations aren't serialized in the fsimage, so I don't know if we need to change it at this point - and perhaps we could file a separate JIRA to remove it entirely if it's unused.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12525687/HADOOP-8361.002.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/943//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/943//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525687/HADOOP-8361.002.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/943//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/943//console This message is automatically generated.
          Hide
          Colin Patrick McCabe added a comment -

          basically, the "real" limits look like this:

          Path length: On Linux, PATH_MAX is 4096
          On HDFS:, we limit the path length to HdfsConstants.MAX_PATH_LENGTH, which is 8000 UCS-2 code points, or 24000 UTF-8 bytes.

          Username length: On some versions of Linux, this is 64. On others, it's 128. I think it's 256 on Windows, but I'm not sure.

          So basically, all of these things are far, far shorter than a megabyte. A megabyte is just a convenient sanity check size. Remember that this is common code, so we need to support all platforms and configurations.

          P.S. on second thought, ACLs aren't used in HDFS, so let's drop that from the patch for now.

          Show
          Colin Patrick McCabe added a comment - basically, the "real" limits look like this: Path length: On Linux, PATH_MAX is 4096 On HDFS:, we limit the path length to HdfsConstants.MAX_PATH_LENGTH, which is 8000 UCS-2 code points, or 24000 UTF-8 bytes. Username length: On some versions of Linux, this is 64. On others, it's 128. I think it's 256 on Windows, but I'm not sure. So basically, all of these things are far, far shorter than a megabyte. A megabyte is just a convenient sanity check size. Remember that this is common code, so we need to support all platforms and configurations. P.S. on second thought, ACLs aren't used in HDFS, so let's drop that from the patch for now.
          Hide
          Hari Mankude added a comment -

          Can you please add negative tests to verify your changes?

          Show
          Hari Mankude added a comment - Can you please add negative tests to verify your changes?
          Hide
          Colin Patrick McCabe added a comment -
          • add unit test
          • remove unused BlockLocation serialization
          • make limits explicit
          Show
          Colin Patrick McCabe added a comment - add unit test remove unused BlockLocation serialization make limits explicit
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12525886/HADOOP-8361.004.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.TestFileStatus
          org.apache.hadoop.fs.viewfs.TestViewFsTrash
          org.apache.hadoop.fs.viewfs.TestViewfsFileStatus

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/950//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/950//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525886/HADOOP-8361.004.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.TestFileStatus org.apache.hadoop.fs.viewfs.TestViewFsTrash org.apache.hadoop.fs.viewfs.TestViewfsFileStatus +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/950//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/950//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12525886/HADOOP-8361.004.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.TestFileStatus
          org.apache.hadoop.fs.viewfs.TestViewfsFileStatus

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/951//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/951//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525886/HADOOP-8361.004.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.TestFileStatus org.apache.hadoop.fs.viewfs.TestViewfsFileStatus +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/951//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/951//console This message is automatically generated.
          Hide
          Colin Patrick McCabe added a comment -
          • fix typo that previous patch introduced in serialization of FileStatus
          Show
          Colin Patrick McCabe added a comment - fix typo that previous patch introduced in serialization of FileStatus
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12525932/HADOOP-8361.005.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/956//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/956//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525932/HADOOP-8361.005.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/956//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/956//console This message is automatically generated.
          Hide
          Colin Patrick McCabe added a comment -

          split off the BlockLocation part into a separate JIRA, since it's really unrelated

          Show
          Colin Patrick McCabe added a comment - split off the BlockLocation part into a separate JIRA, since it's really unrelated
          Hide
          Colin Patrick McCabe added a comment -
          • split off the BlockLocation changes into a separate JIRA, since they're unrelated and there is a (small) risk
          Show
          Colin Patrick McCabe added a comment - split off the BlockLocation changes into a separate JIRA, since they're unrelated and there is a (small) risk
          Hide
          Eli Collins added a comment -

          SHORT_STRING_MAX is dead code, otherwise +1

          Show
          Eli Collins added a comment - SHORT_STRING_MAX is dead code, otherwise +1
          Hide
          Eli Collins added a comment -

          Block locations change is HADOOP-8388.

          Show
          Eli Collins added a comment - Block locations change is HADOOP-8388 .
          Hide
          Colin Patrick McCabe added a comment -
          • remove SHORT_STRING_MAX deadcode
          Show
          Colin Patrick McCabe added a comment - remove SHORT_STRING_MAX deadcode
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12526436/HADOOP-8361.006.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.viewfs.TestViewFsTrash

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/980//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/980//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526436/HADOOP-8361.006.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.viewfs.TestViewFsTrash +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/980//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/980//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          I've removed the dead constant since it was trivial, committed and merged to branch-2. Thanks Colin!

          Show
          Eli Collins added a comment - I've removed the dead constant since it was trivial, committed and merged to branch-2. Thanks Colin!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2299 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2299/)
          HADOOP-8361. Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2299 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2299/ ) HADOOP-8361 . Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2224 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2224/)
          HADOOP-8361. Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2224 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2224/ ) HADOOP-8361 . Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12526446/HADOOP-8361.007.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/982//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526446/HADOOP-8361.007.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/982//console This message is automatically generated.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2242 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2242/)
          HADOOP-8361. Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945)

          Result = ABORTED
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2242 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2242/ ) HADOOP-8361 . Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945) Result = ABORTED eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/)
          HADOOP-8361. Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/ ) HADOOP-8361 . Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1076 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1076/)
          HADOOP-8361. Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1076 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1076/ ) HADOOP-8361 . Avoid out-of-memory problems when deserializing strings. Contributed by Colin Patrick McCabe (Revision 1336945) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336945 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionStatus.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java

            People

            • Assignee:
              Colin Patrick McCabe
              Reporter:
              Colin Patrick McCabe
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development