Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7172

Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI.

Details

    • Bug
    • Status: Closed
    • Trivial
    • Resolution: Fixed
    • None
    • 2.6.0
    • test
    • None
    • Reviewed

    Description

      TestHDFSCLI uses several files at src/test/resource/data* as test input files. Some of the tests expect a specific length for these files. If they get checked out of git with CRLF line endings by mistake, then the test assertions will fail.

      Attachments

        1. HDFS-7172.1.patch
          0.3 kB
          Chris Nauroth
        2. HDFS-7172.rat.patch
          0.6 kB
          Chris Nauroth

        Activity

          cnauroth Chris Nauroth added a comment -

          Here is a trivial patch that adds a .gitattributes file to specify the correct line endings for the file. I've verified that this fix works on a VM that previously had git settings that were causing the data files to be checked out as CRLF.

          cnauroth Chris Nauroth added a comment - Here is a trivial patch that adds a .gitattributes file to specify the correct line endings for the file. I've verified that this fix works on a VM that previously had git settings that were causing the data files to be checked out as CRLF.
          wheat9 Haohui Mai added a comment -

          +1. I'll commit it shortly.

          wheat9 Haohui Mai added a comment - +1. I'll commit it shortly.
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12672093/HDFS-7172.1.patch
          against trunk revision 17d1202.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS
          org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover

          The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.utilTests
          oTests
          org.apache.hadoop.tools.TestJMXGet

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8285//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8285//artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8285//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12672093/HDFS-7172.1.patch against trunk revision 17d1202. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.utilTests oTests org.apache.hadoop.tools.TestJMXGet +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8285//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8285//artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8285//console This message is automatically generated.
          wheat9 Haohui Mai added a comment -

          I've committed the patch to trunk and branch-2. Thanks cnauroth for the contribution.

          wheat9 Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks cnauroth for the contribution.
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #6166 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6166/)
          HDFS-7172. Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. (wheat9: rev 737f280ddeed58a2b1cd42c29533a01e7c6c3426)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/.gitattributes
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #6166 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6166/ ) HDFS-7172 . Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. (wheat9: rev 737f280ddeed58a2b1cd42c29533a01e7c6c3426) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/.gitattributes
          cnauroth Chris Nauroth added a comment -

          Thanks for the commit, Haohui.

          cnauroth Chris Nauroth added a comment - Thanks for the commit, Haohui.

          Can we add a rat exclude to the newly added .gitattributes file?

          kkambatl Karthik Kambatla (Inactive) added a comment - Can we add a rat exclude to the newly added .gitattributes file?
          cnauroth Chris Nauroth added a comment -

          Thanks, kasha. I wonder why Jenkins didn't catch it. Here is an addendum patch. I've tested by running mvn apache-rat:check manually. Does this look good to commit?

          cnauroth Chris Nauroth added a comment - Thanks, kasha . I wonder why Jenkins didn't catch it. Here is an addendum patch. I've tested by running mvn apache-rat:check manually. Does this look good to commit?

          Yep, the addendum looks good to me.

          kkambatl Karthik Kambatla (Inactive) added a comment - Yep, the addendum looks good to me.
          wheat9 Haohui Mai added a comment -

          Thanks kasha for pointing it out. I'll commit the addendum patch shortly.

          wheat9 Haohui Mai added a comment - Thanks kasha for pointing it out. I'll commit the addendum patch shortly.
          wheat9 Haohui Mai added a comment -

          I've committed the addendum patch to trunk, branch-2 and branch-2.6. Thanks again for kasha for reporting the problem and cnauroth for the patch.

          wheat9 Haohui Mai added a comment - I've committed the addendum patch to trunk, branch-2 and branch-2.6. Thanks again for kasha for reporting the problem and cnauroth for the patch.
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #6171 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6171/)
          Addendum patch for HDFS-7172. Contributed by Chris Nauroth. (wheat9: rev 8dfe54f6d3e2f14584846de29ee06ed280bc0f0e)

          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #6171 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6171/ ) Addendum patch for HDFS-7172 . Contributed by Chris Nauroth. (wheat9: rev 8dfe54f6d3e2f14584846de29ee06ed280bc0f0e) hadoop-hdfs-project/hadoop-hdfs/pom.xml
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #698 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/698/)
          HDFS-7172. Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. (wheat9: rev 737f280ddeed58a2b1cd42c29533a01e7c6c3426)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/.gitattributes
            Addendum patch for HDFS-7172. Contributed by Chris Nauroth. (wheat9: rev 8dfe54f6d3e2f14584846de29ee06ed280bc0f0e)
          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #698 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/698/ ) HDFS-7172 . Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. (wheat9: rev 737f280ddeed58a2b1cd42c29533a01e7c6c3426) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/.gitattributes Addendum patch for HDFS-7172 . Contributed by Chris Nauroth. (wheat9: rev 8dfe54f6d3e2f14584846de29ee06ed280bc0f0e) hadoop-hdfs-project/hadoop-hdfs/pom.xml
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1889 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1889/)
          HDFS-7172. Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. (wheat9: rev 737f280ddeed58a2b1cd42c29533a01e7c6c3426)

          • hadoop-hdfs-project/hadoop-hdfs/.gitattributes
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            Addendum patch for HDFS-7172. Contributed by Chris Nauroth. (wheat9: rev 8dfe54f6d3e2f14584846de29ee06ed280bc0f0e)
          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1889 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1889/ ) HDFS-7172 . Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. (wheat9: rev 737f280ddeed58a2b1cd42c29533a01e7c6c3426) hadoop-hdfs-project/hadoop-hdfs/.gitattributes hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Addendum patch for HDFS-7172 . Contributed by Chris Nauroth. (wheat9: rev 8dfe54f6d3e2f14584846de29ee06ed280bc0f0e) hadoop-hdfs-project/hadoop-hdfs/pom.xml
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1914 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1914/)
          HDFS-7172. Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. (wheat9: rev 737f280ddeed58a2b1cd42c29533a01e7c6c3426)

          • hadoop-hdfs-project/hadoop-hdfs/.gitattributes
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            Addendum patch for HDFS-7172. Contributed by Chris Nauroth. (wheat9: rev 8dfe54f6d3e2f14584846de29ee06ed280bc0f0e)
          • hadoop-hdfs-project/hadoop-hdfs/pom.xml
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1914 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1914/ ) HDFS-7172 . Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. Contributed by Chris Nauroth. (wheat9: rev 737f280ddeed58a2b1cd42c29533a01e7c6c3426) hadoop-hdfs-project/hadoop-hdfs/.gitattributes hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Addendum patch for HDFS-7172 . Contributed by Chris Nauroth. (wheat9: rev 8dfe54f6d3e2f14584846de29ee06ed280bc0f0e) hadoop-hdfs-project/hadoop-hdfs/pom.xml

          People

            cnauroth Chris Nauroth
            cnauroth Chris Nauroth
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: