Hadoop Common
  1. Hadoop Common
  2. HADOOP-8912

adding .gitattributes file to prevent CRLF and LF mismatches for source and text files

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha, 1-win
    • Fix Version/s: 3.0.0, 1-win, 2.0.3-alpha
    • Component/s: build
    • Labels:
      None

      Description

      Source code in hadoop-common repo has a bunch of files that have CRLF endings.
      With more development happening on windows there is a higher chance of more CRLF files getting into the source tree.
      I would like to avoid that by creating .gitattributes file which prevents sources from having CRLF entries in text files.

      I am adding a couple of links here to give more primer on what exactly is the issue and how we are trying to fix it.

      1. http://git-scm.com/docs/gitattributes#_checking_out_and_checking_in
      2. http://stackoverflow.com/questions/170961/whats-the-best-crlf-handling-strategy-with-git

      This issue for adding .gitattributes file to the tree.

      1. HADOOP-8912.trunk.patch
        0.5 kB
        Raja Aluri
      2. HADOOP-8912.branch-2.patch
        0.5 kB
        Raja Aluri
      3. HADOOP-8912.branch-1-win.patch
        0.5 kB
        Raja Aluri

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1224 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1224/)
          HADOOP-8912. Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438)
          HADOOP-8912. Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437
          Files :

          • /hadoop/common/trunk/.gitattributes
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1224 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1224/ ) HADOOP-8912 . Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438) HADOOP-8912 . Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437 Files : /hadoop/common/trunk/.gitattributes
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1193 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1193/)
          HADOOP-8912. Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438)
          HADOOP-8912. Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437
          Files :

          • /hadoop/common/trunk/.gitattributes
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1193 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1193/ ) HADOOP-8912 . Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438) HADOOP-8912 . Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437 Files : /hadoop/common/trunk/.gitattributes
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2877 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2877/)
          HADOOP-8912. Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438)
          HADOOP-8912. Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437)

          Result = FAILURE
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437
          Files :

          • /hadoop/common/trunk/.gitattributes
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2877 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2877/ ) HADOOP-8912 . Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438) HADOOP-8912 . Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437 Files : /hadoop/common/trunk/.gitattributes
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2853 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2853/)
          HADOOP-8912. Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438)
          HADOOP-8912. Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437
          Files :

          • /hadoop/common/trunk/.gitattributes
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2853 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2853/ ) HADOOP-8912 . Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438) HADOOP-8912 . Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437 Files : /hadoop/common/trunk/.gitattributes
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2915 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2915/)
          HADOOP-8912. Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438)
          HADOOP-8912. Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437
          Files :

          • /hadoop/common/trunk/.gitattributes
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2915 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2915/ ) HADOOP-8912 . Adding missing CHANGES.txt changes in the previous commit 1397437. (Revision 1397438) HADOOP-8912 . Add .gitattributes file to prevent CRLF and LF mismatches for source and text files. Contributed by Raja Aluri. (Revision 1397437) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397438 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1397437 Files : /hadoop/common/trunk/.gitattributes
          Hide
          Suresh Srinivas added a comment -

          I committed the patch to trunk, branch-2 and 1-win.

          Thank you for the patch Raja.

          Show
          Suresh Srinivas added a comment - I committed the patch to trunk, branch-2 and 1-win. Thank you for the patch Raja.
          Hide
          Suresh Srinivas added a comment -

          +1 for the patch. We should figure out a way to do this for svn and do it in a separate jira.

          Show
          Suresh Srinivas added a comment - +1 for the patch. We should figure out a way to do this for svn and do it in a separate jira.
          Hide
          Raja Aluri added a comment -

          It can definitely be achieved with a pre-commit hook with svn. A small script that disallows CRLF for known file types and text files would do the trick. It should be pretty straight forward to do this.
          I am not quite sure with apache infrastructure, if they let you run these triggers on their servers etc.,

          Show
          Raja Aluri added a comment - It can definitely be achieved with a pre-commit hook with svn. A small script that disallows CRLF for known file types and text files would do the trick. It should be pretty straight forward to do this. I am not quite sure with apache infrastructure, if they let you run these triggers on their servers etc.,
          Hide
          Aaron T. Myers added a comment -

          Ah, got it. Thanks for the explanation. It will help for those developers who clone the git repo and `git add ...' their files before generating a patch. In that case, it will help somewhat to commit this, but it won't solve the problem 100% of the time.

          Raja, do you happen to know if there's a semantically equivalent thing we could do for svn, to ensure that committers don't check in bad line endings?

          Show
          Aaron T. Myers added a comment - Ah, got it. Thanks for the explanation. It will help for those developers who clone the git repo and `git add ...' their files before generating a patch. In that case, it will help somewhat to commit this, but it won't solve the problem 100% of the time. Raja, do you happen to know if there's a semantically equivalent thing we could do for svn, to ensure that committers don't check in bad line endings?
          Hide
          Raja Aluri added a comment -

          I am not sure how many people pull from git, but this would help only a subset of developers.
          This will only prevent CR/LF from developers, who clone from a git repo, make changes and submit a patch.

          Show
          Raja Aluri added a comment - I am not sure how many people pull from git, but this would help only a subset of developers. This will only prevent CR/LF from developers, who clone from a git repo, make changes and submit a patch.
          Hide
          Aaron T. Myers added a comment -

          Maybe I'm missing something, but I don't see how this will actually prevent CRLF and LF files from getting checked in to the repo, since actual commits/merges are done via svn, not git. The svn repo is mirrored to a git repo, but that repo is read only.

          Show
          Aaron T. Myers added a comment - Maybe I'm missing something, but I don't see how this will actually prevent CRLF and LF files from getting checked in to the repo, since actual commits/merges are done via svn, not git. The svn repo is mirrored to a git repo, but that repo is read only.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12548647/HADOOP-8912.trunk.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1606//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1606//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548647/HADOOP-8912.trunk.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1606//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1606//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12548601/HADOOP-8912.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1599//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1599//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548601/HADOOP-8912.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1599//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1599//console This message is automatically generated.
          Hide
          Raja Aluri added a comment -

          I have patches ready for trunk also. I am just waiting for this to be accepted and will submit the other patches.

          Show
          Raja Aluri added a comment - I have patches ready for trunk also. I am just waiting for this to be accepted and will submit the other patches.
          Hide
          Eli Collins added a comment -

          Let's do this for trunk and branch-2 as well.

          Show
          Eli Collins added a comment - Let's do this for trunk and branch-2 as well.
          Hide
          Raja Aluri added a comment -

          Please do not submit this, until HADOOP-8911 is submitted.

          Show
          Raja Aluri added a comment - Please do not submit this, until HADOOP-8911 is submitted.

            People

            • Assignee:
              Raja Aluri
              Reporter:
              Raja Aluri
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development