Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.7.4
    • Fix Version/s: 2.7.4
    • Component/s: build, precommit
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      .gitignore is out of date on branch-2.7, which is causing issues in precommit checks for that branch.

      1. HADOOP-14686-branch-2.7.v0.patch
        2 kB
        Sean Busbey
      2. HADOOP-14686-branch-2.7.v1.patch
        0.6 kB
        Konstantin Shvachko

        Activity

        Hide
        busbey Sean Busbey added a comment -

        -00

        • just blindly copy trunk's .gitignore. looks good enough.
        Show
        busbey Sean Busbey added a comment - -00 just blindly copy trunk's .gitignore. looks good enough.
        Hide
        shv Konstantin Shvachko added a comment - - edited

        Looks like the build has failed with your patch.

        Show
        shv Konstantin Shvachko added a comment - - edited Looks like the build has failed with your patch.
        Hide
        shv Konstantin Shvachko added a comment -

        Thanks for looking into this. Could you please explain how/why .gitignore effects Yetus build?

        Also just a blind copy of trunk .gitignore may not be an exact fit, as branch-2.7 does not even have hadoop-yarn-ui: see hadoop-yarn-project.

        Show
        shv Konstantin Shvachko added a comment - Thanks for looking into this. Could you please explain how/why .gitignore effects Yetus build? Also just a blind copy of trunk .gitignore may not be an exact fit, as branch-2.7 does not even have hadoop-yarn-ui : see hadoop-yarn-project .
        Hide
        busbey Sean Busbey added a comment -

        the patch will fail until the updated .gitignore is in place because of how Hadoop's use of precommit works.

        Essentially, the patchprocess directory is where Yetus Test Patch keeps its working files, like the output logs it writes to. On the Hadoop project, we keep that directory inside of the working directory of the source checkout (due to legacy reasons that Allen Wittenauer has mentioned a few times). As a start-up step, Yetus cleans up the local git working directory to ensure that it is testing the patch against the then-current version of whatever relevant branch the patch is for.

        On most hadoop branches, .gitignore ensure that git doesn't mess with the patchprocess directory when this cleanup is happening. On branch-2.7, the ignore directive is missing and so the "clean up git" step wipes out the in-progress log files that Yetus expects to keep writing to. This leads to the failures you've seen.

        I'm happy to update my patch for whatever review feedback you have, but a blind copy seems like the fastest way to builds working again. {{.gitignore}}ing files that don't exist and won't be created by the build process on a particular branch does no harm, after all.

        Show
        busbey Sean Busbey added a comment - the patch will fail until the updated .gitignore is in place because of how Hadoop's use of precommit works. Essentially, the patchprocess directory is where Yetus Test Patch keeps its working files, like the output logs it writes to. On the Hadoop project, we keep that directory inside of the working directory of the source checkout (due to legacy reasons that Allen Wittenauer has mentioned a few times). As a start-up step, Yetus cleans up the local git working directory to ensure that it is testing the patch against the then-current version of whatever relevant branch the patch is for. On most hadoop branches, .gitignore ensure that git doesn't mess with the patchprocess directory when this cleanup is happening. On branch-2.7, the ignore directive is missing and so the "clean up git" step wipes out the in-progress log files that Yetus expects to keep writing to. This leads to the failures you've seen. I'm happy to update my patch for whatever review feedback you have, but a blind copy seems like the fastest way to builds working again. {{.gitignore}}ing files that don't exist and won't be created by the build process on a particular branch does no harm, after all.
        Hide
        aw Allen Wittenauer added a comment -

        +1 committing to branch.

        Show
        aw Allen Wittenauer added a comment - +1 committing to branch.
        Hide
        shv Konstantin Shvachko added a comment -

        Thank you for clarification Sean Busbey.

        Show
        shv Konstantin Shvachko added a comment - Thank you for clarification Sean Busbey .
        Hide
        shv Konstantin Shvachko added a comment -

        Will have to revert this, because it also removed contract-test-options.xml from .gitignore, which could be considered as incompatible change for 2.7.4.

        Show
        shv Konstantin Shvachko added a comment - Will have to revert this, because it also removed contract-test-options.xml from .gitignore, which could be considered as incompatible change for 2.7.4.
        Hide
        shv Konstantin Shvachko added a comment -

        Here is a modified patch.

        Show
        shv Konstantin Shvachko added a comment - Here is a modified patch.
        Hide
        busbey Sean Busbey added a comment -

        So v1 would be post rollback? If so, +1 (non-binding) looks fine.

        Why not just post an addendum that puts contract-test-options.xml back? (Does the build system create that file? I didn't see it in the repo)

        Show
        busbey Sean Busbey added a comment - So v1 would be post rollback? If so, +1 (non-binding) looks fine. Why not just post an addendum that puts contract-test-options.xml back? (Does the build system create that file? I didn't see it in the repo)
        Hide
        shv Konstantin Shvachko added a comment -

        Also updated CHANGES.txt.
        I just committed this to branch 2.7. Thank you Sean Busbey.

        Show
        shv Konstantin Shvachko added a comment - Also updated CHANGES.txt. I just committed this to branch 2.7. Thank you Sean Busbey .
        Hide
        shv Konstantin Shvachko added a comment - - edited

        There were some discussion about contract-test-options.xml in e.g. HADOOP-13929 and HADOOP-14264. Cannot afford incompatibilities in a dot release.

        BTW, the build is still failing. It went past "Confirming git environment", but broke further down while "Building base image: yetus/hadoop". Any thoughts?
        Here is the build link

        Show
        shv Konstantin Shvachko added a comment - - edited There were some discussion about contract-test-options.xml in e.g. HADOOP-13929 and HADOOP-14264 . Cannot afford incompatibilities in a dot release. BTW, the build is still failing. It went past "Confirming git environment", but broke further down while "Building base image: yetus/hadoop". Any thoughts? Here is the build link
        Hide
        busbey Sean Busbey added a comment -

        The build output says it failed while building the docker image:

        Step 10 : RUN mkdir -p /opt/findbugs &&     curl -L https://sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download          -o /opt/findbugs.tar.gz &&     tar xzf /opt/findbugs.tar.gz --strip-components 1 -C /opt/findbugs
         ---> Running in c1565d29ddf9
          % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                         Dload  Upload   Total   Spent    Left  Speed
        
          0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
          0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
          0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
        curl: (35) Unknown SSL protocol error in connection to sourceforge.net:443 
        The command '/bin/sh -c mkdir -p /opt/findbugs &&     curl -L https://sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download          -o /opt/findbugs.tar.gz &&     tar xzf /opt/findbugs.tar.gz --strip-components 1 -C /opt/findbugs' returned a non-zero code: 35
        
        Total Elapsed time:   0m  0s
        
        ERROR: Docker failed to build image.
        

        The relevant bit:

        Unknown SSL protocol error in connection to sourceforge.net

        Probably a transient issue.

        Show
        busbey Sean Busbey added a comment - The build output says it failed while building the docker image: Step 10 : RUN mkdir -p /opt/findbugs && curl -L https: //sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download -o /opt/findbugs.tar.gz && tar xzf /opt/findbugs.tar.gz --strip-components 1 -C /opt/findbugs ---> Running in c1565d29ddf9  % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (35) Unknown SSL protocol error in connection to sourceforge.net:443 The command '/bin/sh -c mkdir -p /opt/findbugs && curl -L https: //sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download -o /opt/findbugs.tar.gz && tar xzf /opt/findbugs.tar.gz --strip-components 1 -C /opt/findbugs' returned a non-zero code: 35 Total Elapsed time: 0m 0s ERROR: Docker failed to build image. The relevant bit: Unknown SSL protocol error in connection to sourceforge.net Probably a transient issue.
        Hide
        shv Konstantin Shvachko added a comment -

        I tried a few. Here is the most recent failure: hadoop build #12867. While other builds started around same time are doing fine beyond this point, example.
        I created HADOOP-14689 for further testing build failure.

        Show
        shv Konstantin Shvachko added a comment - I tried a few. Here is the most recent failure: hadoop build #12867 . While other builds started around same time are doing fine beyond this point, example . I created HADOOP-14689 for further testing build failure.
        Hide
        aw Allen Wittenauer added a comment -

        It's an apples and oranges comparison, really. Docker images are (effectively) built per-git hash. They are also cached with a one week expiry. trunk's is very well cached because it gets used frequently. You can tell because of lots of " ---> Using cache" lines in the 20424 build. The 12867 build uses a different hash (since 2.7 and trunk have different requirements) and has nearly nothing cached. Thus it is building completely from scratch. The location of the Dockerfile is in dev-support/docker in the hadoop source tree if you'd like to take a look at them.

        Show
        aw Allen Wittenauer added a comment - It's an apples and oranges comparison, really. Docker images are (effectively) built per-git hash. They are also cached with a one week expiry. trunk's is very well cached because it gets used frequently. You can tell because of lots of " ---> Using cache" lines in the 20424 build. The 12867 build uses a different hash (since 2.7 and trunk have different requirements) and has nearly nothing cached. Thus it is building completely from scratch. The location of the Dockerfile is in dev-support/docker in the hadoop source tree if you'd like to take a look at them.
        Hide
        shv Konstantin Shvachko added a comment -

        The question is really how do we make the branch build work. Can do fruit comparison with you later.

        Show
        shv Konstantin Shvachko added a comment - The question is really how do we make the branch build work. Can do fruit comparison with you later.
        Hide
        aw Allen Wittenauer added a comment -

        branch-2.7's ability to use the current precommit setup is sort of irrelevant as far as it's ability to build and get released. Hadoop 2.7.0 was released on the same day that HADOOP-11746 was made official. That's important because it also means that the test-patch used during 2.7.0's development wasn't the rewrite and didn't support branch switching, much less Docker.

        Show
        aw Allen Wittenauer added a comment - branch-2.7's ability to use the current precommit setup is sort of irrelevant as far as it's ability to build and get released. Hadoop 2.7.0 was released on the same day that HADOOP-11746 was made official. That's important because it also means that the test-patch used during 2.7.0's development wasn't the rewrite and didn't support branch switching, much less Docker.
        Hide
        shv Konstantin Shvachko added a comment -

        branch-2.7's ability to use the current precommit setup is sort of irrelevant as far as it's ability to build and get released.

        Not exactly sure what's irrelevant here.
        Hope you are not saying you revoked Yetus support for the Hadoop stable release. By not integrating HADOOP-11746 into branch-2?
        So what is your recommendations for restoring branch-2 build, as it worked 2 weeks ago?

        Show
        shv Konstantin Shvachko added a comment - branch-2.7's ability to use the current precommit setup is sort of irrelevant as far as it's ability to build and get released. Not exactly sure what's irrelevant here. Hope you are not saying you revoked Yetus support for the Hadoop stable release. By not integrating HADOOP-11746 into branch-2? So what is your recommendations for restoring branch-2 build, as it worked 2 weeks ago?
        Hide
        aw Allen Wittenauer added a comment -

        At this point, it's obvious that the conversation is no longer productive or constructive. I'll be removing myself as a watcher.

        BTW, you're very welcome on getting a branch that was never designed to work with Apache Yetus haphazardly working again.

        Show
        aw Allen Wittenauer added a comment - At this point, it's obvious that the conversation is no longer productive or constructive. I'll be removing myself as a watcher. BTW, you're very welcome on getting a branch that was never designed to work with Apache Yetus haphazardly working again.
        Hide
        busbey Sean Busbey added a comment - - edited

        Hope you are not saying you revoked Yetus support for the Hadoop stable release. By not integrating HADOOP-11746 into branch-2?
        So what is your recommendations for restoring branch-2 build, as it worked 2 weeks ago?

        I don't know how Allen could have possibly done this. He's just saying he's not willing to spend his time as a volunteer working on a particular branch, a position he's held consistently about branch-2 derivatives for a long time. Frankly, I was pretty surprised when he was willing to spend time on this JIRA at all. HADOOP-11746 is entirely irrelevant to our current issue. The result of it was a version of the precommit testing that we've since replaced in Hadoop like 2 or 3 time over.

        I am willing to spend my time as a volunteer chasing this down, because I personally try to push folks towards the 2.7.z releases. I only ask that you try to be a little more patient in your phrasing. I tend to be bursty in Hadoop because my attention is often elsewhere, so I have no way to know what the condition of the ASF build server was when you observed things working on branch-2.7. I can help chase down specific failures, but a different JIRA that tracks examples will be more helpful than an extended conversation on an issue we've already solved.

        Show
        busbey Sean Busbey added a comment - - edited Hope you are not saying you revoked Yetus support for the Hadoop stable release. By not integrating HADOOP-11746 into branch-2? So what is your recommendations for restoring branch-2 build, as it worked 2 weeks ago? I don't know how Allen could have possibly done this. He's just saying he's not willing to spend his time as a volunteer working on a particular branch, a position he's held consistently about branch-2 derivatives for a long time. Frankly, I was pretty surprised when he was willing to spend time on this JIRA at all. HADOOP-11746 is entirely irrelevant to our current issue. The result of it was a version of the precommit testing that we've since replaced in Hadoop like 2 or 3 time over. I am willing to spend my time as a volunteer chasing this down, because I personally try to push folks towards the 2.7.z releases. I only ask that you try to be a little more patient in your phrasing. I tend to be bursty in Hadoop because my attention is often elsewhere, so I have no way to know what the condition of the ASF build server was when you observed things working on branch-2.7. I can help chase down specific failures, but a different JIRA that tracks examples will be more helpful than an extended conversation on an issue we've already solved.
        Hide
        shv Konstantin Shvachko added a comment -

        We can use HADOOP-14689 for further fixing the build. Thank you Sean Busbey for your help and appreciate your efforts.

        I am not well familiar with Yetus and definitely missing context on the branch-2.7 / Yetus relationship, as well as on Allen Wittenauer's position in this matter. I assumed the build should work since it did just recently. Sorry if it gave an impression of me being impatient.

        Show
        shv Konstantin Shvachko added a comment - We can use HADOOP-14689 for further fixing the build. Thank you Sean Busbey for your help and appreciate your efforts. I am not well familiar with Yetus and definitely missing context on the branch-2.7 / Yetus relationship, as well as on Allen Wittenauer 's position in this matter. I assumed the build should work since it did just recently. Sorry if it gave an impression of me being impatient.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        FWIW, I still don't know why contract-test-options.xml went out of .gitignore in trunk, only that at some point it happened.

        git status
        On branch trunk
        Your branch is up-to-date with 'apache/trunk'.
        Untracked files:
          (use "git add <file>..." to include in what will be committed)
        
        	../hadoop-aws/src/test/resources/contract-test-options.xml
        

        it only shows up if you are doing things like testing our object stores

        Show
        stevel@apache.org Steve Loughran added a comment - FWIW, I still don't know why contract-test-options.xml went out of .gitignore in trunk, only that at some point it happened. git status On branch trunk Your branch is up-to-date with 'apache/trunk'. Untracked files: (use "git add <file>..." to include in what will be committed) ../hadoop-aws/src/test/resources/contract-test-options.xml it only shows up if you are doing things like testing our object stores

          People

          • Assignee:
            busbey Sean Busbey
            Reporter:
            busbey Sean Busbey
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development