Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0
    • Component/s: None
    • Labels:
      None

      Description

      Some Hive builds started failing after the upgrade to Hive 0.10.0:

      http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-Hive/306/

      I have tried unsuccessfully to reproduce the problem on a couple of the affected operating systems - I believe the problem is a stale ivy cache. We could try invoking the relevant ant target in do-component-build, or perhaps we can just do it manually on each of the affected Jenkins slaves.

      1. BIGTOP-873.1.patch
        0.9 kB
        Sean Mackrory
      2. BIGTOP-873.patch
        0.9 kB
        Sean Mackrory

        Issue Links

          Activity

          Hide
          Sean Mackrory added a comment -

          Resolving - we'll just clear the cache manually if this is a problem again

          Show
          Sean Mackrory added a comment - Resolving - we'll just clear the cache manually if this is a problem again
          Hide
          Sean Mackrory added a comment -

          Sorry, Cos - meant to weigh in yesterday but the times I remembered and the times I had working Internet didn't overlap much

          +1 to rolling it back as you did - it was indeed meant to fix a stale ivy cache which was breaking Jenkins builds. That shouldn't normally be a problem, so if clearing it causes new problems we shouldn't be clearing it. If it happens again we'll just need to figure out how we clear the Ivy cache manually on the machines in question.

          Thanks for resolving the issue!

          Show
          Sean Mackrory added a comment - Sorry, Cos - meant to weigh in yesterday but the times I remembered and the times I had working Internet didn't overlap much +1 to rolling it back as you did - it was indeed meant to fix a stale ivy cache which was breaking Jenkins builds. That shouldn't normally be a problem, so if clearing it causes new problems we shouldn't be clearing it. If it happens again we'll just need to figure out how we clear the Ivy cache manually on the machines in question. Thanks for resolving the issue!
          Hide
          Konstantin Boudnik added a comment -

          I wish I can reopen and then close it as invalid, but JIRA doesn't allow me... So, I have just rolled back the change in df1c345 on master

          Show
          Konstantin Boudnik added a comment - I wish I can reopen and then close it as invalid, but JIRA doesn't allow me... So, I have just rolled back the change in df1c345 on master
          Hide
          Konstantin Boudnik added a comment -

          Mark, looks like you were trying to fix an issue with stale ivy cache. It has to be deal with before the build is kicked in - e.g. at the level of CI job. Doing such a think in a middle of the build is harmful.

          Show
          Konstantin Boudnik added a comment - Mark, looks like you were trying to fix an issue with stale ivy cache. It has to be deal with before the build is kicked in - e.g. at the level of CI job. Doing such a think in a middle of the build is harmful.
          Hide
          Mark Grover added a comment -

          Thanks, Cos for pointing this out. I agree it's a problem and that we should back out this patch. I am don't remember what issue this patch was meant to fix so I would suggest backing this out and fixing any problems that may subsequently appear when they do so.

          Show
          Mark Grover added a comment - Thanks, Cos for pointing this out. I agree it's a problem and that we should back out this patch. I am don't remember what issue this patch was meant to fix so I would suggest backing this out and fixing any problems that may subsequently appear when they do so.
          Hide
          Konstantin Boudnik added a comment -

          So, what's the input from the authors of the patch?

          Show
          Konstantin Boudnik added a comment - So, what's the input from the authors of the patch?
          Hide
          Konstantin Boudnik added a comment -

          As per my last comment.

          Show
          Konstantin Boudnik added a comment - As per my last comment.
          Hide
          Konstantin Boudnik added a comment -

          This patch has a pretty nasty side effect: an attempt to build stack on top of Hadoop bits that aren't published to the snapshots.apache.org - which is the case with many releases off Hadoop's branch-2 - you won't be able to build Hive because it won't resolve freshly installed artifacts in the ~/.m2/repository.

          Another problem of this patch is conceptual - ~/.ivy2/cache is shared by multiple components in the stack, e.g. Pig, and proposed change makes Hive a non well-behaved player of the Bigtop game - cleaning shared resources, etc. I'd say this needs to be rolled back.

          Show
          Konstantin Boudnik added a comment - This patch has a pretty nasty side effect: an attempt to build stack on top of Hadoop bits that aren't published to the snapshots.apache.org - which is the case with many releases off Hadoop's branch-2 - you won't be able to build Hive because it won't resolve freshly installed artifacts in the ~/.m2/repository. Another problem of this patch is conceptual - ~/.ivy2/cache is shared by multiple components in the stack, e.g. Pig, and proposed change makes Hive a non well-behaved player of the Bigtop game - cleaning shared resources, etc. I'd say this needs to be rolled back.
          Hide
          Roman Shaposhnik added a comment -

          +1 and committed!

          Show
          Roman Shaposhnik added a comment - +1 and committed!
          Hide
          Sean Mackrory added a comment -

          Yes - very-clean depends on ivy-clean-cache, so it could help resolve this problem. Since that's cleaner than invoking ivy-clean-cache directly, I amended the patch.

          Show
          Sean Mackrory added a comment - Yes - very-clean depends on ivy-clean-cache, so it could help resolve this problem. Since that's cleaner than invoking ivy-clean-cache directly, I amended the patch.
          Hide
          Mark Grover added a comment -

          Thanks for the patch, Sean. Hive seems to have "clean" and "very-clean" targets. Do you think they could help resolve this problem?

          Show
          Mark Grover added a comment - Thanks for the patch, Sean. Hive seems to have "clean" and "very-clean" targets. Do you think they could help resolve this problem?
          Hide
          Sean Mackrory added a comment -

          If we want to take the approach of always clearing the cache before each build, this patch should do the trick (though I never reproduced the problem locally, so I emphasize should).

          Show
          Sean Mackrory added a comment - If we want to take the approach of always clearing the cache before each build, this patch should do the trick (though I never reproduced the problem locally, so I emphasize should ).

            People

            • Assignee:
              Sean Mackrory
              Reporter:
              Sean Mackrory
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development