Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Jena 2.13.0
    • Component/s: RDF Tools for Hadoop
    • Labels:
      None

      Description

      Currently the Elephas modules no longer build due to a bug in the upstream Hadoop libraries related to Guava - HADOOP-11032

      Rolling back to 752646c i.e. prior to my merging of the latest Jena master changes the issue does not manifest which means this likely relates to the introduction of Guava as a dependency into Jena since the Hadoop libraries were relying on a really old version of Guava. Looking at mvn dependency:tree this indeed appears to be the case since the managed dependency of Guava in the Jena parent POM overrides the transitive dependency from Hadoop

      A temporary workaround may be to override back to the outdated Guava in the elephas POM while awaiting a fix for the Hadoop issue

        Issue Links

          Activity

          Hide
          rvesse Rob Vesse added a comment -

          Yes though HADOOP-11032 which is the specific issue seen in Elephas is likely to get addressed since that can be resolved by replacing that specific usage of Guava

          Show
          rvesse Rob Vesse added a comment - Yes though HADOOP-11032 which is the specific issue seen in Elephas is likely to get addressed since that can be resolved by replacing that specific usage of Guava
          Hide
          andy.seaborne Andy Seaborne added a comment -

          Sorry for breaking the Elephas build.

          For now, <dependencyManagement> of Guava removed and jena-csv fixed to not need Guava (which was quite easy). This Guava version problem is likely to bite even if though this particular case has a workaround from 7ca24c0.

          The trouble with HADOOP-10101 (nice number!) is that it is not making much progress.

          The changes made make least surprise at the moment.

          Show
          andy.seaborne Andy Seaborne added a comment - Sorry for breaking the Elephas build. For now, <dependencyManagement> of Guava removed and jena-csv fixed to not need Guava (which was quite easy). This Guava version problem is likely to bite even if though this particular case has a workaround from 7ca24c0. The trouble with HADOOP-10101 (nice number!) is that it is not making much progress. The changes made make least surprise at the moment.
          Hide
          andy.seaborne Andy Seaborne added a comment -

          I'm fine with just rolling back the <dependencyManagement> for now and making restricted use of Guava in jena-tdb (eventually; cache not done there yet anyway) and jena-csv.

          jena-csv usage is only in PropertyTableHashMapImpl. It looks like it can be replaced with the less good but adequate for the task oaj.atlas.lib.MultiMap.

          Show
          andy.seaborne Andy Seaborne added a comment - I'm fine with just rolling back the <dependencyManagement> for now and making restricted use of Guava in jena-tdb (eventually; cache not done there yet anyway) and jena-csv. jena-csv usage is only in PropertyTableHashMapImpl . It looks like it can be replaced with the less good but adequate for the task oaj.atlas.lib.MultiMap.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 7ca24c0682e80799e9c88612759ff3bf3640d890 in jena's branch refs/heads/hadoop-rdf from Rob Vesse
          [ https://git-wip-us.apache.org/repos/asf?p=jena.git;h=7ca24c0 ]

          Workaround for JENA-842

          Set Elephas to use outdated version of Guava to avoid a conflict with
          Hadoop version that the jena-parent otherwise introduces. This is a
          temporary workaround that can be removed as and when HADOOP-11032 is
          resolved and a release of Hadoop with an up to date version of Guava is
          available

          Show
          jira-bot ASF subversion and git services added a comment - Commit 7ca24c0682e80799e9c88612759ff3bf3640d890 in jena's branch refs/heads/hadoop-rdf from Rob Vesse [ https://git-wip-us.apache.org/repos/asf?p=jena.git;h=7ca24c0 ] Workaround for JENA-842 Set Elephas to use outdated version of Guava to avoid a conflict with Hadoop version that the jena-parent otherwise introduces. This is a temporary workaround that can be removed as and when HADOOP-11032 is resolved and a release of Hadoop with an up to date version of Guava is available
          Hide
          rvesse Rob Vesse added a comment - - edited

          The problem isn't quite the same, Elephas doesn't need Guava at all itself but the Hadoop runtime does have a transitive dependency on Guava. The problem is with the use of <dependencyManagement>, as there is an entry for Guava in jena-parent the managed version there overrides the transitively resolved dependency from Hadoop which leads to the build error.

          In the longer term the Hadoop folks have a patch which would allow them to update to the latest Guava and resolve the conflict.

          Show
          rvesse Rob Vesse added a comment - - edited The problem isn't quite the same, Elephas doesn't need Guava at all itself but the Hadoop runtime does have a transitive dependency on Guava. The problem is with the use of <dependencyManagement> , as there is an entry for Guava in jena-parent the managed version there overrides the transitively resolved dependency from Hadoop which leads to the build error. In the longer term the Hadoop folks have a patch which would allow them to update to the latest Guava and resolve the conflict.
          Hide
          andy.seaborne Andy Seaborne added a comment - - edited

          Breaking the Elephas build is not good - do reverse the jena-parent management of Guava. (jena-csv needs version set but otherwise it's not actually used yet; though I was hoing to use it for the concurrent cache for TDB where it's been shown to make things better.)

          How about the suggestion in:

          http://mail-archives.apache.org/mod_mbox/jena-dev/201501.mbox/%3C54A6F60F.1050106%40apache.org%3E

          ?

          That's is, limit Guava to specific modules (jena-dtb, jena-csv).

          Plan B : Shade the relevant cache bits
          Plan C: Take a code copy of the relevant cache bits and repackage.

          Show
          andy.seaborne Andy Seaborne added a comment - - edited Breaking the Elephas build is not good - do reverse the jena-parent management of Guava. (jena-csv needs version set but otherwise it's not actually used yet; though I was hoing to use it for the concurrent cache for TDB where it's been shown to make things better.) How about the suggestion in: http://mail-archives.apache.org/mod_mbox/jena-dev/201501.mbox/%3C54A6F60F.1050106%40apache.org%3E ? That's is, limit Guava to specific modules (jena-dtb, jena-csv). Plan B : Shade the relevant cache bits Plan C: Take a code copy of the relevant cache bits and repackage.

            People

            • Assignee:
              rvesse Rob Vesse
              Reporter:
              rvesse Rob Vesse
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development