Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7040

HDFS dangerously uses @Beta methods from very old versions of Guava

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 2.4.0, 2.5.0, 2.4.1
    • None
    • None

    Description

      HDFS uses LimitInputStream from Guava. This was introduced as @Beta and is risky for any application to use.

      The problem is further exacerbated by Hadoop's dependency on Guava version 11.0.2, which is quite old for an active project (Feb. 2012).

      Because Guava is very stable, projects which depend on Hadoop and use Guava themselves, can use up through Guava version 14.x

      However, in version 14, Guava deprecated LimitInputStream and provided a replacement. Because they make no guarantees about compatibility about @Beta classes, they removed it in version 15.

      What should be done: Hadoop should updated its dependency on Guava to at least version 14 (currently Guava is on version 19). This should have little impact on users, because Guava is so stable.

      HDFS should then be patched to use the provided alternative to LimitInputStream, so that downstream packagers, users, and application developers requiring more recent versions of Guava (to fix bugs, to use new features, etc.) will be able to swap out the Guava dependency without breaking Hadoop.

      Alternative: While Hadoop cannot predict the marking and removal of deprecated code, it can, and should, avoid the use of @Beta classes and methods that do not offer guarantees. If the dependency cannot be bumped, then it should be relatively trivial to provide an internal class with the same functionality, that does not rely on the older version of Guava.

      Attachments

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. ACCUMULO-2714 Integration test classpath issues with Guava

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. HDFS-5518 HDFS doesn't compile/run against Guava 1.5

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10101 Update guava dependency to the latest version

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-11286 Map/Reduce dangerously adds Guava @Beta class to CryptoUtils

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. ACCUMULO-3100 Accumulo fails to test against recent Hadoop 2.6.0-SNAPSHOT

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. PHOENIX-1250 guava dependency out-dated causing NoSuchMethod error

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-9667 NullOutputStream removed from Guava 15

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              Unassigned Unassigned
              ctubbsii Christopher Tubbs
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: