Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6200

Create a separate jar for hdfs-client

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • build
    • None
    • Hide
      Projects that access HDFS can depend on the hadoop-hdfs-client module instead of the hadoop-hdfs module to avoid pulling in unnecessary dependency.
      Please note that hadoop-hdfs-client module could miss class like ConfiguredFailoverProxyProvider. So if a cluster is in HA deployment, we should still use hadoop-hdfs instead.
      Show
      Projects that access HDFS can depend on the hadoop-hdfs-client module instead of the hadoop-hdfs module to avoid pulling in unnecessary dependency. Please note that hadoop-hdfs-client module could miss class like ConfiguredFailoverProxyProvider. So if a cluster is in HA deployment, we should still use hadoop-hdfs instead.

    Description

      Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie.

      This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency.

      Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar.

      Attachments

        1. HDFS-6200.000.patch
          453 kB
          Haohui Mai
        2. HDFS-6200.001.patch
          478 kB
          Haohui Mai
        3. HDFS-6200.002.patch
          480 kB
          Haohui Mai
        4. HDFS-6200.003.patch
          481 kB
          Haohui Mai
        5. HDFS-6200.004.patch
          481 kB
          Haohui Mai
        6. HDFS-6200.005.patch
          481 kB
          Haohui Mai
        7. HDFS-6200.006.patch
          529 kB
          Haohui Mai
        8. HDFS-6200.007.patch
          537 kB
          Haohui Mai

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-9301 HDFS clients can't construct HdfsConfiguration instances

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6566 Separate public and private keys in DFSConfigKey

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6564 Use slf4j instead of common-logging in hdfs-client

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6567 Normalize the order of public final in HdfsFileStatus

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-6565 Use jackson instead jetty json in hdfs-client

          • Major - Major loss of function.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-11656 Classpath isolation for downstream clients

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-1815 Separate client and server jars

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HDFS-9815 Move o.a.h.fs.Hdfs to hadoop-hdfs-client

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-10741 A lightweight WebHDFS client library

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12499 Create a client jar for hadoop-common

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-8048 Umbrella jira for moving HDFS client implmentation to a package

          • Major - Major loss of function.
          • Open
          There are no Sub-Tasks for this issue.

          Activity

            People

              wheat9 Haohui Mai
              wheat9 Haohui Mai
              Votes:
              0 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: