Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6200

Create a separate jar for hdfs-client

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: build
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Hide
      Projects that access HDFS can depend on the hadoop-hdfs-client module instead of the hadoop-hdfs module to avoid pulling in unnecessary dependency.
      Please note that hadoop-hdfs-client module could miss class like ConfiguredFailoverProxyProvider. So if a cluster is in HA deployment, we should still use hadoop-hdfs instead.
      Show
      Projects that access HDFS can depend on the hadoop-hdfs-client module instead of the hadoop-hdfs module to avoid pulling in unnecessary dependency. Please note that hadoop-hdfs-client module could miss class like ConfiguredFailoverProxyProvider. So if a cluster is in HA deployment, we should still use hadoop-hdfs instead.

      Description

      Currently the hadoop-hdfs jar contain both the hdfs server and the hdfs client. As discussed in the hdfs-dev mailing list (http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201404.mbox/browser), downstream projects are forced to bring in additional dependency in order to access hdfs. The additional dependency sometimes can be difficult to manage for projects like Apache Falcon and Apache Oozie.

      This jira proposes to create a new project, hadoop-hdfs-cliient, which contains the client side of the hdfs code. Downstream projects can use this jar instead of the hadoop-hdfs to avoid unnecessary dependency.

      Note that it does not break the compatibility of downstream projects. This is because old downstream projects implicitly depend on hadoop-hdfs-client through the hadoop-hdfs jar.

        Attachments

        1. HDFS-6200.000.patch
          453 kB
          Haohui Mai
        2. HDFS-6200.001.patch
          478 kB
          Haohui Mai
        3. HDFS-6200.002.patch
          480 kB
          Haohui Mai
        4. HDFS-6200.003.patch
          481 kB
          Haohui Mai
        5. HDFS-6200.004.patch
          481 kB
          Haohui Mai
        6. HDFS-6200.005.patch
          481 kB
          Haohui Mai
        7. HDFS-6200.006.patch
          529 kB
          Haohui Mai
        8. HDFS-6200.007.patch
          537 kB
          Haohui Mai

          Issue Links

          There are no Sub-Tasks for this issue.

            Activity

              People

              • Assignee:
                wheat9 Haohui Mai
                Reporter:
                wheat9 Haohui Mai
              • Votes:
                0 Vote for this issue
                Watchers:
                32 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: