Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2060

DFS client RPCs using protobufs

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 0.23.0
    • None
    • None
    • None

    Description

      The most important place for wire-compatibility in DFS is between clients and the cluster, since lockstep upgrade is very difficult and a single client may want to talk to multiple server versions. So, I'd like to focus this JIRA on making the RPCs between the DFS client and the NN/DNs wire-compatible using protocol buffer based serialization.

      Attachments

        1. hdfs-2060-getblocklocations.txt
          196 kB
          Todd Lipcon

        Issue Links

          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-7379 Add ability to include Protobufs in ObjectWritable

          • Major - Major loss of function.
          • Closed
          is part of

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-7347 IPC Wire Compatibility

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-2478 HDFS Protocols in Protocol Buffers

          • Major - Major loss of function.
          • Resolved
          1.
          		 ClientNameNodeProtocol in Protocol Buffers Sub-task Closed Sanjay Radia
          2.
          		 ClientDatanodeProtocol in Protocol Buffers Sub-task Resolved Unassigned
          3.
          		 ClientNameNodeProtocol Translators for Protocol Buffers Sub-task Closed Sanjay Radia
          4.
          		 Turn off TestDFSOverAvroRpc Sub-task Resolved Suresh Srinivas
          5.
          		 Implement protobuf service for NamenodeProtocol Sub-task Closed Suresh Srinivas
          6.
          		 Implement protobuf service for JournalProtocol Sub-task Closed Suresh Srinivas
          7.
          		 Implement protobuf service for InterDatanodeProtocol Sub-task Closed Suresh Srinivas
          8.
          		 Implement protobuf service for ClientDatanodeProtocol Sub-task Closed Suresh Srinivas
          9.
          		 Protobuf - Add protobuf service for DatanodeProtocol Sub-task Closed Suresh Srinivas
          10.
          		 Protobuf - Add protobuf service for NamenodeProtocol Sub-task Closed Suresh Srinivas
          11.
          		 Protobuf - Add protocol service for JournalProtocol Sub-task Closed Suresh Srinivas
          12.
          		 Protobuf - Add protobuf service for InterDatanodeProtocol Sub-task Closed Suresh Srinivas
          13.
          		 Enable protobuf RPC for InterDatanodeProtocol, ClientDatanodeProtocol, JournalProtocol and NamenodeProtocol Sub-task Closed Suresh Srinivas
          14.
          		 Enable protobuf RPC for DatanodeProtocol Sub-task Closed Jitendra Nath Pandey
          15.
          		 Optional parameters are not handled correctly Sub-task Closed Suresh Srinivas
          16.
          		 Remove TestDFSOverAvroRpc Sub-task Resolved Suresh Srinivas
          17.
          		 Protobuf translators for DatanodeProtocol Sub-task Closed Jitendra Nath Pandey
          18.
          		 TestBackupNode fails Sub-task Closed Suresh Srinivas
          19.
          		 Enable protobuf rpc for ClientNamenodeProtocol Sub-task Closed Sanjay Radia
          20.
          		 Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings protocol to protocol buffers Sub-task Closed Jitendra Nath Pandey
          21.
          		 Tests are failing with ClassCastException, due to new protocol changes Sub-task Closed Suresh Srinivas
          22.
          		 Fix RPC client creation bug from HDFS-2459 Sub-task Closed Suresh Srinivas
          23.
          		 Provide a method in client side translators to check for a methods supported in underlying protocol. Sub-task Closed Jitendra Nath Pandey

          Activity

            People

              tlipcon Todd Lipcon
              tlipcon Todd Lipcon
              Votes:
              2 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: