Hadoop Common
  1. Hadoop Common
  2. HADOOP-6904

A baby step towards inter-version RPC communications

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.23.0
    • Component/s: ipc
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      Currently RPC communications in Hadoop is very strict. If a client has a different version from that of the server, a VersionMismatched exception is thrown and the client can not connect to the server. This force us to update both client and server all at once if a RPC protocol is changed. But sometime different versions do not mean the client & server are not compatible. It would be nice if we could relax this restriction and allows us to support inter-version communications.

      My idea is that DfsClient catches VersionMismatched exception when it connects to NameNode. It then checks if the client & the server is compatible. If yes, it sets the NameNode version in the dfs client and allows the client to continue talking to NameNode. Otherwise, rethrow the VersionMismatch exception.

      1. rpcVersion.patch
        17 kB
        Hairong Kuang
      2. rpcVersion1.patch
        18 kB
        Hairong Kuang
      3. majorMinorVersion.patch
        16 kB
        Hairong Kuang
      4. majorMinorVersion1.patch
        19 kB
        Hairong Kuang
      5. rpcCompatible-trunk.patch
        22 kB
        Hairong Kuang
      6. rpcCompatible-trunk1.patch
        29 kB
        Hairong Kuang
      7. rpcCompatible-trunk2.patch
        34 kB
        Hairong Kuang
      8. rpcCompatible-trunk4.patch
        36 kB
        Hairong Kuang
      9. rpcCompatible-trunk5.patch
        37 kB
        Hairong Kuang
      10. rpcCompatible-trunk6.patch
        37 kB
        Hairong Kuang
      11. rpcCompatible-trunk7.patch
        37 kB
        Hairong Kuang
      12. rpcCompatible-trunk8.patch
        38 kB
        Hairong Kuang
      13. rpcCompatible-trunk9.patch
        38 kB
        Hairong Kuang
      14. rpcCompatible-trunk10.patch
        43 kB
        Hairong Kuang
      15. rpcCompatible-trunk11.patch
        43 kB
        Hairong Kuang

        Issue Links

          Activity

          Hairong Kuang created issue -
          Jeff Hammerbacher made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-6659 [ HADOOP-6659 ]
          Hairong Kuang made changes -
          Project Hadoop HDFS [ 12310942 ] Hadoop Common [ 12310240 ]
          Key HDFS-1329 HADOOP-6904
          Affects Version/s 0.22.0 [ 12314296 ]
          Affects Version/s 0.22.0 [ 12314241 ]
          Fix Version/s 0.22.0 [ 12314296 ]
          Fix Version/s 0.22.0 [ 12314241 ]
          Component/s ipc [ 12310688 ]
          Component/s hdfs client [ 12312928 ]
          Hairong Kuang made changes -
          Attachment rpcVersion.patch [ 12451469 ]
          Hairong Kuang made changes -
          Attachment rpcVersion1.patch [ 12451492 ]
          Hairong Kuang made changes -
          Attachment rpcVersion1.patch [ 12451492 ]
          Hairong Kuang made changes -
          Attachment rpcVersion1.patch [ 12451523 ]
          Doug Cutting made changes -
          Link This issue blocks HDFS-1335 [ HDFS-1335 ]
          Hairong Kuang made changes -
          Attachment majorMinorVersion.patch [ 12460154 ]
          Hairong Kuang made changes -
          Attachment majorMinorVersion1.patch [ 12460393 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk.patch [ 12467485 ]
          Hairong Kuang made changes -
          Summary A baby step towards inter-version communications between dfs client and NameNode A baby step towards inter-version RPC communications
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk1.patch [ 12467709 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk2.patch [ 12467748 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk4.patch [ 12467905 ]
          Hairong Kuang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 0.23.0 [ 12315569 ]
          Fix Version/s 0.22.0 [ 12314296 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk5.patch [ 12467934 ]
          Hairong Kuang made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk6.patch [ 12467971 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk7.patch [ 12468085 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk8.patch [ 12468146 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk9.patch [ 12468169 ]
          Hairong Kuang made changes -
          Link This issue blocks MAPREDUCE-2263 [ MAPREDUCE-2263 ]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk10.patch [ 12469591 ]
          Hairong Kuang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags [Incompatible change, Reviewed]
          Hairong Kuang made changes -
          Attachment rpcCompatible-trunk11.patch [ 12469706 ]
          Hairong Kuang made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hairong Kuang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags [Reviewed, Incompatible change] [Reviewed]
          Hairong Kuang made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed] [Incompatible change, Reviewed]
          Resolution Fixed [ 1 ]
          Sanjay Radia made changes -
          Link This issue is related to HADOOP-7347 [ HADOOP-7347 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Eugene Koontz made changes -
          Link This issue is related to GIRAPH-168 [ GIRAPH-168 ]
          Gavin made changes -
          Link This issue blocks HDFS-1335 [ HDFS-1335 ]
          Gavin made changes -
          Link This issue is depended upon by HDFS-1335 [ HDFS-1335 ]
          Gavin made changes -
          Link This issue blocks MAPREDUCE-2263 [ MAPREDUCE-2263 ]
          Gavin made changes -
          Link This issue is depended upon by MAPREDUCE-2263 [ MAPREDUCE-2263 ]

            People

            • Assignee:
              Hairong Kuang
              Reporter:
              Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development