Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.9.0, 2.8.3, 3.0.0
    • Component/s: None
    • Labels:
      None

      Description

      The kms client appears to have no retry logic – at all. It's completely decoupled from the ipc retry logic. This has major impacts if the KMS is unreachable for any reason, including but not limited to network connection issues, timeouts, the restart during an upgrade.

      This has some major ramifications:

      1. Jobs may fail to submit, although oozie resubmit logic should mask it
      2. Non-oozie launchers may experience higher rates if they do not already have retry logic.
      3. Tasks reading EZ files will fail, probably be masked by framework reattempts
      4. EZ file creation fails after creating a 0-length file – client receives EDEK in the create response, then fails when decrypting the EDEK
      5. Bulk hadoop fs copies, and maybe distcp, will prematurely fail

        Attachments

        1. HADOOP-14521.11.patch
          31 kB
          Xiao Chen
        2. HADOOP-14521-branch-2.8.002.patch
          31 kB
          Rushabh S Shah
        3. HADOOP-14521-branch-2.8.2.patch
          31 kB
          Rushabh S Shah
        4. HADOOP-14521-trunk-10.patch
          31 kB
          Rushabh S Shah
        5. HDFS-11804-branch-2.8.patch
          30 kB
          Rushabh S Shah
        6. HADOOP-14521.09.patch
          31 kB
          Xiao Chen
        7. HDFS-11804-trunk-8.patch
          30 kB
          Rushabh S Shah
        8. HDFS-11804-trunk-7.patch
          30 kB
          Rushabh S Shah
        9. HDFS-11804-trunk-6.patch
          30 kB
          Rushabh S Shah
        10. HDFS-11804-trunk-5.patch
          27 kB
          Rushabh S Shah
        11. HDFS-11804-trunk-4.patch
          24 kB
          Rushabh S Shah
        12. HDFS-11804-trunk-3.patch
          24 kB
          Rushabh S Shah
        13. HDFS-11804-trunk-2.patch
          24 kB
          Rushabh S Shah
        14. HDFS-11804-trunk-1.patch
          22 kB
          Rushabh S Shah
        15. HDFS-11804-trunk.patch
          22 kB
          Rushabh S Shah

          Activity

            People

            • Assignee:
              shahrs87 Rushabh S Shah
              Reporter:
              shahrs87 Rushabh S Shah
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: