Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17433

YarnShuffleService doesn't handle moving credentials levelDb

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.0
    • Component/s: YARN
    • Labels:
      None

      Description

      In SPARK-16711, I added a leveldb to store credentials to fix an issue with NM restart. I missed that getRecoveryPath also handles moving the DB from the old local dirs to the yarn recoverypath. This routine is hardcoded for the registeredexecutors db and needs to be updated to handle the new credentials db I added.

        Attachments

          Activity

            People

            • Assignee:
              tgraves Thomas Graves
              Reporter:
              tgraves Thomas Graves
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: