Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6113

An ability to override LocalResourceVisibility for public resources

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: client, distributed-cache
    • Labels:
      None

      Description

      Currently it is impossible to specify APPLICATION level for files in distributed cache in YARN. For applications with big common data it is very crucial to have ability to share data only for application and do not spam public cache. Moreover, public localiser is limited for parallel downloads, so applications with big caches affect application startup of other applications.
      Current logic is complicated and based on file permissions. Making proposed changes, user can override public level to any level (PUBLIC/APP/PRIVATE) without breaking anything, and ever not need to bother with permission, if private cache is needed.

        Attachments

        1. MAPREDUCE-6113.patch
          6 kB
          Andrey Stepachev

          Issue Links

            Activity

              People

              • Assignee:
                octo47 Andrey Stepachev
                Reporter:
                octo47 Andrey Stepachev
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: