Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4079

Allow MR AppMaster to limit ephemeral port range.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.23.2, 2.0.0-alpha
    • 0.23.3, 2.0.2-alpha
    • mr-am, mrv2
    • None

    Description

      Having the MapReduce Application Masters bind to any ephemeral port makes it very difficult to setup ACLs. mapreduce.job.am-access-disabled from MAPREDUCE-3251 is not a practical permanent solution for all jobs. Especially for tools like pig where they are not aware of mapreduce.job.am-access-disabled and may deal with it properly.

      We should add in a config option that would allow someone to restrict the range of ports that the MR-AM can bind to. It will slow down startup in some cases because we will have to probe for open ports instead of just asking the OS to find one for us. But we can make that conditional on this config so users who do not set this config do not see any performance degradation.

      Attachments

        1. MR-4079-full-branch-0.23.txt
          35 kB
          Robert Joseph Evans
        2. MR-4079-full-branch-0.23.txt
          35 kB
          Robert Joseph Evans
        3. MR-4079-full-branch-0.23.txt
          32 kB
          Robert Joseph Evans
        4. MR-4079-full-branch-0.23.txt
          31 kB
          Robert Joseph Evans
        5. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        6. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        7. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        8. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        9. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        10. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            revans2 Robert Joseph Evans
            revans2 Robert Joseph Evans
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment