Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4079

Allow MR AppMaster to limit ephemeral port range.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.2, 2.0.0-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mr-am, mrv2
    • Labels:
      None

      Description

      Having the MapReduce Application Masters bind to any ephemeral port makes it very difficult to setup ACLs. mapreduce.job.am-access-disabled from MAPREDUCE-3251 is not a practical permanent solution for all jobs. Especially for tools like pig where they are not aware of mapreduce.job.am-access-disabled and may deal with it properly.

      We should add in a config option that would allow someone to restrict the range of ports that the MR-AM can bind to. It will slow down startup in some cases because we will have to probe for open ports instead of just asking the OS to find one for us. But we can make that conditional on this config so users who do not set this config do not see any performance degradation.

        Attachments

        1. MR-4079-full-branch-0.23.txt
          35 kB
          Robert Joseph Evans
        2. MR-4079-full-branch-0.23.txt
          35 kB
          Robert Joseph Evans
        3. MR-4079-full-branch-0.23.txt
          32 kB
          Robert Joseph Evans
        4. MR-4079-full-branch-0.23.txt
          31 kB
          Robert Joseph Evans
        5. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        6. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        7. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        8. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        9. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        10. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans

          Issue Links

            Activity

              People

              • Assignee:
                revans2 Robert Joseph Evans
                Reporter:
                revans2 Robert Joseph Evans
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: