Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4079

Allow MR AppMaster to limit ephemeral port range.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.2, 2.0.0-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mr-am, mrv2
    • Labels:
      None

      Description

      Having the MapReduce Application Masters bind to any ephemeral port makes it very difficult to setup ACLs. mapreduce.job.am-access-disabled from MAPREDUCE-3251 is not a practical permanent solution for all jobs. Especially for tools like pig where they are not aware of mapreduce.job.am-access-disabled and may deal with it properly.

      We should add in a config option that would allow someone to restrict the range of ports that the MR-AM can bind to. It will slow down startup in some cases because we will have to probe for open ports instead of just asking the OS to find one for us. But we can make that conditional on this config so users who do not set this config do not see any performance degradation.

      1. MR-4079-trunk.txt
        9 kB
        Robert Joseph Evans
      2. MR-4079-full-branch-0.23.txt
        35 kB
        Robert Joseph Evans
      3. MR-4079-trunk.txt
        9 kB
        Robert Joseph Evans
      4. MR-4079-trunk.txt
        9 kB
        Robert Joseph Evans
      5. MR-4079-trunk.txt
        9 kB
        Robert Joseph Evans
      6. MR-4079-full-branch-0.23.txt
        35 kB
        Robert Joseph Evans
      7. MR-4079-trunk.txt
        9 kB
        Robert Joseph Evans
      8. MR-4079-full-branch-0.23.txt
        32 kB
        Robert Joseph Evans
      9. MR-4079-trunk.txt
        9 kB
        Robert Joseph Evans
      10. MR-4079-full-branch-0.23.txt
        31 kB
        Robert Joseph Evans

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Robert Joseph Evans
              Reporter:
              Robert Joseph Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development