Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4079

Allow MR AppMaster to limit ephemeral port range.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.23.2, 2.0.0-alpha
    • 0.23.3, 2.0.2-alpha
    • mr-am, mrv2
    • None

    Description

      Having the MapReduce Application Masters bind to any ephemeral port makes it very difficult to setup ACLs. mapreduce.job.am-access-disabled from MAPREDUCE-3251 is not a practical permanent solution for all jobs. Especially for tools like pig where they are not aware of mapreduce.job.am-access-disabled and may deal with it properly.

      We should add in a config option that would allow someone to restrict the range of ports that the MR-AM can bind to. It will slow down startup in some cases because we will have to probe for open ports instead of just asking the OS to find one for us. But we can make that conditional on this config so users who do not set this config do not see any performance degradation.

      Attachments

        1. MR-4079-full-branch-0.23.txt
          31 kB
          Robert Joseph Evans
        2. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        3. MR-4079-full-branch-0.23.txt
          32 kB
          Robert Joseph Evans
        4. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        5. MR-4079-full-branch-0.23.txt
          35 kB
          Robert Joseph Evans
        6. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        7. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        8. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans
        9. MR-4079-full-branch-0.23.txt
          35 kB
          Robert Joseph Evans
        10. MR-4079-trunk.txt
          9 kB
          Robert Joseph Evans

        Issue Links

          Activity

            People

              revans2 Robert Joseph Evans
              revans2 Robert Joseph Evans
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: