Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3761

AM info in job -list does not reflect the actual AM hostname

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.24.0
    • Component/s: mrv2
    • Labels:
      None

      Description

      The AM info field on "bin/mapred job -list" currently has a value <resourcemanager hostname>:8088/proxy/appID. This info is irrelevant unless it shows the real information of where the AM was launched. This needs to be fixed to show the AM host details.

      1. MAPREDUCE-3761-20120214.1.txt
        19 kB
        Vinod Kumar Vavilapalli
      2. MAPREDUCE-3761-20120202.txt
        13 kB
        Vinod Kumar Vavilapalli

        Activity

        Hide
        Robert Joseph Evans added a comment -

        Also the whole point of the proxy is to mitigate the opportunities for an AM to steal cookies from an end user. It does not do a very good job of it, and I really would prefer to see a different architecture for how it does this, but this is what we have. If you want to put in the change as is I am not going to block it, but I suspect that the security people here will force me to change it before we can deploy into production, so I would prefer to just have a separate field that is not a URL.

        Show
        Robert Joseph Evans added a comment - Also the whole point of the proxy is to mitigate the opportunities for an AM to steal cookies from an end user. It does not do a very good job of it, and I really would prefer to see a different architecture for how it does this, but this is what we have. If you want to put in the change as is I am not going to block it, but I suspect that the security people here will force me to change it before we can deploy into production, so I would prefer to just have a separate field that is not a URL.
        Hide
        Robert Joseph Evans added a comment -

        The issue is that the AM is arbitrary code. It does not have to be something that we wrote. The only thing that we can do to restrict it is what the OS allows us to do. If there is a good way for us to make the OS not allow any incoming connections to the HTTP port of the AM then I am all for that. Sadly the HTTP port is an ephemeral port and so is the RPC port that needs to be open to allow connections from other locations.

        I would prefer a different column that can show the host that the AM is running on. It should come from the RM indicating where it was scheduler, because it is the AM that reports back both the web app URL and the RPC url.

        Show
        Robert Joseph Evans added a comment - The issue is that the AM is arbitrary code. It does not have to be something that we wrote. The only thing that we can do to restrict it is what the OS allows us to do. If there is a good way for us to make the OS not allow any incoming connections to the HTTP port of the AM then I am all for that. Sadly the HTTP port is an ephemeral port and so is the RPC port that needs to be open to allow connections from other locations. I would prefer a different column that can show the host that the AM is running on. It should come from the RM indicating where it was scheduler, because it is the AM that reports back both the web app URL and the RPC url.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        What we really want to show is the AM hostname, either through the scheduling-Info field or another.

        If an end user sees this output they are going to copy and past this into the browser and go to that link. If the AM is well behaved it will redirect the user back through the proxy, but if the AM is malicious, it will do exactly what the proxy is intended to help mitigate.

        This clearly seems like an issue with proxy or the AMs. We cannot rely on the fact that users will behave properly and will always click through the proxy for the sake of security. I agree that exposing this directly reduces security that exists today but this is as good as anyone doing an RPC call to figure out where an AM is running.

        The real issue (if there is one) is that AMs accept requests that do not originate from the proxy. We should fix that if at all, instead of trying to avoid users from not seeing information like this. What do you think?

        Show
        Vinod Kumar Vavilapalli added a comment - What we really want to show is the AM hostname, either through the scheduling-Info field or another. If an end user sees this output they are going to copy and past this into the browser and go to that link. If the AM is well behaved it will redirect the user back through the proxy, but if the AM is malicious, it will do exactly what the proxy is intended to help mitigate. This clearly seems like an issue with proxy or the AMs. We cannot rely on the fact that users will behave properly and will always click through the proxy for the sake of security. I agree that exposing this directly reduces security that exists today but this is as good as anyone doing an RPC call to figure out where an AM is running. The real issue (if there is one) is that AMs accept requests that do not originate from the proxy. We should fix that if at all, instead of trying to avoid users from not seeing information like this. What do you think?
        Hide
        Robert Joseph Evans added a comment -

        I am a bit confused by why we want to expose this to the end user. I can see that it is very useful to QE trying to test, or a developer trying to debug an issue, but this can potentially bypass the web application proxy, which is a security issue. If an end user sees this output they are going to copy and past this into the browser and go to that link. If the AM is well behaved it will redirect the user back through the proxy, but if the AM is malicious, it will do exactly what the proxy is intended to help mitigate. I don't think that this is critical, there are enough holes in the proxy when it is working properly anyways, but this patch is potentially reducing the security of the system.

        Show
        Robert Joseph Evans added a comment - I am a bit confused by why we want to expose this to the end user. I can see that it is very useful to QE trying to test, or a developer trying to debug an issue, but this can potentially bypass the web application proxy, which is a security issue. If an end user sees this output they are going to copy and past this into the browser and go to that link. If the AM is well behaved it will redirect the user back through the proxy, but if the AM is malicious, it will do exactly what the proxy is intended to help mitigate. I don't think that this is critical, there are enough holes in the proxy when it is working properly anyways, but this patch is potentially reducing the security of the system.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12514547/MAPREDUCE-3761-20120214.1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1856//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1856//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514547/MAPREDUCE-3761-20120214.1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1856//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1856//console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Fixing the test.

        Show
        Vinod Kumar Vavilapalli added a comment - Fixing the test.
        Hide
        Mahadev konar added a comment -

        Vinod,
        Can you take a look at the test?

        Show
        Mahadev konar added a comment - Vinod, Can you take a look at the test?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12513071/MAPREDUCE-3761-20120202.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.mapreduce.TestTypeConverter

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1777//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1777//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513071/MAPREDUCE-3761-20120202.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapreduce.TestTypeConverter +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1777//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1777//console This message is automatically generated.
        Hide
        Mahadev konar added a comment -

        Looks like the test failed because of:

        ContainerMemoryManager while managing memory of container_0_0000_01_000000
        java.lang.NumberFormatException: For input string: "18446743988089421650"
        	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
        	at java.lang.Long.parseLong(Long.java:422)
        	at java.lang.Long.parseLong(Long.java:468)
        	at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:427)
        	at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:168)
        

        which is MAPREDUCE-3583. Running hudson again.

        Show
        Mahadev konar added a comment - Looks like the test failed because of: ContainerMemoryManager while managing memory of container_0_0000_01_000000 java.lang.NumberFormatException: For input string: "18446743988089421650" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang. Long .parseLong( Long .java:422) at java.lang. Long .parseLong( Long .java:468) at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:427) at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:168) which is MAPREDUCE-3583 . Running hudson again.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12513071/MAPREDUCE-3761-20120202.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainersMonitor

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1758//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1758//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12513071/MAPREDUCE-3761-20120202.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.TestContainersMonitor +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1758//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1758//console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        This should fix it. Modified test to verify the same.

        Also added a BuilderUtil for ApplicationResourceUsageReport which I am then using it everywhere.

        Show
        Vinod Kumar Vavilapalli added a comment - This should fix it. Modified test to verify the same. Also added a BuilderUtil for ApplicationResourceUsageReport which I am then using it everywhere.

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Ramya Sunil
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development