Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9728

 ResourceManager REST API can produce an illegal xml response

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.3
    • Fix Version/s: 3.3.0
    • Component/s: api, resourcemanager
    • Labels:
      None
    • Target Version/s:

      Description

      When a spark job throws an exception with a message containing a character out of the range supported by xml 1.0, then
      the application fails and the stack trace will be stored into the diagnostics field. So far, so good.

      But the issue occurred when we try to get application information with the ResourceManager REST API
      The xml response will contain the illegal xml 1.0 char and will be invalid.

       Examples of illegals characters in xml 1.0 : 

      • \u0000
      • \u0001
      • \u0002
      • \u0003
      • \u0004

      For more information about supported characters :
      https://www.w3.org/TR/xml/#charsets

      Example of illegal response from the Ressource Manager API : 

      <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
      <app>
        <id>application_1326821518301_0005</id>
        <user>user1</user>
        <name>job</name>
        <queue>a1</queue>
        <state>FINISHED</state>
        <finalStatus>FAILED</finalStatus>
        <progress>100.0</progress>
        <trackingUI>History</trackingUI>
        <trackingUrl>http://host.domain.com:8088/proxy/application_1326821518301_0005/jobhistory/job/job_1326821518301_5_5</trackingUrl>
        <diagnostics>Exception in thread "main" java.lang.Exception: \u0001
      	at com.XXXXXXXX.main(JobWithSpecialCharMain.java:6)</diagnostics>
      
        [...]
      
      </app>
      

       

      Example of job to reproduce :

      public class JobWithSpecialCharMain {
      
       public static void main(String[] args) throws Exception {
        throw new Exception("\u0001");
       }
      
      }
      
      javac -d . JobWithSpecialCharMain.java
      jar cvf repro.jar com/
      spark-submit --class com.JobWithSpecialCharMain --master yarn-cluster repro.jar
      

        Attachments

        1. YARN-9728-007.patch
          14 kB
          Prabhu Joseph
        2. YARN-9728-006.patch
          14 kB
          Prabhu Joseph
        3. YARN-9728-005.patch
          14 kB
          Prabhu Joseph
        4. YARN-9728-004.patch
          14 kB
          Prabhu Joseph
        5. YARN-9728-003.patch
          14 kB
          Prabhu Joseph
        6. YARN-9728-002.patch
          14 kB
          Prabhu Joseph
        7. YARN-9728-001.patch
          8 kB
          Prabhu Joseph
        8. IllegalResponseChrome.png
          31 kB
          Thomas

          Activity

            People

            • Assignee:
              prabhujoseph Prabhu Joseph
              Reporter:
              tde Thomas
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: