XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.0
    • Component/s: capacityscheduler
    • Labels:
      None

      Description

      FSSchedulerConfigurationStore fails to update with hdfs path - "java.io.IOException: Filesystem closed"

      RM Logs:

      2019-10-06 16:50:40,829 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore: write temp capacity configuration fail, schedulerConfigFile=hdfs:/tmp/yarn/system/capacity-scheduler.xml.1570380640828.tmp
      java.io.IOException: Filesystem closed
              at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:475)
              at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1232)
              at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1214)
              at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1196)
              at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1134)
              at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:530)
              at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
              at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
              at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
              at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:468)
              at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1136)
              at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1116)
              at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1005)
              at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:993)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.writeTmpConfig(FSSchedulerConfigurationStore.java:251)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.FSSchedulerConfigurationStore.logMutation(FSSchedulerConfigurationStore.java:130)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.MutableCSConfigurationProvider.logAndApplyMutation(MutableCSConfigurationProvider.java:153)
              at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2597)
              at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices$13.run(RMWebServices.java:2587)
      

      Repro:

      yarn-site.xml:
      
      <property>
        <name>yarn.scheduler.configuration.fs.path</name> 
        <value>hdfs:///tmp/yarn/system</value>
      </property>
      
       <property>
        <name>yarn.scheduler.configuration.store.class&lt;/name>
        <value>fs</value>
      </property>
      
      [yarn@yarndocker-1 yarn]$ cat /tmp/abc.xml 
      <sched-conf>
              <update-queue>
                <queue-name>root.default</queue-name>
                <params>
                  <entry>
                    <key>priority</key>
                    <value>10</value>
                  </entry>
                </params>
              </update-queue>
            </sched-conf>
      
      [yarn@yarndocker-1 yarn]$ curl -v -X PUT -d @/tmp/abc.xml -H "Content-type: application/xml"  'http://yarndocker-1:8088/ws/v1/cluster/scheduler-conf?user.name=yarn'
      Filesystem closed
      

        Attachments

        1. YARN-9875-001.patch
          6 kB
          Prabhu Joseph
        2. YARN-9875-002.patch
          6 kB
          Prabhu Joseph

          Activity

            People

            • Assignee:
              prabhujoseph Prabhu Joseph
              Reporter:
              prabhujoseph Prabhu Joseph
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: