Hadoop Common
  1. Hadoop Common
  2. HADOOP-9041

FileSystem initialization can go into infinite loop

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: fs
    • Labels:
      None

      Description

      More information is there: https://jira.springsource.org/browse/SHDP-111

      Referenced source code from example is: https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java

      from isolating that cause it looks like if you register: org.apache.hadoop.fs.FsUrlStreamHandlerFactory before calling FileSystem.loadFileSystems() then it goes into infinite loop.

      1. fsinit6.txt
        3 kB
        Radim Kolar
      2. fsinit5.txt
        3 kB
        Radim Kolar
      3. fsinit4.txt
        3 kB
        Radim Kolar
      4. fsinit3.txt
        3 kB
        Radim Kolar
      5. TestFileSystem.java
        0.7 kB
        Yanbo Liang
      6. fsinit2.txt
        3 kB
        Radim Kolar
      7. HADOOP-9041.patch
        3 kB
        Yanbo Liang
      8. fsinit-unit.txt
        2 kB
        Radim Kolar
      9. HADOOP-9041.patch
        0.9 kB
        Yanbo Liang
      10. fstest.groovy
        0.6 kB
        Radim Kolar

        Activity

        Allen Wittenauer made changes -
        Fix Version/s 3.0.0 [ 12320357 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Luke Lu made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 3.0.0 [ 12320357 ]
        Fix Version/s 2.0.3-alpha [ 12323273 ]
        Resolution Fixed [ 1 ]
        Radim Kolar made changes -
        Attachment fsinit6.txt [ 12561146 ]
        Radim Kolar made changes -
        Attachment fsinit5.txt [ 12561140 ]
        Radim Kolar made changes -
        Attachment fsinit4.txt [ 12561133 ]
        Radim Kolar made changes -
        Attachment fsinit3.txt [ 12561053 ]
        Yanbo Liang made changes -
        Attachment TestFileSystem.java [ 12560342 ]
        Radim Kolar made changes -
        Attachment fsinit2.txt [ 12559802 ]
        Radim Kolar made changes -
        Assignee Yanbo Liang [ yanboliang ] Radim Kolar [ hsn ]
        Yanbo Liang made changes -
        Attachment HADOOP-9041.patch [ 12553639 ]
        Yanbo Liang made changes -
        Attachment HADOOP-9041.patch [ 12559801 ]
        Radim Kolar made changes -
        Attachment fsinit-unit.txt [ 12554193 ]
        Yanbo Liang made changes -
        Attachment HADOOP-9041.patch [ 12553761 ]
        Yanbo Liang made changes -
        Attachment HADOOP-9041.patch [ 12553900 ]
        Alejandro Abdelnur made changes -
        Assignee Yanbo Liang [ yanboliang ]
        Yanbo Liang made changes -
        Attachment HADOOP-9041.patch [ 12553761 ]
        Yanbo Liang made changes -
        Release Note The infinite loop caused by FileSystem.loadFileSystems(). If the users register org.apache.hadoop.fs.FsUrlStreamHandlerFactory as the current URLStreamHandlerFactory, then calling FsUrlStreamHandlerFactory.createURLStreamHandler() will lead to invoke FileSystem.loadFileSystems(). FileSystem.loadFileSystems() will cause to invoke FsUrlStreamHandlerFactory.createURLStreamHandler() when resolving a URL. Then the infinite loop occurs.

        Just moving the function of FileSystem.loadFileSystems() into a static code block. It will initialize as the construction of Class and only once.
        Yanbo Liang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note The infinite loop caused by FileSystem.loadFileSystems(). If the users register org.apache.hadoop.fs.FsUrlStreamHandlerFactory as the current URLStreamHandlerFactory, then calling FsUrlStreamHandlerFactory.createURLStreamHandler() will lead to invoke FileSystem.loadFileSystems(). FileSystem.loadFileSystems() will cause to invoke FsUrlStreamHandlerFactory.createURLStreamHandler() when resolving a URL. Then the infinite loop occurs.

        Just moving the function of FileSystem.loadFileSystems() into a static code block. It will initialize as the construction of Class and only once.
        Yanbo Liang made changes -
        Attachment HADOOP-9041.patch [ 12553639 ]
        Radim Kolar made changes -
        Attachment fstest.groovy [ 12553591 ]
        Suresh Srinivas made changes -
        Field Original Value New Value
        Summary FileSystem inicialization can go into infinite looop FileSystem initialization can go into infinite loop
        Radim Kolar created issue -

          People

          • Assignee:
            Radim Kolar
            Reporter:
            Radim Kolar
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development