Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15565

ViewFileSystem.close doesn't close child filesystems and causes FileSystem objects leak.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0, 3.1.4, 3.2.2, 2.10.1
    • None
    • None

    Description

      ViewFileSystem.close() does nothing but remove itself from FileSystem.CACHE. It's children filesystems are cached in FileSystem.CACHE and shared by all the ViewFileSystem instances. We could't simply close all the children filesystems because it will break the semantic of FileSystem.newInstance().
      We might add an inner cache to ViewFileSystem, let it cache all the children filesystems. The children filesystems are not shared any more. When ViewFileSystem is closed we close all the children filesystems in the inner cache. The ViewFileSystem is still cached by FileSystem.CACHE so there won't be too many FileSystem instances.

      The FileSystem.CACHE caches the ViewFileSysem instance and the other instances(the children filesystems) are cached in the inner cache.

      Attachments

        1. HADOOP-15565-branch-3.2.002.patch
          24 kB
          Virajith Jalaparti
        2. HADOOP-15565-branch-3.2.001.patch
          25 kB
          Virajith Jalaparti
        3. HADOOP-15565-branch-3.1.001.patch
          24 kB
          Virajith Jalaparti
        4. HADOOP-15565-branch-2.10.001.patch
          24 kB
          Virajith Jalaparti
        5. HADOOP-15565.0008.patch
          24 kB
          Jinglun
        6. HADOOP-15565.0007.patch
          24 kB
          Jinglun
        7. HADOOP-15565.0006.patch
          23 kB
          Jinglun
        8. HADOOP-15565.0006.bak
          23 kB
          Jinglun
        9. HADOOP-15565.0005.patch
          21 kB
          Jinglun
        10. HADOOP-15565.0004.patch
          21 kB
          Jinglun
        11. HADOOP-15565.0003.patch
          11 kB
          Jinglun
        12. HADOOP-15565.0002.patch
          7 kB
          Jinglun
        13. HADOOP-15565.0001.patch
          7 kB
          Jinglun

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            LiJinglun Jinglun
            LiJinglun Jinglun
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment