Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3566

OOM because of too many DFSClient instances in TokenCache.obtainTokensForFileSystems

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0, 0.8.5
    • None
    • None

    Description

      When FileSystem cache is disabled, invoking TokenCache.obtainTokensForFileSystems creates lot of FileSystem objects which are unique and stored in a hash set. This takes up lot of memory and causes OOM.

      Attachments

        1. TEZ-3566.01.patch
          4 kB
          Harish JP
        2. TEZ-3566.02.patch
          5 kB
          Harish JP
        3. TEZ-3566.03.patch
          5 kB
          Harish JP

        Activity

          People

            harishjp Harish JP
            harishjp Harish JP
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: