Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20243

DebugFilesystem.assertNoOpenStreams thread race

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.1.1, 2.2.0
    • Tests
    • None

    Description

      Introduced by SPARK-19946.

      DebugFilesystem.assertNoOpenStreams gets the size of the openStreams ConcurrentHashMap and then later, if the size was > 0, accesses the first element in openStreams.values. But, the ConcurrentHashMap might be cleared by another thread between getting its size and accessing it, resulting in an exception when trying to call .head on an empty collection.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bograd Bogdan Raducanu
            bograd Bogdan Raducanu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment