[SPARK-4715] ShuffleMemoryManager.tryToAcquire may return a negative value - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 1.1.0
Fix Version/s: 1.1.2, 1.2.0
Component/s: Spark Core
Labels:
None

Target Version/s:

1.1.2, 1.2.0

Description

Here is a unit test to demonstrate it:

  test("threads should not be granted a negative size") {
    val manager = new ShuffleMemoryManager(1000L)
    manager.tryToAcquire(700L)

    val latch = new CountDownLatch(1)
    startThread("t1") {
       manager.tryToAcquire(300L)
      latch.countDown()
    }
    latch.await() // Wait until `t1` calls `tryToAcquire`

    val granted = manager.tryToAcquire(300L)
    assert(0 === granted, "granted is negative")
  }

It outputs "0 did not equal -200 granted is negative"