[YUNIKORN-1442] Data race in TestRejectApplications

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: test - unit
• Labels: pull-request-available
• Target Version: 1.2.0
• Assignee: Peter Bacsko
• Reporter: Peter Bacsko
• Created: 01/Dec/22 18:51
• Updated: 07/Feb/23 22:33
• Resolved: 02/Dec/22 17:17

Description

Found the following data race when running unit tests locally for shim:

==================
WARNING: DATA RACE
Read at 0x00c005593dd0 by goroutine 98:
  runtime.mapiterinit()
      /opt/hostedtoolcache/go/1.18.6/x64/src/runtime/map.go:815 +0x0
  github.com/apache/yunikorn-k8shim/pkg/cache.(*PlaceholderManager).cleanUp()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/cache/placeholder_manager.go:117 +0x246
  github.com/apache/yunikorn-k8shim/pkg/cache.(*Application).handleFailApplicationEvent.func1()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/cache/application.go:584 +0x35

Previous write at 0x00c005593dd0 by goroutine 75:
  runtime.mapdelete_faststr()
      /opt/hostedtoolcache/go/1.18.6/x64/src/runtime/map_faststr.go:301 +0x0
  github.com/apache/yunikorn-k8shim/pkg/cache.(*Application).removeTask()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/cache/application.go:250 +0x16c
  github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).RemoveTask()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/cache/context.go:742 +0x2a9
  github.com/apache/yunikorn-k8shim/pkg/shim.TestRejectApplications()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/shim/scheduler_test.go:138 +0x66f
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1439 +0x213
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1486 +0x47

Goroutine 98 (running) created at:
  github.com/apache/yunikorn-k8shim/pkg/cache.(*Application).handleFailApplicationEvent()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/cache/application.go:583 +0xca
  github.com/apache/yunikorn-k8shim/pkg/cache.newAppState.func7()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/cache/application_state.go:547 +0x2e6
  github.com/looplab/fsm.(*FSM).afterEventCallbacks()
      /home/runner/go/pkg/mod/github.com/looplab/fsm@v0.1.0/fsm.go:414 +0xc7
  github.com/looplab/fsm.(*FSM).Event.func1()
      /home/runner/go/pkg/mod/github.com/looplab/fsm@v0.1.0/fsm.go:309 +0xb7
  github.com/looplab/fsm.transitionerStruct.transition()
      /home/runner/go/pkg/mod/github.com/looplab/fsm@v0.1.0/fsm.go:354 +0x99
  github.com/looplab/fsm.(*transitionerStruct).transition()
      <autogenerated>:1 +0x29
  github.com/looplab/fsm.(*FSM).doTransition()
      /home/runner/go/pkg/mod/github.com/looplab/fsm@v0.1.0/fsm.go:339 +0x701
  github.com/looplab/fsm.(*FSM).Event()
      /home/runner/go/pkg/mod/github.com/looplab/fsm@v0.1.0/fsm.go:321 +0x6da
  github.com/apache/yunikorn-k8shim/pkg/cache.(*Application).handle()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/cache/application.go:104 +0x249
  github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).ApplicationEventHandler.func1()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/cache/context.go:910 +0xfb
  github.com/apache/yunikorn-k8shim/pkg/dispatcher.Start.func1()
      /home/runner/work/yunikorn-k8shim/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:197 +0x2eb

Goroutine 75 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1486 +0x724
  testing.runTests.func1()
      /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1839 +0x99
  testing.tRunner()
      /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1439 +0x213
  testing.runTests()
      /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1837 +0x7e4
  testing.(*M).Run()
      /opt/hostedtoolcache/go/1.18.6/x64/src/testing/testing.go:1719 +0xa71
  main.main()
      _testmain.go:103 +0x3a9
==================

We perform an unsafe access to app.taskMap in PlaceholderManager.cleanUp(). We need to call Application.GetPlaceholderTasks().

Issue Links

• links to: GitHub Pull Request #495