[YUNIKORN-2874] Cannot generate reproducible builds during release

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.0
• Component/s: release
• Labels: pull-request-available
• Target Version: 1.7.0
• Assignee: Craig Condit
• Reporter: Peter Bacsko
• Created: 11/Sep/24 16:20
• Updated: 17/Sep/24 17:07
• Resolved: 17/Sep/24 17:07

Description

When trying to release Yunikorn with REPRODUCIBLE_BUILDS=1 (default), then the following error occurs:

~/repos/yunikorn-release/staging/tmp/apache-yunikorn-1.6.0-src/k8shim$ make REPRODUCIBLE_BUILDS=1 scheduler
building binary for scheduler docker image
docker run -t --rm=true --volume "/home/bacskop/repos/yunikorn-release/staging/tmp/apache-yunikorn-1.6.0-src/k8shim/:/buildroot" "golang:1.22.1" sh -c "cd /buildroot && \
CGO_ENABLED=0 GOOS=linux GOARCH=\"amd64\" go build \
-a \
-o=build/bin/yunikorn-scheduler \
-trimpath \
-ldflags '-buildid= -extldflags \"-static\" -X github.com/apache/yunikorn-k8shim/pkg/conf.buildVersion=1.6.0 -X github.com/apache/yunikorn-k8shim/pkg/conf.buildDate=2024-09-10T08:26:23+00:00 -X github.com/apache/yunikorn-k8shim/pkg/conf.isPluginVersion=false -X github.com/apache/yunikorn-k8shim/pkg/conf.goVersion=1.22.1 -X github.com/apache/yunikorn-k8shim/pkg/conf.arch=amd64 -X github.com/apache/yunikorn-k8shim/pkg/conf.coreSHA=a2d40c81fee104356f9e33120fd557a928f74f2b -X github.com/apache/yunikorn-k8shim/pkg/conf.siSHA=68e8c6cca28a743d797e7908b1225392dddda3a2 -X github.com/apache/yunikorn-k8shim/pkg/conf.shimSHA=240aeb90951a30c677890b61b50f6bfcafb227b5' \
-tags netgo \
-installsuffix netgo \
./pkg/cmd/shim/"
go: downloading go.uber.org/zap v1.26.0
go: downloading k8s.io/api v0.31.0
...
go: downloading github.com/spf13/cobra v1.8.1
go: downloading golang.org/x/sync v0.8.0
go: downloading github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a
pkg/cmd/shim/main.go:31:2: github.com/apache/yunikorn-core@v1.6.0-1: replacement directory ../core/ does not exist
pkg/common/constants/constants.go:22:2: github.com/apache/yunikorn-scheduler-interface@v1.6.0-1: replacement directory ../scheduler-interface/ does not exist
pkg/locking/locking.go:26:2: github.com/apache/yunikorn-core@v1.6.0-1: replacement directory ../core/ does not exist
pkg/common/test/schedulerapi_mock.go:25:2: github.com/apache/yunikorn-scheduler-interface@v1.6.0-1: replacement directory ../scheduler-interface/ does not exist
pkg/common/test/recoverable_apps_mock.go:25:2: github.com/apache/yunikorn-scheduler-interface@v1.6.0-1: replacement directory ../scheduler-interface/ does not exist

The problem is, we don't mount "../core" and "../scheduler-interface" during the release procedure.  This is slightly different from a normal build, because we use the "replace" directive in "go.mod", so the two extra directories are necessary.

Issue Links

• is caused by: YUNIKORN-2487 [Release] Force REPRODUCIBLE_BUILDS=1 on release
• links to: GitHub Pull Request #914