[YUNIKORN-2287] Decompress function doesn't need to decode base64

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: shim - kubernetes
• Labels: e2e pull-request-available release-notes
• Target Version: 1.5.0
• Assignee: PoAn Yang
• Reporter: PoAn Yang
• Created: 22/Dec/23 15:35
• Updated: 20/Mar/24 14:32
• Resolved: 14/Jan/24 18:06

Description

I'm working on https://issues.apache.org/jira/browse/YUNIKORN-2267. I added an example as following, but compression configmap doesn't work.

 

1. Use gzip to compress and use base64 to encode the config

```

echo "
partitions:
  - name: default
    placementrules:
      - name: tag
        value: namespace
        create: true
    queues:
      - name: root
        submitacl: '*'
        queues:
          - name: parent
            submitacl: '*'" | gzip | base64

```

2. Set the result to `queues.yaml.gz` in binaryData field.

 

Finally, we can see an error log in scheduler:

```

2023-12-22T15:32:15.913Z    ERROR    shim.config    conf/schedulerconf.go:458    failed to decode schedulerConfig entry    {"error": "illegal base64 data at input byte 0"}
github.com/apache/yunikorn-k8shim/pkg/conf.Decompress
    github.com/apache/yunikorn-k8shim/pkg/conf/schedulerconf.go:458
github.com/apache/yunikorn-k8shim/pkg/conf.FlattenConfigMaps
    github.com/apache/yunikorn-k8shim/pkg/conf/schedulerconf.go:495
github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).triggerReloadConfig
    github.com/apache/yunikorn-k8shim/pkg/cache/context.go:496
github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updateConfigMaps
    github.com/apache/yunikorn-k8shim/pkg/cache/context.go:394
k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate
    k8s.io/client-go@v0.27.7/tools/cache/controller.go:250
k8s.io/client-go/tools/cache.FilteringResourceEventHandler.OnUpdate
    k8s.io/client-go@v0.27.7/tools/cache/controller.go:315
k8s.io/client-go/tools/cache.(*processorListener).run.func1
    k8s.io/client-go@v0.27.7/tools/cache/shared_informer.go:971
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
    k8s.io/apimachinery@v0.27.7/pkg/util/wait/backoff.go:226
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
    k8s.io/apimachinery@v0.27.7/pkg/util/wait/backoff.go:227
k8s.io/apimachinery/pkg/util/wait.JitterUntil
    k8s.io/apimachinery@v0.27.7/pkg/util/wait/backoff.go:204
k8s.io/apimachinery/pkg/util/wait.Until
    k8s.io/apimachinery@v0.27.7/pkg/util/wait/backoff.go:161
k8s.io/client-go/tools/cache.(*processorListener).run
    k8s.io/client-go@v0.27.7/tools/cache/shared_informer.go:967
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1
    k8s.io/apimachinery@v0.27.7/pkg/util/wait/wait.go:72

```

 

I think the root cause is that the golang object is already decoded, so we don't need to decode it again.

Issue Links

• is part of: YUNIKORN-2267 add docs of using BinaryData to configure scheduler
• is related to: YUNIKORN-1875 Need compression method for queue config larger than 1MB size
• links to: GitHub Pull Request #752