Uploaded image for project: 'Apache YuniKorn'
  1. Apache YuniKorn
  2. YUNIKORN-1943

[Yunikorn] Logging Enhancement for Quota Enforcement checks

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • shim - kubernetes
    • None

    Description

      Hi Team,

      Currently, there is indefinite logging when we have quota enforcement checks for max application. When the App or User is not satisfying the max application setting which was set in Yunikorn-Config we are observing indefinite logging of the below warning.

      2023-08-31T07:32:25.922Z    WARN    core.scheduler.ugm    ugm/queue_tracker.go:428    can't run app as allowing new application to run would exceed configured max applications limit of specific user/group    {"tracking type": 2, "queue path": "test3", "current running applications": 1, "max running applications": 1}
      2023-08-31T07:32:25.922Z    WARN    core.scheduler.ugm    ugm/queue_tracker.go:428    can't run app as allowing new application to run would exceed configured max applications limit of specific user/group    {"tracking type": 2, "queue path": "test3", "current running applications": 1, "max running applications": 1}
      2023-08-31T07:32:26.023Z    WARN    core.scheduler.ugm    ugm/queue_tracker.go:428    can't run app as allowing new application to run would exceed configured max applications limit of specific user/group    {"tracking type": 2, "queue path": "test3", "current running applications": 1, "max running applications": 1}
      2023-08-31T07:32:26.023Z    WARN    core.scheduler.ugm    ugm/queue_tracker.go:428    can't run app as allowing new application to run would exceed configured max applications limit of specific user/group    {"tracking type": 2, "queue path": "test3", "current running applications": 1, "max running applications": 1}

      The warning will keep coming in the logs until the pod/Application is either scheduled or removed.

      {key:\"memory\"  value:{value:50000000}}  resources:{key:\"pods\"  value:{value:1}}  resources:{key:\"vcore\"  value:{value:50}}}"}
      2023-08-29T08:35:15.935Z    WARN    shim.context    cache/context.go:952    task event is not published because task is not found    {"appID": "830c91f6-d89b-4c2a-8e9a-5c9ece9fd83a", "taskID": "testkk", "event": "type:REQUEST  objectID:\"830c91f6-d89b-4c2a-8e9a-5c9ece9fd83a\"  message:\"Application testkk does not fit into root.default queue\"  timestampNano:1693298115801596877  referenceID:\"testkk\"  resource:{resources:{key:\"memory\"  value:{value:50000000}}  resources:{key:\"pods\"  value:{value:1}}  resources:{key:\"vcore\"  value:{value:50}}}"}
      2023-08-29T08:35:15.935Z    WARN    shim.context    cache/context.go:952    task event is not published because task is not found    {"appID": "830c91f6-d89b-4c2a-8e9a-5c9ece9fd83a", "taskID": "testkk", "event": "type:REQUEST  objectID:\"830c91f6-d89b-4c2a-8e9a-5c9ece9fd83a\"  message:\"Application testkk does not fit into root.default queue\"  timestampNano:1693298115902320752  referenceID:\"testkk\"  resource:{resources:{key:\"memory\"  value:{value:50000000}}  resources:{key:\"pods\"  value:{value:1}}  resources:{key:\"vcore\"  value:{value:50}}}"} 

      Also if you check the logs it's not very clearly formatted, Need some better formatting as well.

      Please let me know if something is required.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mani Manikandan R
            rrajesh Rajesh Kanhaiya Lal

            Dates

              Created:
              Updated:

              Slack

                Issue deployment