Uploaded image for project: 'Aurora'
  1. Aurora
  2. AURORA-1320

when instance is running in docker container, thermos observer reports 0 resources

Add voteWatch issue
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Docker, Thermos
    • None

    Description

      To see the problem start a job inside a docker container and view the task/instance page. You'll cpu/ram/disk all at zero regardless of their actual usage.

      I see errors like this in the thermos observer log:

          W0513 18:41:39.415406 3564 process_collector_psutil.py:42] Error during process sampling [pid=112]: process no longer exists (pid=112)
          W0513 18:41:39.415612 3564 process_collector_psutil.py:76] Error during process sampling: process no longer exists (pid=112)
          W0513 18:41:39.513972 3564 process_collector_psutil.py:76] Error during process sampling: no process found with pid 122
      

      This is likely because observer is running in a different pid namespace than the process. One solution would be for the runner to write out the pid namespace it is running in to the checkpoint and then have observer enter that namespace while sampling.

      Or we can just get rid of the observer?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jaybuff Jay Buffington

              Dates

                Created:
                Updated:

                Slack

                  Issue deployment