Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-25081

Up the container nproc uplimit to 30000

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1
    • test
    • None
    • Reviewed
    • Ups the nproc (processes) limit from 12500 to 30000 in yetus (so build container can have new limit).

    Description

      We (Apache Phoenix team) have recently switched our precommit tests to Dockerized Yetus (mostly adopted from the solution in Hbase)

      We see 
      java.lang.OutOfMemoryError: unable to create new native thread
      errors , while Yetus shows

      Max. process+thread count 6833 (vs. ulimit of 12500)

      While I couldn't determine what was the job that we shared the Agent with at the time, statistically it was very likely HBase, and an HBase job probably failed with a similar error.

      Some research has thrown up the official Docker docs: 

      https://docs.docker.com/engine/reference/commandline/run/#set-ulimits-in-container-ulimit

      According to which it is not possible to set container level nprocs ulimit with Docker.

      All settings apply to the docker Daemon user instead, and the limit is shared between all containers.

      Based on this, I think that it makes no sense to set a container (really docker user) nprocs ulimit any lower than the current hard limit of 30000.

      I have already set PROC_LIMIT=30000 in the Phoenix Yetus personality, but it is only a half solution until some Docker users set lower values, as the later setting will apply as soon as the container is started.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stoty Istvan Toth
            stoty Istvan Toth
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment