Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-4998

ResourceManager fails when num task slots > Yarn vcores

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 1.1.3
    • Fix Version/s: 1.2.0, 1.1.4
    • Component/s: ResourceManager, YARN
    • Labels:
      None

      Description

      The ResourceManager fails to acquire containers when the users configures the number of task slots to be greater than the maximum number of virtual cores of the Yarn cluster.

      We should check during deployment that the task slots are not configured to be larger than the virtual cores.

      2016-11-02 14:39:01,948 ERROR org.apache.flink.yarn.YarnFlinkResourceManager                - FATAL ERROR IN YARN APPLICATION MASTER: Connection to YARN Resource Manager failed
      org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid resource request, requested virtual cores < 0, or requested virtual cores > max configured, requestedVirtualCores=3, maxVirtualCores=1
      

        Issue Links

          Activity

          Hide
          mxm Maximilian Michels added a comment -

          This is related to FLINK-2213.

          Show
          mxm Maximilian Michels added a comment - This is related to FLINK-2213 .
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user mxm opened a pull request:

          https://github.com/apache/flink/pull/2741

          FLINK-4998[yarn] fail if too many task slots are configured

          This fails the deployment of the Yarn application if the number of task
          slots are configured to be larger than the maximum virtual cores of the
          Yarn cluster.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/mxm/flink FLINK-4998

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/2741.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2741


          commit 35c4ad3cb086abe6fa85c5755daa8a83fbdfbf56
          Author: Maximilian Michels <mxm@apache.org>
          Date: 2016-11-02T15:37:56Z

          FLINK-4998[yarn] fail if too many task slots are configured

          This fails the deployment of the Yarn application if the number of task
          slots are configured to be larger than the maximum virtual cores of the
          Yarn cluster.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user mxm opened a pull request: https://github.com/apache/flink/pull/2741 FLINK-4998 [yarn] fail if too many task slots are configured This fails the deployment of the Yarn application if the number of task slots are configured to be larger than the maximum virtual cores of the Yarn cluster. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mxm/flink FLINK-4998 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/2741.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2741 commit 35c4ad3cb086abe6fa85c5755daa8a83fbdfbf56 Author: Maximilian Michels <mxm@apache.org> Date: 2016-11-02T15:37:56Z FLINK-4998 [yarn] fail if too many task slots are configured This fails the deployment of the Yarn application if the number of task slots are configured to be larger than the maximum virtual cores of the Yarn cluster.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mxm commented on the issue:

          https://github.com/apache/flink/pull/2741

          Added a test case to verify the error reporting.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mxm commented on the issue: https://github.com/apache/flink/pull/2741 Added a test case to verify the error reporting.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/2741

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/2741
          Hide
          mxm Maximilian Michels added a comment -

          master: e4807621b8f41fc4f9fa69f423f1fbf7bba05218
          release-1.1: fe2c4ba6a8dbbe5bde75c1dd816ae5d2004910e0

          Show
          mxm Maximilian Michels added a comment - master: e4807621b8f41fc4f9fa69f423f1fbf7bba05218 release-1.1: fe2c4ba6a8dbbe5bde75c1dd816ae5d2004910e0

            People

            • Assignee:
              mxm Maximilian Michels
              Reporter:
              mxm Maximilian Michels
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development