Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5058

taskManagerMemory attribute set wrong value in FlinkShell

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.3
    • Fix Version/s: 1.2.0, 1.1.4
    • Component/s: Scala Shell
    • Labels:
      None

      Description

      In FlinkShell.scala, someone mistakenly set slots value to taskManagerMemory attribute as follow:

      // set configuration from user input
      yarnConfig.jobManagerMemory.foreach((jmMem) => args ++= Seq("-yjm", jmMem.toString))
      yarnConfig.slots.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString))
      yarnConfig.name.foreach((name) => args ++= Seq("-ynm", name.toString))
      yarnConfig.queue.foreach((queue) => args ++= Seq("-yqu", queue.toString))
      yarnConfig.slots.foreach((slots) => args ++= Seq("-ys", slots.toString))
      

      Note on the third line: yarnConfig.slots.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString)) , we set slots value to -ytm attribute, the right code should be:

      // set configuration from user input
      yarnConfig.jobManagerMemory.foreach((jmMem) => args ++= Seq("-yjm", jmMem.toString))
      yarnConfig.taskManagerMemory.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString))
      yarnConfig.name.foreach((name) => args ++= Seq("-ynm", name.toString))
      yarnConfig.queue.foreach((queue) => args ++= Seq("-yqu", queue.toString))
      yarnConfig.slots.foreach((slots) => args ++= Seq("-ys", slots.toString))
      

        Issue Links

          Activity

          Hide
          fhueske Fabian Hueske added a comment -

          Thanks for the fix!

          Fixed for 1.1.4 with 68585d145178beb103ae95f0a80580e8786ee176
          Fixed for 1.2.0 with a67d3f80c9782cfc8f70586b086a86ba897e6dbc

          Show
          fhueske Fabian Hueske added a comment - Thanks for the fix! Fixed for 1.1.4 with 68585d145178beb103ae95f0a80580e8786ee176 Fixed for 1.2.0 with a67d3f80c9782cfc8f70586b086a86ba897e6dbc
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/2799

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/2799
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on the issue:

          https://github.com/apache/flink/pull/2799

          Merging

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on the issue: https://github.com/apache/flink/pull/2799 Merging
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user fhueske commented on the issue:

          https://github.com/apache/flink/pull/2799

          Thanks for the fix @397090770!
          +1 to merge

          Show
          githubbot ASF GitHub Bot added a comment - Github user fhueske commented on the issue: https://github.com/apache/flink/pull/2799 Thanks for the fix @397090770! +1 to merge
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user 397090770 opened a pull request:

          https://github.com/apache/flink/pull/2799

          FLINK-5058taskManagerMemory attribute set wrong value

          In FlinkShell.scala, someone mistakenly set `slots` value to `taskManagerMemory` attribute as follow:
          ```
          // set configuration from user input
          yarnConfig.jobManagerMemory.foreach((jmMem) => args ++= Seq("-yjm", jmMem.toString))
          yarnConfig.slots.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString))
          yarnConfig.name.foreach((name) => args ++= Seq("-ynm", name.toString))
          yarnConfig.queue.foreach((queue) => args ++= Seq("-yqu", queue.toString))
          yarnConfig.slots.foreach((slots) => args ++= Seq("-ys", slots.toString))
          ```
          Note on the third line: `yarnConfig.slots.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString))` , we set `slots` value to `taskManagerMemory` attribute, But in fact, we want to set `taskManagerMemory` attribute, the right code is as follow:
          ```
          // set configuration from user input
          yarnConfig.jobManagerMemory.foreach((jmMem) => args ++= Seq("-yjm", jmMem.toString))
          yarnConfig.taskManagerMemory.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString))
          yarnConfig.name.foreach((name) => args ++= Seq("-ynm", name.toString))
          yarnConfig.queue.foreach((queue) => args ++= Seq("-yqu", queue.toString))
          yarnConfig.slots.foreach((slots) => args ++= Seq("-ys", slots.toString))
          ```
          please review tihs pull, thanks.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/397090770/flink master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/2799.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2799


          commit f71f1d3f04f61500834c4d0d99d9b09df812781b
          Author: iteblog <wyphao.2007@163.com>
          Date: 2016-11-14T11:04:28Z

          taskManagerMemory attribute set wrong value

          taskManagerMemory attribute set wrong value


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user 397090770 opened a pull request: https://github.com/apache/flink/pull/2799 FLINK-5058 taskManagerMemory attribute set wrong value In FlinkShell.scala, someone mistakenly set `slots` value to `taskManagerMemory` attribute as follow: ``` // set configuration from user input yarnConfig.jobManagerMemory.foreach((jmMem) => args ++= Seq("-yjm", jmMem.toString)) yarnConfig.slots.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString)) yarnConfig.name.foreach((name) => args ++= Seq("-ynm", name.toString)) yarnConfig.queue.foreach((queue) => args ++= Seq("-yqu", queue.toString)) yarnConfig.slots.foreach((slots) => args ++= Seq("-ys", slots.toString)) ``` Note on the third line: `yarnConfig.slots.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString))` , we set `slots` value to `taskManagerMemory` attribute, But in fact, we want to set `taskManagerMemory` attribute, the right code is as follow: ``` // set configuration from user input yarnConfig.jobManagerMemory.foreach((jmMem) => args ++= Seq("-yjm", jmMem.toString)) yarnConfig.taskManagerMemory.foreach((tmMem) => args ++= Seq("-ytm", tmMem.toString)) yarnConfig.name.foreach((name) => args ++= Seq("-ynm", name.toString)) yarnConfig.queue.foreach((queue) => args ++= Seq("-yqu", queue.toString)) yarnConfig.slots.foreach((slots) => args ++= Seq("-ys", slots.toString)) ``` please review tihs pull, thanks. You can merge this pull request into a Git repository by running: $ git pull https://github.com/397090770/flink master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/2799.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2799 commit f71f1d3f04f61500834c4d0d99d9b09df812781b Author: iteblog <wyphao.2007@163.com> Date: 2016-11-14T11:04:28Z taskManagerMemory attribute set wrong value taskManagerMemory attribute set wrong value

            People

            • Assignee:
              Unassigned
              Reporter:
              397090770 wyp
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 8h
                8h
                Remaining:
                Remaining Estimate - 8h
                8h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development