Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: Mesos
    • Labels:
      None
    • Target Version/s:

      Issue Links

        Activity

        Hide
        rxin Reynold Xin added a comment -

        cc Iulian Dragos / Timothy Chen do we still want to do this?

        Show
        rxin Reynold Xin added a comment - cc Iulian Dragos / Timothy Chen do we still want to do this?
        Hide
        dragos Iulian Dragos added a comment - - edited

        There's one pending issue in the email thread, regarding a 20-30% performance drop by switching to the coarse-grained mode. Maybe it makes sense to just deprecate it as a first step, until we figure out the issue?

        PS: http://apache-spark-developers-list.1001551.n3.nabble.com/Removing-the-Mesos-fine-grained-mode-tp15277p15388.html

        Show
        dragos Iulian Dragos added a comment - - edited There's one pending issue in the email thread, regarding a 20-30% performance drop by switching to the coarse-grained mode. Maybe it makes sense to just deprecate it as a first step, until we figure out the issue? PS: http://apache-spark-developers-list.1001551.n3.nabble.com/Removing-the-Mesos-fine-grained-mode-tp15277p15388.html
        Hide
        rxin@databricks.com Reynold Xin added a comment -

        That sounds good to me. At the very least we should switch the default to
        coarsed grain first. Can you submit a pull request? Thanks.

        On Wednesday, January 13, 2016, Iulian Dragos (JIRA) <jira@apache.org>

        Show
        rxin@databricks.com Reynold Xin added a comment - That sounds good to me. At the very least we should switch the default to coarsed grain first. Can you submit a pull request? Thanks. On Wednesday, January 13, 2016, Iulian Dragos (JIRA) <jira@apache.org>
        Hide
        dragos Iulian Dragos added a comment -

        Sure, but that seems to be done already... by you: https://github.com/apache/spark/pull/9795

        Show
        dragos Iulian Dragos added a comment - Sure, but that seems to be done already... by you: https://github.com/apache/spark/pull/9795
        Hide
        rxin Reynold Xin added a comment -

        Oh yes

        Let me know if you want to deprecate it and submit a patch. Otherwise I will just close this ticket.

        Show
        rxin Reynold Xin added a comment - Oh yes Let me know if you want to deprecate it and submit a patch. Otherwise I will just close this ticket.
        Hide
        rxin Reynold Xin added a comment -

        Closing this as later until we can figure out what's going on with resource utilization.

        Show
        rxin Reynold Xin added a comment - Closing this as later until we can figure out what's going on with resource utilization.
        Hide
        rxin Reynold Xin added a comment -

        Maybe we should deprecate this first.

        Show
        rxin Reynold Xin added a comment - Maybe we should deprecate this first.
        Hide
        andrewor14 Andrew Or added a comment -

        Iulian DragosTimothy Chen any thoughts on at least deprecating this?

        Show
        andrewor14 Andrew Or added a comment - Iulian Dragos Timothy Chen any thoughts on at least deprecating this?
        Hide
        mgummelt Michael Gummelt added a comment -

        I endorse the deprecation. Fine-grained mode would be more useful if the JVM could shrink in memory as well as cores, but alas...

        We at Mesosphere haven't heard any objections from users regarding the loss of fine-grained.

        Andrew Or Please cc me if you need Mesos input. Tim is still active, I believe, but no longer at Mesosphere. I work (mostly) full-time on Spark on Mesos.

        Show
        mgummelt Michael Gummelt added a comment - I endorse the deprecation. Fine-grained mode would be more useful if the JVM could shrink in memory as well as cores, but alas... We at Mesosphere haven't heard any objections from users regarding the loss of fine-grained. Andrew Or Please cc me if you need Mesos input. Tim is still active, I believe, but no longer at Mesosphere. I work (mostly) full-time on Spark on Mesos.
        Hide
        mgummelt Michael Gummelt added a comment -

        Adam McElwee Do you have any more input on this issue. We're moving forward with deprecating fine-grained mode, but we're willing to solve your issue first.

        Show
        mgummelt Michael Gummelt added a comment - Adam McElwee Do you have any more input on this issue. We're moving forward with deprecating fine-grained mode, but we're willing to solve your issue first.
        Hide
        rxin@databricks.com Reynold Xin added a comment -

        Michael can you submit a pull request to log deprecation and perhaps also
        update the docs?

        Show
        rxin@databricks.com Reynold Xin added a comment - Michael can you submit a pull request to log deprecation and perhaps also update the docs?
        Hide
        mgummelt Michael Gummelt added a comment -

        For completeness, here's my theoretical analysis to augment our empirical observation that users don't mind fine-grained mode being removed.

        Fine-grained mode provides two benefits:
        1) Slow-start
        Executors are brought up lazily

        2) Relinquishing cores
        Cores are relinquished back to Mesos as Spark tasks terminate

        Fine-grained mdoe does not provide the following benefits, though some think it does:
        a) Relinquishing memory
        The JVM doesn't relinquish memory, so it would be unsafe for us to resize the cgroup

        b) Relinquishing executors

        As for alternatives to the benefits, 1) is provided by dynamic allocation, though we need a better recommended setup for this as I document here: http://apache-spark-developers-list.1001551.n3.nabble.com/HDFS-as-Shuffle-Service-td17340.html
        There is no alternative to 2), but we've generally found that the executor-level granularity of dynamic allocation is sufficient for most.

        Show
        mgummelt Michael Gummelt added a comment - For completeness, here's my theoretical analysis to augment our empirical observation that users don't mind fine-grained mode being removed. Fine-grained mode provides two benefits: 1) Slow-start Executors are brought up lazily 2) Relinquishing cores Cores are relinquished back to Mesos as Spark tasks terminate Fine-grained mdoe does not provide the following benefits, though some think it does: a) Relinquishing memory The JVM doesn't relinquish memory, so it would be unsafe for us to resize the cgroup b) Relinquishing executors As for alternatives to the benefits, 1) is provided by dynamic allocation, though we need a better recommended setup for this as I document here: http://apache-spark-developers-list.1001551.n3.nabble.com/HDFS-as-Shuffle-Service-td17340.html There is no alternative to 2), but we've generally found that the executor-level granularity of dynamic allocation is sufficient for most.
        Hide
        mgummelt Michael Gummelt added a comment -

        I'll give Adam McElwee a couple days to respond.

        Iulian Dragos Stavros Kontopoulos Timothy Chen speak now or forever hold your peace.

        Show
        mgummelt Michael Gummelt added a comment - I'll give Adam McElwee a couple days to respond. Iulian Dragos Stavros Kontopoulos Timothy Chen speak now or forever hold your peace.
        Hide
        amcelwee Adam McElwee added a comment -

        Nah, carry on. I'm not using spark at the new job, so I won't be able to pass along any new info.

        Show
        amcelwee Adam McElwee added a comment - Nah, carry on. I'm not using spark at the new job, so I won't be able to pass along any new info.
        Hide
        mgummelt Michael Gummelt added a comment -

        Thanks!

        Show
        mgummelt Michael Gummelt added a comment - Thanks!
        Hide
        apachespark Apache Spark added a comment -

        User 'mgummelt' has created a pull request for this issue:
        https://github.com/apache/spark/pull/14078

        Show
        apachespark Apache Spark added a comment - User 'mgummelt' has created a pull request for this issue: https://github.com/apache/spark/pull/14078

          People

          • Assignee:
            mgummelt Michael Gummelt
            Reporter:
            rxin Reynold Xin
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development