Uploaded image for project: 'SystemML'
  1. SystemML
  2. SYSTEMML-1346

Python MLContext Should Accept New SparkSession

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: SystemML 0.15
    • Component/s: APIs
    • Labels:
      None

      Description

      Currently, our Python MLContext API only accepts the older SparkContext, typically referred to with the variable name sc. We should extend this API to accept either the older SparkContext, or the newer SparkSession, the latter of which is typically referred to with the variable name spark.

        Issue Links

          Activity

          Hide
          deron Deron Eriksson added a comment -

          Great idea [~mwdusenb@us.ibm.com]. I'll create a JIRA to also accept SparkSession in Java/Scala MLContext API.

          Show
          deron Deron Eriksson added a comment - Great idea [~mwdusenb@us.ibm.com] . I'll create a JIRA to also accept SparkSession in Java/Scala MLContext API.
          Hide
          deron Deron Eriksson added a comment -

          Also see SYSTEMML-1347 (Accept SparkSession in Java/Scala MLContext API).

          Show
          deron Deron Eriksson added a comment - Also see SYSTEMML-1347 (Accept SparkSession in Java/Scala MLContext API).
          Hide
          gweidner Glenn Weidner added a comment -

          This has been fixed with Niketan's commit for SYSTEMML-1798.

          Show
          gweidner Glenn Weidner added a comment - This has been fixed with Niketan's commit for SYSTEMML-1798 .
          Hide
          gweidner Glenn Weidner added a comment -

          Marking resolved from PR 590.

          Show
          gweidner Glenn Weidner added a comment - Marking resolved from PR 590 .

            People

            • Assignee:
              niketanpansare Niketan Pansare
              Reporter:
              dusenberrymw Mike Dusenberry
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development

                  Agile