Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3918

Forget Unpersist in RandomForest.scala(train Method)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: MLlib
    • Environment:

      All

    • Target Version/s:

      Description

      In version 1.1.0 DecisionTree.scala, train Method, treeInput has been persisted in Memory, but without unpersist. It caused heavy DISK usage.
      In github version(1.2.0 maybe), RandomForest.scala, train Method, baggedInput has been persisted but without unpersisted too.

        Attachments

          Activity

            People

            • Assignee:
              josephkb Joseph K. Bradley
              Reporter:
              Junlong Liu junlong
              Shepherd:
              paulbits
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 10m
                10m
                Remaining:
                Remaining Estimate - 10m
                10m
                Logged:
                Time Spent - Not Specified
                Not Specified