Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1415

Remove redundant jars from spark package.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.7.0
    • 0.8.0
    • spark
    • None

    Description

      Looking at the current package for Spark - it packages a lot of things that are not necessary, and is missing some others that might be useful.

      bagel/lib/spark-bagel_2.10-1.0.0-*.jar
      core/lib/spark-core_2.10-1.0.0-*.jar
      lib/*
      mllib/lib/spark-mllib_2.10-1.0.0-*.jar
      repl/lib/spark-repl_2.10-1.0.0-*.jar
      streaming/lib/spark-streaming_2.10-1.0.0-*.jar

      None of the above are necessary, since they are already packaged in assembly/lib/spark-assembly-1.0.0-*.jar, which is what everybody should be using.
      examples/lib/spark-examples_2.10-1.0.0-*-javadoc.jar
      examples/lib/spark-examples_2.10-1.0.0-*-sources.jar
      Again not necessary, just because, well, anyone who needs that can use maven.

      Attachments

        1. BIGTOP-1415.patch
          3 kB
          Manikandan Narayanaswamy

        Activity

          People

            Unassigned Unassigned
            mnarayan Manikandan Narayanaswamy
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: