Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3191

Add explanation of supporting building spark with maven in http proxy environment

    XMLWordPrintableJSON

    Details

    • Type: Documentation
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Won't Fix
    • Affects Version/s: 1.0.2
    • Fix Version/s: 1.2.0
    • Component/s: Documentation
    • Labels:
    • Environment:
    • Target Version/s:

      Description

      When I use "mvn -Pyarn -Phadoop-2.2 -Dhadoop.version=2.2.0 -DskipTests clean package" in http proxy enviroment, I cannot finish this task.Error is as follows:
      [INFO] Spark Project YARN Stable API ..................... SUCCESS [34.217s]
      [INFO] Spark Project Assembly ............................ FAILURE [43.133s]
      [INFO] Spark Project External Twitter .................... SKIPPED
      [INFO] Spark Project External Kafka ...................... SKIPPED
      [INFO] Spark Project External Flume ...................... SKIPPED
      [INFO] Spark Project External ZeroMQ ..................... SKIPPED
      [INFO] Spark Project External MQTT ....................... SKIPPED
      [INFO] Spark Project Examples ............................ SKIPPED
      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD FAILURE
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 27:57.309s
      [INFO] Finished at: Sat Aug 23 09:43:21 CST 2014
      [INFO] Final Memory: 51M/1080M
      [INFO] ------------------------------------------------------------------------
      [ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:2.2:shade (default) on project spark-assembly_2.10: Execution default of goal org.apache.maven.plugins:maven-shade-plugin:2.2:shade failed: Plugin org.apache.maven.plugins:maven-shade-plugin:2.2 or one of its dependencies could not be resolved: Could not find artifact com.google.code.findbugs:jsr305:jar:1.3.9 -> [Help 1]
      [ERROR]
      [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
      [ERROR] Re-run Maven using the -X switch to enable full debug logging.
      [ERROR]
      [ERROR] For more information about the errors and possible solutions, please read the following articles:
      [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
      [ERROR]
      [ERROR] After correcting the problems, you can resume the build with the command
      [ERROR] mvn <goals> -rf :spark-assembly_2.10

      If you use this command, It is ok
      mvn -Pyarn -Phadoop-2.2 -Dhadoop.version=2.2.0 -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -DskipTests clean package

      The error is not very obvious, I spent a long time to solve this issues
      In order to facilitate other guys who use spark in http proxy environment, I highly recommed add this to documents

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              bing zhengbing li
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified