Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20336

org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad fails

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Cannot Reproduce
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: spark
    • Labels:
      None

      Description

      running mvn verify for the spark integration tests against current master fails

      $ mvn -DskipTests install
      11:37:26,815 [INFO] Scanning for projects...
      ...
      11:43:36,711 [INFO] ------------------------------------------------------------------------
      11:43:36,711 [INFO] Reactor Summary:
      11:43:36,711 [INFO] 
      11:43:36,712 [INFO] Apache HBase ....................................... SUCCESS [  6.294 s]
      11:43:36,712 [INFO] Apache HBase - Checkstyle .......................... SUCCESS [  1.742 s]
      11:43:36,712 [INFO] Apache HBase - Build Support ....................... SUCCESS [  0.070 s]
      11:43:36,712 [INFO] Apache HBase - Error Prone Rules ................... SUCCESS [  2.206 s]
      11:43:36,712 [INFO] Apache HBase - Annotations ......................... SUCCESS [  1.413 s]
      11:43:36,712 [INFO] Apache HBase - Build Configuration ................. SUCCESS [  0.254 s]
      11:43:36,712 [INFO] Apache HBase - Shaded Protocol ..................... SUCCESS [ 37.870 s]
      11:43:36,712 [INFO] Apache HBase - Common .............................. SUCCESS [ 12.526 s]
      11:43:36,712 [INFO] Apache HBase - Metrics API ......................... SUCCESS [  2.412 s]
      11:43:36,712 [INFO] Apache HBase - Hadoop Compatibility ................ SUCCESS [  3.260 s]
      11:43:36,712 [INFO] Apache HBase - Metrics Implementation .............. SUCCESS [  2.756 s]
      11:43:36,713 [INFO] Apache HBase - Hadoop Two Compatibility ............ SUCCESS [  3.959 s]
      11:43:36,713 [INFO] Apache HBase - Protocol ............................ SUCCESS [ 11.295 s]
      11:43:36,713 [INFO] Apache HBase - Client .............................. SUCCESS [ 15.360 s]
      11:43:36,713 [INFO] Apache HBase - Zookeeper ........................... SUCCESS [  4.389 s]
      11:43:36,713 [INFO] Apache HBase - Replication ......................... SUCCESS [  4.202 s]
      11:43:36,713 [INFO] Apache HBase - Resource Bundle ..................... SUCCESS [  0.206 s]
      11:43:36,713 [INFO] Apache HBase - HTTP ................................ SUCCESS [  8.530 s]
      11:43:36,713 [INFO] Apache HBase - Procedure ........................... SUCCESS [  4.196 s]
      11:43:36,713 [INFO] Apache HBase - Server .............................. SUCCESS [ 44.604 s]
      11:43:36,713 [INFO] Apache HBase - MapReduce ........................... SUCCESS [ 11.122 s]
      11:43:36,713 [INFO] Apache HBase - Testing Util ........................ SUCCESS [  6.633 s]
      11:43:36,713 [INFO] Apache HBase - Thrift .............................. SUCCESS [  9.771 s]
      11:43:36,713 [INFO] Apache HBase - RSGroup ............................. SUCCESS [  6.703 s]
      11:43:36,713 [INFO] Apache HBase - Shell ............................... SUCCESS [  7.094 s]
      11:43:36,713 [INFO] Apache HBase - Coprocessor Endpoint ................ SUCCESS [  7.542 s]
      11:43:36,713 [INFO] Apache HBase - Backup .............................. SUCCESS [  6.246 s]
      11:43:36,713 [INFO] Apache HBase - Integration Tests ................... SUCCESS [  7.461 s]
      11:43:36,713 [INFO] Apache HBase - Examples ............................ SUCCESS [  9.054 s]
      11:43:36,713 [INFO] Apache HBase - Rest ................................ SUCCESS [  8.972 s]
      11:43:36,713 [INFO] Apache HBase - External Block Cache ................ SUCCESS [  5.180 s]
      11:43:36,713 [INFO] Apache HBase - Spark ............................... SUCCESS [01:10 min]
      11:43:36,713 [INFO] Apache HBase - Spark Integration Tests ............. SUCCESS [  7.774 s]
      11:43:36,713 [INFO] Apache HBase - Assembly ............................ SUCCESS [ 15.075 s]
      11:43:36,713 [INFO] Apache HBase - Shaded .............................. SUCCESS [  0.323 s]
      11:43:36,713 [INFO] Apache HBase - Shaded - Client ..................... SUCCESS [  3.004 s]
      11:43:36,713 [INFO] Apache HBase - Shaded - MapReduce .................. SUCCESS [  3.542 s]
      11:43:36,714 [INFO] Apache HBase Shaded Packaging Invariants ........... SUCCESS [  3.105 s]
      11:43:36,714 [INFO] Apache HBase - Archetypes .......................... SUCCESS [  0.063 s]
      11:43:36,714 [INFO] Apache HBase - Exemplar for hbase-client archetype . SUCCESS [  4.843 s]
      11:43:36,714 [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SUCCESS [  4.523 s]
      11:43:36,714 [INFO] Apache HBase - Archetype builder ................... SUCCESS [  0.739 s]
      11:43:36,714 [INFO] ------------------------------------------------------------------------
      11:43:36,714 [INFO] BUILD SUCCESS
      11:43:36,714 [INFO] ------------------------------------------------------------------------
      11:43:36,714 [INFO] Total time: 06:09 min
      11:43:36,714 [INFO] Finished at: 2018-04-03T11:43:36-05:00
      11:43:38,408 [INFO] Final Memory: 305M/1087M
      11:43:38,408 [INFO] ------------------------------------------------------------------------
      $ mvn -pl hbase-spark -pl hbase-spark-it verify
      11:45:14,664 [INFO] Scanning for projects...
      ...
      
      11:49:27,439 [INFO] --- maven-failsafe-plugin:2.21.0:integration-test (integration-test) @ hbase-spark-it ---
      11:49:27,576 [WARNING] The parameter forkMode is deprecated since version 2.14. Use forkCount and reuseForks instead.
      11:49:27,641 [INFO] 
      11:49:27,641 [INFO] -------------------------------------------------------
      11:49:27,641 [INFO]  T E S T S
      11:49:27,641 [INFO] -------------------------------------------------------
      11:49:27,866 [WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file /Users/busbey/tmp_projects/hbase/hbase-spark-it/target/failsafe-reports/2018-04-03T11-49-27_574-jvmRun1.dumpstream
      11:49:28,752 [INFO] Running org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad
      11:49:36,776 [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.995 s <<< FAILURE! - in org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad
      11:49:36,776 [ERROR] testBulkLoad(org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad)  Time elapsed: 7.699 s  <<< ERROR!
      java.io.IOException: Shutting down
      	at org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad.setUpCluster(IntegrationTestSparkBulkLoad.java:613)
      Caused by: java.lang.RuntimeException: Failed construction of RegionServer: class org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer
      	at org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad.setUpCluster(IntegrationTestSparkBulkLoad.java:613)
      Caused by: java.lang.IllegalArgumentException: port out of range:-1
      	at org.apache.hadoop.hbase.spark.IntegrationTestSparkBulkLoad.setUpCluster(IntegrationTestSparkBulkLoad.java:613)
      
      11:49:37,190 [INFO] 
      11:49:37,190 [INFO] Results:
      11:49:37,191 [INFO] 
      11:49:37,191 [ERROR] Errors: 
      11:49:37,191 [ERROR]   IntegrationTestSparkBulkLoad>IntegrationTestBase.setUp:170->setUpCluster:613 ยป IO
      11:49:37,191 [INFO] 
      11:49:37,191 [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
      11:49:37,191 [INFO] 
      11:49:37,199 [INFO] 
      11:49:37,199 [INFO] --- maven-failsafe-plugin:2.21.0:verify (verify) @ hbase-spark-it ---
      11:49:37,848 [INFO] ------------------------------------------------------------------------
      11:49:37,848 [INFO] Reactor Summary:
      11:49:37,848 [INFO] 
      11:49:37,848 [INFO] Apache HBase - Spark ............................... SUCCESS [03:57 min]
      11:49:37,848 [INFO] Apache HBase - Spark Integration Tests ............. FAILURE [ 22.686 s]
      11:49:37,849 [INFO] ------------------------------------------------------------------------
      11:49:37,849 [INFO] BUILD FAILURE
      11:49:37,849 [INFO] ------------------------------------------------------------------------
      11:49:37,849 [INFO] Total time: 04:23 min
      11:49:37,849 [INFO] Finished at: 2018-04-03T11:49:37-05:00
      11:49:38,093 [INFO] Final Memory: 75M/762M
      11:49:38,094 [INFO] ------------------------------------------------------------------------
      11:49:38,096 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.21.0:verify (verify) on project hbase-spark-it: There are test failures.
      11:49:38,096 [ERROR] 
      11:49:38,096 [ERROR] Please refer to /Users/busbey/tmp_projects/hbase/hbase-spark-it/target/failsafe-reports for the individual test results.
      11:49:38,096 [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
      11:49:38,096 [ERROR] -> [Help 1]
      11:49:38,096 [ERROR] 
      11:49:38,096 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
      11:49:38,096 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
      11:49:38,096 [ERROR] 
      11:49:38,096 [ERROR] For more information about the errors and possible solutions, please read the following articles:
      11:49:38,096 [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
      11:49:38,096 [ERROR] 
      11:49:38,096 [ERROR] After correcting the problems, you can resume the build with the command
      11:49:38,096 [ERROR]   mvn <goals> -rf :hbase-spark-it
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                busbey Sean Busbey
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: