Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Solr smoke test fails with the following error.
> Task :bigtop-tests:smoke-tests:solr:test Downloading https://repo.maven.apache.org/maven2/org/codehaus/woodstox/wstx-asl/3.2.7/wstx-asl-3.2.7.pom to /tmp/gradle_download5228771710184587059bin Downloading https://repo.maven.apache.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.pom to /tmp/gradle_download7723517104372501069bin Downloading https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.pom to /tmp/gradle_download5040172534686132912bin Downloading https://repo.maven.apache.org/maven2/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar to /tmp/gradle_download8433979289897434251bin Downloading https://repo.maven.apache.org/maven2/org/codehaus/woodstox/wstx-asl/3.2.7/wstx-asl-3.2.7.jar to /tmp/gradle_download7704021333708043047bin Downloading https://repo.maven.apache.org/maven2/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar to /tmp/gradle_download963266749008733834bin Caching disabled for task ':bigtop-tests:smoke-tests:solr:test' because: Build cache is disabled Task ':bigtop-tests:smoke-tests:solr:test' is not up-to-date because: No history is available. Starting process 'command '/usr/bin/solrctl''. Working directory: /bigtop-home/bigtop-tests/smoke-tests/solr Command: /usr/bin/solrctl instancedir --generate /tmp/smoke_data Successfully started process 'command '/usr/bin/solrctl'' Starting process 'command '/usr/bin/solrctl''. Working directory: /bigtop-home/bigtop-tests/smoke-tests/solr Command: /usr/bin/solrctl instancedir --create smoke /tmp/smoke_data Successfully started process 'command '/usr/bin/solrctl'' Uploading configs from /tmp/smoke_data/conf to f028d5337d92.bigtop.apache.org:2181/solr. This may take up to a minute. Starting process 'command '/usr/bin/solrctl''. Working directory: /bigtop-home/bigtop-tests/smoke-tests/solr Command: /usr/bin/solrctl collection --create smoke Successfully started process 'command '/usr/bin/solrctl'' Error: can't discover Solr URI. Please specify it explicitly via --solr. > Task :bigtop-tests:smoke-tests:solr:test FAILED :bigtop-tests:smoke-tests:solr:test (Thread[Execution worker for ':',5,main]) completed. Took 11.357 secs. FAILURE: Build failed with an exception.
I could reproduce this failure by running the test on the ppc64le server directly. It looks like that solr-server didn't start due to a memory issue.
jenkins@bigtop1slave:~/bigtop/provisioner/docker$ ./docker-hadoop.sh --exec 1 jps
1073 NameNode
1747 GradleDaemon
1257 DataNode
553 QuorumPeerMain
2031 Jps
jenkins@bigtop1slave:~/bigtop/provisioner/docker$ ./docker-hadoop.sh --exec 1 systemctl status solr-server
● solr-server.service - LSB: Apache Solr as a Service
Loaded: loaded (/etc/rc.d/init.d/solr-server; generated)
Active: active (exited) since Fri 2021-08-13 06:22:02 UTC; 1min 30s ago
Docs: man:systemd-sysv-generator(8)
Aug 13 06:21:51 31fbfd1187ec systemd[1]: Starting LSB: Apache Solr as a Service...
Aug 13 06:21:51 31fbfd1187ec su[841]: (to solr) root on none
Aug 13 06:21:52 31fbfd1187ec su[841]: pam_unix(su:session): session opened for user solr(uid=993) by (uid=0)
Aug 13 06:22:02 31fbfd1187ec solr-server[931]: Starting Apache Solr server in cloud mode[ OK ]
Aug 13 06:22:02 31fbfd1187ec systemd[1]: Started LSB: Apache Solr as a Service.
jenkins@bigtop1slave:~/bigtop/provisioner/docker$ ./docker-hadoop.sh --exec 1 cat /var/log/solr/solr-8983-console.log
The stack size specified is too small, Specify at least 328k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
That was not resolved by extending memory limit to 8GB.
jenkins@bigtop1slave:~/bigtop/provisioner/docker$ docker stats --no-stream CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 31fbfd1187ec 20210813_061339_r31566_bigtop_1 0.03% 882.9MiB / 8GiB 10.78% 253MB / 1.27MB 655kB / 259MB 28
