[LUCENE-3349] Place warning about today's released Java7 version on Lucene/Solr/Root webpage's news and send mail to java-user

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: general/test
• Labels: Java7

Environment

Java7

• Lucene Fields: New
• Assignee: Uwe Schindler
• Reporter: Uwe Schindler
• Due: 28/Jul/11
• Created: 28/Jul/11 17:20
• Updated: 28/Aug/22 12:54
• Resolved: 28/Jul/11 21:15

Description

Today, JDK/JRE 1.7.0 GA was released by Oracle. Unfortunately they didn't fix the Hotspot problems affecting loops to be miscompiled (LUCENE-3335, LUCENE-3346). This can lead to Solr crashing with default configuration on startup or sudden index corrumption depending on configuration.

We should send an email to the java-user and solr-user list describing the problem. Also place a note in the news section of Solr, Lucene Core and top-level website.

I propose the following text:

Jul 28th, 2011: WARNING: Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Oracle released Java 7 today. Unfortunately it contains hotspot compiler optimizations, which miscompile some loops. This can affect code of several Apache projects. Sometimes JVMs only crash, but in several cases, results calculated can be incorrect, leading to bugs in applications (see Hotspot bugs http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134, http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738, http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051).

Apache Lucene Core and Apache Solr are two Apache projects, which are affected by these bugs, namely all versions released until today. Solr users with the default configuration will have Java crashing with SIGSEGV as soon as they start to index documents, as one affected part is the well-known Porter stemmer (see LUCENE-3335). Other loops in Lucene may be miscompiled, too, leading to index corruption (especially on Lucene trunk with pulsing codec; other loops may be affected, too - LUCENE-3346, ).

These problems were detected only 5 days before the official Java 7 release, so Oracle had no time to fix those bugs, affecting also many more applications. In response to our questions, they proposed to include the fixes into service release u2 (eventually into service release u1, see http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2011-July/005971.html). This means you cannot use Apache Lucene/Solr with Java 7 releases before Update 2! If you do, please don't open bug reports, it is not the committers' fault! At least disable loop optimizations using the -XX:-UseLoopPredicate JVM options to not risk index corruptions.

Please note: Also Java 6 users are affected, if they use one of those JVM options, which are not enabled by default: -XX:+OptimizeStringConcat or -XX:+AggressiveOpts

It is strongly recommended not to use any hotspot optimization switches in any Java version without extensive testing!

In case you upgrade to Java 7, remember that you may have to reindex, as the unicode version shipped with Java 7 changed and tokenization behaves differently (e.g. lowercasing). For more information, read JRE_VERSION_MIGRATION.txt in your distribution package!

Issue Links

• is superceded by: LUCENE-3537 Add note about Java 7u1 and 6u29 to Lucene/Solr sites