[LUCENE-8567] TextIndexManager checksum failed on s390x

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 7.5
• Component/s: core/codecs

Environment

SLES 12.3 on s390x with IBM J9 VM

• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Stefan Franz
• Created: 19/Nov/18 13:50
• Updated: 28/Aug/22 15:38

Description

Hello,

we are using Apache Lucene within Jetbrain's Youtrack and have runtime problems with IBM's J9 VM version 8, but the same bug occurs when using openjdk on the same platform. Maybe there is a problem with the big-endian byte order of the z-platform? 

  

[2018-11-14 18:08:11,470] 18:08:11,462 ERROR [@extIndexManager] [TextIndexManager ] [] Failed to index 111 documents
[2018-11-14 18:08:11,471] org.apache.lucene.index.CorruptIndexException: checksum failed (hardware problem?) : expected=fa0a2c04 actual=a7c0aac0 (resource=BufferedChecksumIndexInput(ExodusIndexInput[_0_Lucene50_0.doc]))
[2018-11-14 18:08:11,471] at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:419) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,472] at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:99) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,472] at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4997) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,473] at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:576) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,473] at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:515) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,473] at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:554) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,474] at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:719) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,474] at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3602) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,474] at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3577) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,475] at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1035) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,475] at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1078) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]
[2018-11-14 18:08:11,475] at jetbrains.youtrack.textindex.TextIndexManagerImpl$LuceneIndexingContext.close(TextIndexManagerImpl.java:843) ~[youtrack-text-search-2018.3.jar:?]
[2018-11-14 18:08:11,476] at jetbrains.youtrack.textindex.TextIndexManagerBase$BatchIndexingJob.execute(TextIndexManagerBase.java:995) [youtrack-text-search-2018.3.jar:?]
[2018-11-14 18:08:11,476] at jetbrains.exodus.core.execution.DecoratorJob.executeDecorated(DecoratorJob.java:72) [xodus-utils-1.2.3501.jar:1.2.3501]
[2018-11-14 18:08:11,476] at jetbrains.youtrack.textindex.settings.TransactionalTextIndexJob.access$000(TransactionalTextIndexJob.java:10) [youtrack-text-search-2018.3.jar:?]
[2018-11-14 18:08:11,477] at jetbrains.youtrack.textindex.settings.TransactionalTextIndexJob$1.invoke(TransactionalTextIndexJob.java:25) [youtrack-text-search-2018.3.jar:?]
[2018-11-14 18:08:11,477] at jetbrains.teamsys.dnq.runtime.txn._Txn.runReadonly(_Txn.java:351) [jetbrains.teamsys.dnq.runtime-3438.jar:?]
[2018-11-14 18:08:11,477] at jetbrains.youtrack.textindex.settings.TransactionalTextIndexJob.execute(TransactionalTextIndexJob.java:23) [youtrack-text-search-2018.3.jar:?]
[2018-11-14 18:08:11,477] at jetbrains.exodus.core.execution.Job.run(Job.java:99) [xodus-utils-1.2.3501.jar:1.2.3501]
[2018-11-14 18:08:11,478] at jetbrains.exodus.core.execution.ThreadJobProcessor.executeJob(ThreadJobProcessor.java:133) [xodus-utils-1.2.3501.jar:1.2.3501]
[2018-11-14 18:08:11,478] at jetbrains.exodus.core.execution.JobProcessorQueueAdapter.doExecuteJob(JobProcessorQueueAdapter.java:243) [xodus-utils-1.2.3501.jar:1.2.3501]
[2018-11-14 18:08:11,478] at jetbrains.exodus.core.execution.JobProcessorQueueAdapter.doTimedJobs(JobProcessorQueueAdapter.java:195) [xodus-utils-1.2.3501.jar:1.2.3501]
[2018-11-14 18:08:11,479] at jetbrains.exodus.core.execution.JobProcessorQueueAdapter.doJobs(JobProcessorQueueAdapter.java:163) [xodus-utils-1.2.3501.jar:1.2.3501]
[2018-11-14 18:08:11,479] at jetbrains.exodus.core.execution.ThreadJobProcessor.run(ThreadJobProcessor.java:111) [xodus-utils-1.2.3501.jar:1.2.3501]
[2018-11-14 18:08:11,479] at jetbrains.exodus.core.execution.ThreadJobProcessor$1.run(ThreadJobProcessor.java:43) [xodus-utils-1.2.3501.jar:1.2.3501]
[2018-11-14 18:08:11,479] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]