[IGNITE-5558] Add ability to read WAL outside of an Ignite node

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1
• Fix Version/s: 2.1
• Component/s: persistence
• Assignee: Dmitry Pavlov
• Reporter: Alexey Goncharuk
• Created: 20/Jun/17 11:55
• Updated: 10/Jan/18 14:24
• Resolved: 07/Jul/17 09:45

Description

Currently, the WAL iterator can be obtained from the WAL manager when an Ignite node is up and running.
However, it may be extremely useful to read WAL in an 'offline' mode, when a node is not started up. This may be required for crash analysis or to export committed data to some external systems.

In the future we can make this even a public interface, however as a starting point, I would like to keep it in private package because moving to the public package will require Iterator and records to be public too.

So, as a starting point, we need:

• An object that will allow us to get WALIterator instances (probably, should be closeable)
• A method on this object which will create an iterator by a file name or file names

Using this object should not require an active Ignite instance running.

Issue Links

• is related to: IGNITE-6553 Standalone WAL iterator fails to handle delete WAL data records
• is related to: IGNITE-6788 Ignite WAL reader fails on Tx marker record for persistent store with new style folder naming
• is related to: IGNITE-5598 WAL Iteration stopped with exception after writting log with background mode
• is related to: IGNITE-6539 Human readable WAL parser fails if empty log files exists in directory
• is related to: IGNITE-6540 Human readable WAL parser result has no human readable data
• is related to: IGNITE-5587 Generate File WAL Segment Archive Completed Event
• is related to: IGNITE-5589 Archive WAL segment after significant period of grid inactivity
• is related to: IGNITE-6235 Add ability to handle CacheObject from DataRecord in standalone WAL iterator
• is related to: IGNITE-7378 WAL converter: add records statistic to WAL reader dev-util
• is related to: IGNITE-6225 Improve tests of WAL iterator with checking DataRecord entries and TxRecords
• is related to: IGNITE-6421 Ignite WAL reader: add shell script to start converter
• relates to: IGNITE-6277 Convert WAL to human readable form
• relates to: IGNITE-6014 Add transaction prepare and commit markers to WAL
• relates to: IGNITE-6235 Add ability to handle CacheObject from DataRecord in standalone WAL iterator
• links to: GitHub Pull Request #2174
• mentioned in: Page Loading...