[HBASE-26464] WALPrettyPrinter: JSON output is broken/malformed

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-alpha-1, 1.7.1, 2.4.8
• Component/s: tooling
• Assignee: Nihal Jain
• Reporter: Nihal Jain
• Created: 18/Nov/21 06:26
• Updated: 25/Nov/21 13:09

Description

The JSON output of wal pretty printer is malformed and cannot be parsed.
There are 2 major issue:

• The writer classes and cell codec classes are added (as normal text) whenever a new WAL file printing starts (NOTE: WALPrettyPrinter can take multiple WAL files as input)
• The edit heap size and position are outside json formatting

These are side effects of some patches done in past which missed to cover json output scenarios.

Current sample output, which is malformed JSON:

[Writer Classes: ProtobufLogWriter AsyncProtobufLogWriter
Cell Codec Class: org.apache.hadoop.hbase.regionserver.wal.WALCellCodec
{"sequence":"3","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":176,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170532553","total_size_sum":"288"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit heap size: 328
position: 389
,{"sequence":"26","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":218,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170533215","total_size_sum":"336"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit heap size: 376
position: 726
,{"sequence":"4","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":9,"row":"default","type":"Put","family":"info","timestamp":"1637170533278","total_size_sum":"96"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit heap size: 136
position: 834
,{"sequence":"5","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":7,"row":"hbase","type":"Put","family":"info","timestamp":"1637170533340","total_size_sum":"88"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit heap size: 128
position: 938
,{"sequence":"27","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"","vlen":2,"row":"r6","type":"Put","family":"cf","timestamp":"1637170714545","total_size_sum":"80"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit heap size: 120
position: 1020
Writer Classes: ProtobufLogWriter AsyncProtobufLogWriter
Cell Codec Class: org.apache.hadoop.hbase.regionserver.wal.WALCellCodec
,{"sequence":"3","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":176,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170532553","total_size_sum":"288"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit heap size: 328
position: 389
,{"sequence":"26","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"HBASE::REGION_EVENT::REGION_OPEN","vlen":218,"row":"\\x00","type":"Put","family":"METAFAMILY","timestamp":"1637170533215","total_size_sum":"336"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit heap size: 376
position: 726
,{"sequence":"4","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":9,"row":"default","type":"Put","family":"info","timestamp":"1637170533278","total_size_sum":"96"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit heap size: 136
position: 834
,{"sequence":"5","region":"f0153c5d890d96f06ec304eebc4aacf9","actions":[{"qualifier":"d","vlen":7,"row":"hbase","type":"Put","family":"info","timestamp":"1637170533340","total_size_sum":"88"}],"table":{"name":[104,98,97,115,101,58,110,97,109,101,115,112,97,99,101],"nameAsString":"hbase:namespace","namespace":[104,98,97,115,101],"namespaceAsString":"hbase","qualifier":[110,97,109,101,115,112,97,99,101],"qualifierAsString":"namespace","systemTable":true,"hashCode":771659354}}edit heap size: 128
position: 938
,{"sequence":"27","region":"e42fbcf354fcf7e4e425f2d06130797c","actions":[{"qualifier":"","vlen":2,"row":"r6","type":"Put","family":"cf","timestamp":"1637170714545","total_size_sum":"80"}],"table":{"name":[116,101,115,116],"nameAsString":"test","namespace":[100,101,102,97,117,108,116],"namespaceAsString":"default","qualifier":[116,101,115,116],"qualifierAsString":"test","systemTable":false,"hashCode":3556498}}edit heap size: 120
position: 1020
]

Issue Links

• relates to: HBASE-21185 WALPrettyPrinter: Additional useful info to be printed by wal printer tool, for debugability purposes
• relates to: HBASE-11864 Enhance HLogPrettyPrinter to print information from WAL Header