Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-25109

Add MR Counters to WALPlayer; currently hard to tell if it is doing anything

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha-1, 2.3.3, 2.4.0, 2.2.7
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Adds a WALPlayer to MR Counter output:

      org.apache.hadoop.hbase.mapreduce.WALPlayer$Counter
      CELLS_READ=89574
      CELLS_WRITTEN=89572
      DELETES=64
      PUTS=5305
      WALEDITS=4375
      Show
      Adds a WALPlayer to MR Counter output: org.apache.hadoop.hbase.mapreduce.WALPlayer$Counter CELLS_READ=89574 CELLS_WRITTEN=89572 DELETES=64 PUTS=5305 WALEDITS=4375

      Description

      For example, when WALPlayer runs, it emits this:

      
      020-09-28 11:16:05,489 INFO  [LocalJobRunner Map Task Executor #0] mapred.Task: Final Counters for attempt_local1916643172_0001_m_000000_0: Counters: 20
      	File System Counters
      		FILE: Number of bytes read=268891453
      		FILE: Number of bytes written=1018719
      		FILE: Number of read operations=0
      		FILE: Number of large read operations=0
      		FILE: Number of write operations=0
      	Map-Reduce Framework
      		Map input records=4375
      		Map output records=5369
      		Input split bytes=245
      		Spilled Records=0
      		Failed Shuffles=0
      		Merged Map outputs=0
      		GC time elapsed (ms)=59
      		Total committed heap usage (bytes)=518979584
      	File Input Format Counters
      		Bytes Read=0
      	File Output Format Counters
      		Bytes Written=0 

      Change it so it does this:

      020-09-28 11:16:05,489 INFO  [LocalJobRunner Map Task Executor #0] mapred.Task: Final Counters for attempt_local1916643172_0001_m_000000_0: Counters: 20
      	File System Counters
      		FILE: Number of bytes read=268891453
      		FILE: Number of bytes written=1018719
      		FILE: Number of read operations=0
      		FILE: Number of large read operations=0
      		FILE: Number of write operations=0
      	Map-Reduce Framework
      		Map input records=4375
      		Map output records=5369
      		Input split bytes=245
      		Spilled Records=0
      		Failed Shuffles=0
      		Merged Map outputs=0
      		GC time elapsed (ms)=59
      		Total committed heap usage (bytes)=518979584
      	org.apache.hadoop.hbase.mapreduce.WALPlayer$Counter
      		CELLS_READ=89574
      		CELLS_WRITTEN=89572
      		DELETES=64
      		PUTS=5305
      		WALEDITS=4375
      	File Input Format Counters
      		Bytes Read=0
      	File Output Format Counters
      		Bytes Written=0 

        Attachments

          Activity

            People

            • Assignee:
              stack Michael Stack
              Reporter:
              stack Michael Stack
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: