Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10592

Refactor PerformanceEvaluation tool

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.96.2, 0.98.1, 0.99.0
    • Fix Version/s: 0.96.2, 0.98.1, 0.99.0, hbase-10070
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      PerfEval is kind of a mess. It's painful to add new features because the test options are itemized and passed as parameters to internal methods. Serialization is hand-rolled and tedious. Ensuring support for mapreduce mode is a chore because of it.

      This patch refactors the tool. Options are now passed around to methods and such as a POJO instead of one-by-one. Get rid of accessors that don't help anyone. On the mapreduce side, serialization is now handled using json (jackson is a dependency anyway) instead of the hand-rolled regex we used before. Also do away with custom InputSplit and FileFormat, instead using Text and NLineInputFormat. On the local mode side, combine 1 client and N clients into the same implementation. That implementation now uses an ExecutorService, so we can later decouple number of client workers from number of client tasks. Finally, drop a bunch of confusing local state, instead use the new TestOptions POJO as a parameter to static methods.

        Attachments

        1. HBASE-10592.00.patch
          52 kB
          Nick Dimiduk
        2. HBASE-10592.00-0.96.patch
          52 kB
          Nick Dimiduk
        3. HBASE-10592.00-0.98.patch
          52 kB
          Nick Dimiduk
        4. HBASE-10592.01.patch
          53 kB
          Nick Dimiduk
        5. HBASE-10592.01-0.96.patch
          53 kB
          Nick Dimiduk
        6. HBASE-10592.01-0.98.patch
          53 kB
          Nick Dimiduk

          Issue Links

            Activity

              People

              • Assignee:
                ndimiduk Nick Dimiduk
                Reporter:
                ndimiduk Nick Dimiduk
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: