HBase
  1. HBase
  2. HBASE-9940

PerformanceEvaluation should have a test with many table options on (Bloom, compression, FAST_DIFF, etc.)

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.96.0, 0.94.13
    • Fix Version/s: None
    • Component/s: Performance, test
    • Labels:
      None

      Activity

      Hide
      Nick Dimiduk added a comment -

      I hadn't checked vs 0.94. Yes, these are useful features that should be back-ported. +1 for that.

      Can't blockSize be effectively set via -D option?

      I'd also like to add a --sampleSize option, at least RandomRead, because it runs a looong time if you're running it against a table of any size. I have this feature in my own branch, which I've been using locally. I have it dumping the times taken by individual test threads so that they can be re-assembled for request latency analysis.

      Show
      Nick Dimiduk added a comment - I hadn't checked vs 0.94. Yes, these are useful features that should be back-ported. +1 for that. Can't blockSize be effectively set via -D option? I'd also like to add a --sampleSize option, at least RandomRead, because it runs a looong time if you're running it against a table of any size. I have this feature in my own branch, which I've been using locally. I have it dumping the times taken by individual test threads so that they can be re-assembled for request latency analysis.
      Hide
      Jean-Marc Spaggiari added a comment -

      Correct. the idea is to add all what's missing. Those was just examples.

      also, --compress and --blockEncoding are not there in 0.94. So they should be backported.

      So list might look like:

      • Add bloom filters options in trunk, 0.96 and 0.94
      • Add blocSize options in trunk, 0.96 and 0.94
      • Add compression options in 0.94
      • Add block encoding options in 0.94

      Do you see any other which might be good to have? We need to make sure they are on all versions to make comparisons easier.

      Show
      Jean-Marc Spaggiari added a comment - Correct. the idea is to add all what's missing. Those was just examples. also, --compress and --blockEncoding are not there in 0.94. So they should be backported. So list might look like: Add bloom filters options in trunk, 0.96 and 0.94 Add blocSize options in trunk, 0.96 and 0.94 Add compression options in 0.94 Add block encoding options in 0.94 Do you see any other which might be good to have? We need to make sure they are on all versions to make comparisons easier.
      Hide
      Matt Corgan added a comment -

      blockSize could be valuable.

      If you leave blockSize set to the default 64KB, then encoded blocks with long keys and small values could really be, say, 32KB in the block cache. If you then double the blockSize setting to 128KB in order to get the encoded size to 64KB, PerformanceEvaluation will show slower random seeks because of the sequential seeking within blocks.

      Show
      Matt Corgan added a comment - blockSize could be valuable. If you leave blockSize set to the default 64KB, then encoded blocks with long keys and small values could really be, say, 32KB in the block cache. If you then double the blockSize setting to 128KB in order to get the encoded size to 64KB, PerformanceEvaluation will show slower random seeks because of the sequential seeking within blocks.
      Hide
      Nick Dimiduk added a comment -

      Can you be more specific as to what additional features you'd like exposed via cli options? The list so far looks like:

      • bloom filters (0.96 enables row-level filters by default. We can expose this)
      • compression (this is already supported via --compress=)
      • block encodings (this is already supported via --blockEncoding=)
      Show
      Nick Dimiduk added a comment - Can you be more specific as to what additional features you'd like exposed via cli options? The list so far looks like: bloom filters (0.96 enables row-level filters by default. We can expose this) compression (this is already supported via --compress= ) block encodings (this is already supported via --blockEncoding= )

        People

        • Assignee:
          Unassigned
          Reporter:
          Jean-Marc Spaggiari
        • Votes:
          0 Vote for this issue
          Watchers:
          3 Start watching this issue

          Dates

          • Created:
            Updated:

            Development