HBase
  1. HBase
  2. HBASE-6891 Hadoop 2 unit test failures
  3. HBASE-8335

Investigate unit test performance regressions between hadoop1 and hadoop2

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      HBASE-7606 and HBASE-7605 showed 2-3x performance regressions when comparing running on top of hadoop1 (1.1.2) and hadoop2 (2.0.2-alpha).

      We need to figure out why.

        Activity

        Hide
        Nicolas Liochon added a comment -

        I wonder if the increased memory consumption observed in hadoop 2 when we use read short circuit is not one of the causes?

        Show
        Nicolas Liochon added a comment - I wonder if the increased memory consumption observed in hadoop 2 when we use read short circuit is not one of the causes?
        Hide
        Jonathan Hsieh added a comment - - edited

        I suspected something simlar when I figured out HBASE-7636. I did a run with shortcircuit reads disabled (-Dhbase.tests.use.shortciruit.reads=false) and experienced the same slowdown, so I don't think that is it.

        (Had typo in 'short circuit' in comment. )

        Show
        Jonathan Hsieh added a comment - - edited I suspected something simlar when I figured out HBASE-7636 . I did a run with shortcircuit reads disabled (-Dhbase.tests.use.shortciruit.reads=false) and experienced the same slowdown, so I don't think that is it. (Had typo in 'short circuit' in comment. )
        Hide
        stack added a comment -

        Resolving as invalid.

        hbase trunk on hadoop2 takes 1.30 on hadoop1 it taks 2:00. Both run 1600 tests.

        https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4330/consoleText
        https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK-on-Hadoop-2.0.0/648/consoleText

        Looking over more history, the variability is high. Tests run on different machines and at different times of the day.

        Resovling as invalid for now. Lets reopen if we get more to hang on.

        Show
        stack added a comment - Resolving as invalid. hbase trunk on hadoop2 takes 1.30 on hadoop1 it taks 2:00. Both run 1600 tests. https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4330/consoleText https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK-on-Hadoop-2.0.0/648/consoleText Looking over more history, the variability is high. Tests run on different machines and at different times of the day. Resovling as invalid for now. Lets reopen if we get more to hang on.
        Hide
        stack added a comment -

        Of note, HBASE-8957 turned off SSR for unit tests. Perhaps that made a difference?

        Show
        stack added a comment - Of note, HBASE-8957 turned off SSR for unit tests. Perhaps that made a difference?

          People

          • Assignee:
            Unassigned
            Reporter:
            Jonathan Hsieh
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development