Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      hive.exec.submit.local.task.via.child

      Determines whether local tasks (typically mapjoin hashtable generation phase) runs in separate JVM ("true", recommended) or not. Avoids the overhead of spawning new JVM, but can lead to out-of-memory issues.
      Show
      hive.exec.submit.local.task.via.child Determines whether local tasks (typically mapjoin hashtable generation phase) runs in separate JVM ("true", recommended) or not. Avoids the overhead of spawning new JVM, but can lead to out-of-memory issues.

      Description

      Did some experiments to see if there's a way to speed up unit tests. TestCliDriver seemed to take a lot of time just spinning up/tearing down JVMs. I was also curious to see if running everything on a ram disk would help.

      Results (I ran tests up to authorization_2):

      • Current setup: 40 minutes
      • Single JVM (not using child JVM to run all queries): 8 minutes
      • Single JVM + ram disk: 7 minutes

      So the ram disk didn't help that much. But running tests in single JVM seems worthwhile doing.

      1. HIVE-7271.7.patch
        36 kB
        Gunther Hagleitner
      2. HIVE-7271.6.patch
        35 kB
        Gunther Hagleitner
      3. HIVE-7271.5.patch
        32 kB
        Gunther Hagleitner
      4. HIVE-7271.4.patch
        32 kB
        Gunther Hagleitner
      5. HIVE-7271.3.patch
        32 kB
        Gunther Hagleitner
      6. HIVE-7271.2.patch
        33 kB
        Gunther Hagleitner
      7. HIVE-7271.1.patch
        10 kB
        Gunther Hagleitner

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Gunther Hagleitner
              Reporter:
              Gunther Hagleitner
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development