Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4054

Remove serial test workarounds for IMPALA-2479.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.5.0, Impala 2.6.0, Impala 2.7.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Infrastructure
    • Labels:

      Description

      The tests in test_runtime_filters.py are currently marked as serial due to IMPALA-3367 which has been fixed. These tests take around 10 minutes when serially.

      Other tests have also been marked as serial tow workaround IMPALA-2479.

      Let's remove all workarounds and run the tests in parallel again.

        Activity

        Hide
        alex.behm Alexander Behm added a comment -

        Actually it looks like the root cause of IMPALA-3367 is not fixed (and maybe never will be?). So running these tests in parallel might just be wishful thinking.

        Show
        alex.behm Alexander Behm added a comment - Actually it looks like the root cause of IMPALA-3367 is not fixed (and maybe never will be?). So running these tests in parallel might just be wishful thinking.
        Hide
        henryr Henry Robinson added a comment -

        Lars Volker - do you know if IMPALA-2479 (specifically this comment) has been or can be addressed by your scheduler changes?

        Show
        henryr Henry Robinson added a comment - Lars Volker - do you know if IMPALA-2479 (specifically this comment ) has been or can be addressed by your scheduler changes?
        Hide
        lv Lars Volker added a comment -

        Henry Robinson - yes, IMPALA-2479 should not happen after this change. The scheduler now takes a copy of the current backend configuration and does round-robin based on that. Concurrent queries and concurrent updates to the scheduler's backend configuration should not interfere with this.

        Show
        lv Lars Volker added a comment - Henry Robinson - yes, IMPALA-2479 should not happen after this change . The scheduler now takes a copy of the current backend configuration and does round-robin based on that. Concurrent queries and concurrent updates to the scheduler's backend configuration should not interfere with this.
        Hide
        alex.behm Alexander Behm added a comment -

        Thanks for confirming, Lars Volker! FWIW I could also not reproduce the issue by running test_runtime_filters.py in a loop with parallelism.

        Show
        alex.behm Alexander Behm added a comment - Thanks for confirming, Lars Volker ! FWIW I could also not reproduce the issue by running test_runtime_filters.py in a loop with parallelism.
        Hide
        alex.behm Alexander Behm added a comment -

        commit 16f1c8d8de0ad33e52524fd4761e7728852c27cc
        Author: Alex Behm <alex.behm@cloudera.com>
        Date: Wed Aug 31 18:53:32 2016 -0700

        IMPALA-4054: Remove serial test workarounds for IMPALA-2479.

        The underlying issue IMPALA-2479 has been fixed, so it
        should be safe to execute these tests in parallel again:

        • test_runtime_filters.py (all tests)
        • test_scanners.py::TestParquet::test_multiple_blocks
        • test_scanners.py::testParquet::test_multiple_blocks_one_row_group

        Testing: Ran the tests locally in a loop. Did a private core/hdfs run.

        Change-Id: I8f046e67eb1de1c6ff87980f906870ec9816f551
        Reviewed-on: http://gerrit.cloudera.org:8080/4291
        Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
        Reviewed-by: Lars Volker <lv@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        alex.behm Alexander Behm added a comment - commit 16f1c8d8de0ad33e52524fd4761e7728852c27cc Author: Alex Behm <alex.behm@cloudera.com> Date: Wed Aug 31 18:53:32 2016 -0700 IMPALA-4054 : Remove serial test workarounds for IMPALA-2479 . The underlying issue IMPALA-2479 has been fixed, so it should be safe to execute these tests in parallel again: test_runtime_filters.py (all tests) test_scanners.py::TestParquet::test_multiple_blocks test_scanners.py::testParquet::test_multiple_blocks_one_row_group Testing: Ran the tests locally in a loop. Did a private core/hdfs run. Change-Id: I8f046e67eb1de1c6ff87980f906870ec9816f551 Reviewed-on: http://gerrit.cloudera.org:8080/4291 Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com> Reviewed-by: Lars Volker <lv@cloudera.com> Tested-by: Internal Jenkins

          People

          • Assignee:
            alex.behm Alexander Behm
            Reporter:
            alex.behm Alexander Behm
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development