Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4960

Remove unnecessary table/file formats from analytic fn test matrix

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • Impala 2.8.0
    • None
    • Infrastructure

    Description

      From dknupp :
      e2e test timing to find slowest tests (strategy = exhaustive)

      ------------------------------------------------------------------------------------------------------------------------
          TIME   TABLE FORMAT                Slowest Exhaustive Parallel Tests
      ------------------------------------------------------------------------------------------------------------------------
      1098.89s   text/none                   data/test_views_compatibility.py::TestViewCompatibility::test_view_compatibility
       833.21s   text/none-unique_database0  data/test_ddl.py::TestDdlStatements::test_alter_tab
       388.71s   seq/def/block               query_test/test_queries.py::TestQueries::test_analytic_fn
       387.68s   rc/gzip/block               query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       382.00s   parquet/none                query_test/test_join_queries.py::TestSemiJoinQueries::test_semi_joins_exhaustiv
       381.82s   parquet/none                query_test/test_join_queries.py::TestSemiJoinQueries::test_semi_joins_exhaustiv
       380.61s   text/bzip/block             query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       371.18s   seq/none                    query_test/test_queries.py::TestQueries::test_analytic_fn
       370.11s   text/none                   query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       369.31s   seq/bzip/record             query_test/test_queries.py::TestQueries::test_analytic_fn
       369.18s   text/lzo/block              query_test/test_queries.py::TestQueries::test_analytic_fn
       366.88s   seq/gzip/block              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       366.27s   seq/def/record              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       365.91s   seq/bzip/record             query_test/test_queries.py::TestQueries::test_analytic_fn
       363.56s   seq/snap/block              query_test/test_queries.py::TestQueries::test_analytic_fn
       359.66s   seq/def/record              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       355.43s   text/gzip/block             query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       353.24s   text/none                   query_test/test_queries.py::TestQueries::test_analytic_fn
       351.96s   seq/def/block               query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       349.25s   text/snap/block             query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       348.43s   text/none                   query_test/test_queries.py::TestQueries::test_analytic_fn
       347.70s   seq/def/block               query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       347.54s   seq/snap/block              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       344.59s   seq/def/record              query_test/test_queries.py::TestQueries::test_analytic_fn
       344.57s   seq/gzip/block              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       344.56s   seq/none                    query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       343.30s   seq/gzip/record             query_test/test_queries.py::TestQueries::test_analytic_fn
       342.96s   rc/gzip/block               query_test/test_queries.py::TestQueries::test_analytic_fn
       341.99s   seq/bzip/block              query_test/test_queries.py::TestQueries::test_analytic_fn
       341.57s   seq/def/record              query_test/test_queries.py::TestQueries::test_analytic_fn
       341.31s   seq/bzip/block              query_test/test_queries.py::TestQueries::test_analytic_fn
       340.44s   seq/none                    query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       340.40s   text/snap/block             query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       337.42s   text/none                   query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       336.99s   seq/snap/block              query_test/test_queries.py::TestQueries::test_analytic_fn
       336.35s   seq/gzip/record             query_test/test_queries.py::TestQueries::test_analytic_fn
       333.26s   seq/gzip/block              query_test/test_queries.py::TestQueries::test_analytic_fn
       332.55s   text/lzo/block              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       332.35s   seq/snap/record             query_test/test_queries.py::TestQueries::test_analytic_fn
       330.21s   text/lzo/block              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       329.77s   seq/bzip/block              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       329.38s   seq/gzip/record             query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       325.61s   text/lzo/block              query_test/test_queries.py::TestQueries::test_analytic_fn
       325.36s   seq/bzip/record             query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       325.04s   seq/bzip/block              query_test/test_queries.py::TestHdfsQueries::test_analytic_fn
       324.84s   seq/gzip/block              query_test/test_queries.py::TestQueries::test_analytic_fn
       321.08s   text/bzip/block             query_test/test_queries.py::TestQueries::test_analytic_fn
       319.84s   text/gzip/block             query_test/test_queries.py::TestQueries::test_analytic_fn
       318.10s   rc/none                     query_test/test_queries.py::TestQueries::test_analytic_fn
       316.15s   rc/def/block                query_test/test_queries.py::TestQueries::test_analytic_fn
      

      This measures CPU time rather then test response time, but given how much analytic functions dominate this list it seems fair to consider removing some unnecessary table/file formats.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mjacobs Matthew Jacobs
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: