Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4920

pytest metadata for custom cluster tests being put in wrong path

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      Pytest emits a log and junit-style xml about tests executed, skipped, failed, etc. It's sometimes useful to check these artifacts after a run-all-tests.sh run, because the stdout/stderr console is so cluttered, or maybe you forgot to tee.

      There is a bug in which the custom cluster test pytest metadata are ending up in the tests directory, not in the logs directory. The bug is escaped quotes in a shell script being taken literally. The logs are typically going to be in $IMPALA_HOME/tests/"/$IMPALA_HOME/logs/custom_cluster_tests/results/TEST-impala-custom-cluster.xml" instead.

        Issue Links

          Activity

          Show
          mikesbrown Michael Brown added a comment - https://gerrit.cloudera.org/#/c/5978/
          Hide
          mikesbrown Michael Brown added a comment -
          commit 9e36088c8d2859bfb862aa19fc1c6b68172a195e
          Author: Michael Brown <mikeb@cloudera.com>
          Date:   Mon Feb 13 09:45:36 2017 -0800
          
              IMPALA-4920: custom cluster tests: fix generation of py.test options
          
              This patch fixes a problem in which unnecessary quote characters were
              part of some impala-py.test arguments for running the custom cluster
              tests. The bug was causing py.test metadata files as described by the
              py.test options --junit-xml and --result-log to end up outside the
              intended path for such artifacts. By removing the unnecessary escaped
              quotes, custom cluster py.test metadata artifacts will now be in the
              proper location. I doubly made sure it was safe to remove these quotes
              by adding a space in the RESULTS_DIR. The space isn't expanded but is
              instead properly treated as part of RESULTS_DIR.
          
              Change-Id: If6a56bc6bca826a8b50340e5caea7687441899e6
              Reviewed-on: http://gerrit.cloudera.org:8080/5978
              Reviewed-by: Jim Apple <jbapple-impala@apache.org>
              Tested-by: Impala Public Jenkins
          
          Show
          mikesbrown Michael Brown added a comment - commit 9e36088c8d2859bfb862aa19fc1c6b68172a195e Author: Michael Brown <mikeb@cloudera.com> Date: Mon Feb 13 09:45:36 2017 -0800 IMPALA-4920: custom cluster tests: fix generation of py.test options This patch fixes a problem in which unnecessary quote characters were part of some impala-py.test arguments for running the custom cluster tests. The bug was causing py.test metadata files as described by the py.test options --junit-xml and --result-log to end up outside the intended path for such artifacts. By removing the unnecessary escaped quotes, custom cluster py.test metadata artifacts will now be in the proper location. I doubly made sure it was safe to remove these quotes by adding a space in the RESULTS_DIR. The space isn't expanded but is instead properly treated as part of RESULTS_DIR. Change-Id: If6a56bc6bca826a8b50340e5caea7687441899e6 Reviewed-on: http://gerrit.cloudera.org:8080/5978 Reviewed-by: Jim Apple <jbapple-impala@apache.org> Tested-by: Impala Public Jenkins
          Hide
          laszlog Laszlo Gaal added a comment -

          Michael Brown, thank you for the fix. This bug has also disrupted repeated test runs: once the custom_cluster_tests created the directory named " (quote), subsequent end2end test runs failed immediately on startup, as the Python code collecting the tests from subdirectories freaked out when it encountered this directory name enclosed in....yes, quotes. It stopped instantly with an "unmatched quotes" error.

          Show
          laszlog Laszlo Gaal added a comment - Michael Brown , thank you for the fix. This bug has also disrupted repeated test runs: once the custom_cluster_tests created the directory named " (quote), subsequent end2end test runs failed immediately on startup, as the Python code collecting the tests from subdirectories freaked out when it encountered this directory name enclosed in....yes, quotes. It stopped instantly with an "unmatched quotes" error.

            People

            • Assignee:
              mikesbrown Michael Brown
              Reporter:
              mikesbrown Michael Brown
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development