Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.4, 1.5.0
    • Fix Version/s: 1.4.5, 1.5.1
    • Component/s: test
    • Labels:

      Description

      The MR job that runs functional tests, RunTests, does not report back on which tests passed and failed. The mapper that runs each functional test merely outputs the text from standard output, and the reducer is just an identity reducer. As noted in a comment:

          // don't do anything with the results (yet) a summary would be nice
      

      Update the job to produce results that allow a tester to ascertain the pass/fail status of each test. For example:

      • which tests passed and which failed
      • a count of pass and fail
      • which hosts ran which tests, for drilling down

      While this only applies to 1.4/1.5, it's worth checking if 1.6 has a similar capability and the nature of its reporting.

        Issue Links

          Activity

          Bill Havanki created issue -
          Bill Havanki made changes -
          Field Original Value New Value
          Remote Link This issue links to "Review (Web Link)" [ 14206 ]
          Hide
          Bill Havanki added a comment -

          Review up.

          Show
          Bill Havanki added a comment - Review up.
          Hide
          ASF subversion and git services added a comment -

          Commit 7059c767bfbcd2def59556ed63cdfe443f2d336b in accumulo's branch refs/heads/1.4.5-SNAPSHOT from Bill Havanki
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7059c76 ]

          ACCUMULO-2380 Improve RunTests to produce useful mapper and job output

          When passed the -m option, run.py for functional tests will emit one line on standard
          output for each test that passes or fails. The output is suitable for processing when
          tests are run within a mapreduce job.

          The mapper for RunTests, which runs functional tests, is updated to pass -m to run.py.
          It also inserts the map task attempt ID into the custom output, so that testers can trace
          back any test to the task where it ran, to view its logs for example.

          The mapper also now uses counters to track the number of successes, failures, errors, etc.
          Note that since the job may attempt to re-run tests that error out, the total for these
          counters can exceed the total number of unique tests run.

          Show
          ASF subversion and git services added a comment - Commit 7059c767bfbcd2def59556ed63cdfe443f2d336b in accumulo's branch refs/heads/1.4.5-SNAPSHOT from Bill Havanki [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7059c76 ] ACCUMULO-2380 Improve RunTests to produce useful mapper and job output When passed the -m option, run.py for functional tests will emit one line on standard output for each test that passes or fails. The output is suitable for processing when tests are run within a mapreduce job. The mapper for RunTests, which runs functional tests, is updated to pass -m to run.py. It also inserts the map task attempt ID into the custom output, so that testers can trace back any test to the task where it ran, to view its logs for example. The mapper also now uses counters to track the number of successes, failures, errors, etc. Note that since the job may attempt to re-run tests that error out, the total for these counters can exceed the total number of unique tests run.
          Hide
          ASF subversion and git services added a comment -

          Commit 7059c767bfbcd2def59556ed63cdfe443f2d336b in accumulo's branch refs/heads/1.5.1-SNAPSHOT from Bill Havanki
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7059c76 ]

          ACCUMULO-2380 Improve RunTests to produce useful mapper and job output

          When passed the -m option, run.py for functional tests will emit one line on standard
          output for each test that passes or fails. The output is suitable for processing when
          tests are run within a mapreduce job.

          The mapper for RunTests, which runs functional tests, is updated to pass -m to run.py.
          It also inserts the map task attempt ID into the custom output, so that testers can trace
          back any test to the task where it ran, to view its logs for example.

          The mapper also now uses counters to track the number of successes, failures, errors, etc.
          Note that since the job may attempt to re-run tests that error out, the total for these
          counters can exceed the total number of unique tests run.

          Show
          ASF subversion and git services added a comment - Commit 7059c767bfbcd2def59556ed63cdfe443f2d336b in accumulo's branch refs/heads/1.5.1-SNAPSHOT from Bill Havanki [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7059c76 ] ACCUMULO-2380 Improve RunTests to produce useful mapper and job output When passed the -m option, run.py for functional tests will emit one line on standard output for each test that passes or fails. The output is suitable for processing when tests are run within a mapreduce job. The mapper for RunTests, which runs functional tests, is updated to pass -m to run.py. It also inserts the map task attempt ID into the custom output, so that testers can trace back any test to the task where it ran, to view its logs for example. The mapper also now uses counters to track the number of successes, failures, errors, etc. Note that since the job may attempt to re-run tests that error out, the total for these counters can exceed the total number of unique tests run.
          Hide
          ASF subversion and git services added a comment -

          Commit 7059c767bfbcd2def59556ed63cdfe443f2d336b in accumulo's branch refs/heads/1.6.0-SNAPSHOT from Bill Havanki
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7059c76 ]

          ACCUMULO-2380 Improve RunTests to produce useful mapper and job output

          When passed the -m option, run.py for functional tests will emit one line on standard
          output for each test that passes or fails. The output is suitable for processing when
          tests are run within a mapreduce job.

          The mapper for RunTests, which runs functional tests, is updated to pass -m to run.py.
          It also inserts the map task attempt ID into the custom output, so that testers can trace
          back any test to the task where it ran, to view its logs for example.

          The mapper also now uses counters to track the number of successes, failures, errors, etc.
          Note that since the job may attempt to re-run tests that error out, the total for these
          counters can exceed the total number of unique tests run.

          Show
          ASF subversion and git services added a comment - Commit 7059c767bfbcd2def59556ed63cdfe443f2d336b in accumulo's branch refs/heads/1.6.0-SNAPSHOT from Bill Havanki [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7059c76 ] ACCUMULO-2380 Improve RunTests to produce useful mapper and job output When passed the -m option, run.py for functional tests will emit one line on standard output for each test that passes or fails. The output is suitable for processing when tests are run within a mapreduce job. The mapper for RunTests, which runs functional tests, is updated to pass -m to run.py. It also inserts the map task attempt ID into the custom output, so that testers can trace back any test to the task where it ran, to view its logs for example. The mapper also now uses counters to track the number of successes, failures, errors, etc. Note that since the job may attempt to re-run tests that error out, the total for these counters can exceed the total number of unique tests run.
          Hide
          ASF subversion and git services added a comment -

          Commit 7059c767bfbcd2def59556ed63cdfe443f2d336b in accumulo's branch refs/heads/master from Bill Havanki
          [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7059c76 ]

          ACCUMULO-2380 Improve RunTests to produce useful mapper and job output

          When passed the -m option, run.py for functional tests will emit one line on standard
          output for each test that passes or fails. The output is suitable for processing when
          tests are run within a mapreduce job.

          The mapper for RunTests, which runs functional tests, is updated to pass -m to run.py.
          It also inserts the map task attempt ID into the custom output, so that testers can trace
          back any test to the task where it ran, to view its logs for example.

          The mapper also now uses counters to track the number of successes, failures, errors, etc.
          Note that since the job may attempt to re-run tests that error out, the total for these
          counters can exceed the total number of unique tests run.

          Show
          ASF subversion and git services added a comment - Commit 7059c767bfbcd2def59556ed63cdfe443f2d336b in accumulo's branch refs/heads/master from Bill Havanki [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7059c76 ] ACCUMULO-2380 Improve RunTests to produce useful mapper and job output When passed the -m option, run.py for functional tests will emit one line on standard output for each test that passes or fails. The output is suitable for processing when tests are run within a mapreduce job. The mapper for RunTests, which runs functional tests, is updated to pass -m to run.py. It also inserts the map task attempt ID into the custom output, so that testers can trace back any test to the task where it ran, to view its logs for example. The mapper also now uses counters to track the number of successes, failures, errors, etc. Note that since the job may attempt to re-run tests that error out, the total for these counters can exceed the total number of unique tests run.
          Hide
          Bill Havanki added a comment -

          Committed to 1.4.5-SNAPSHOT and 1.5.1-SNAPSHOT. During commit I noticed an update to RunTests in 1.5 to explicitly use UTF-8 for encoding functional test output, probably added from Findbugs runs, so I added it as well.

          Show
          Bill Havanki added a comment - Committed to 1.4.5-SNAPSHOT and 1.5.1-SNAPSHOT. During commit I noticed an update to RunTests in 1.5 to explicitly use UTF-8 for encoding functional test output, probably added from Findbugs runs, so I added it as well.
          Bill Havanki made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 1.4.5 [ 12324754 ]
          Fix Version/s 1.5.1 [ 12324399 ]
          Resolution Fixed [ 1 ]
          Hide
          Bill Havanki added a comment -

          And ... thanks Eric for the review.

          Show
          Bill Havanki added a comment - And ... thanks Eric for the review.
          Hide
          Josh Elser added a comment -

          to explicitly use UTF-8 for encoding functional test output, probably added from Findbugs runs, so I added it as well.

          Oh thank you so much!

          Show
          Josh Elser added a comment - to explicitly use UTF-8 for encoding functional test output, probably added from Findbugs runs, so I added it as well. Oh thank you so much!
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          1d 5h 34m 1 Bill Havanki 20/Feb/14 21:57

            People

            • Assignee:
              Bill Havanki
              Reporter:
              Bill Havanki
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development