Hadoop Common
  1. Hadoop Common
  2. HADOOP-6810

Extract a subset of tests for smoke (DOA) validation

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Later
    • Affects Version/s: 0.21.0
    • Fix Version/s: None
    • Component/s: test
    • Labels:
      None

      Description

      Currently, there's no way to quickly find out if a new build of Hadoop is reasonably viable and can be used for future development/testing cycle.

      Similarly to 'commit' testing it'd be great to have a wider subset of existing tests to be executed inside of 30 minutes. The main goal of such a subset is to have a reasonable guarantees that different parts of Hadoop's functionality aren't broken and the system is not DOA.

      Full test run takes about 3+ hours and it is a too long wait time in many cases.

      1. smoke-tests.1.patch
        7 kB
        Konstantin Boudnik
      2. smoke-tests.patch
        5 kB
        Konstantin Boudnik
      3. smoke-tests.patch
        6 kB
        Konstantin Boudnik
      4. smoke-tests.patch
        6 kB
        Konstantin Boudnik
      5. smoke-tests.patch
        5 kB
        Konstantin Boudnik
      6. smoke-tests.patch
        2 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Hide
          Konstantin Boudnik added a comment -

          I'd like to close it for now as 'Later' until Common has enough tests to separate a smoke suite. Right not it doesn't make much sense.

          Show
          Konstantin Boudnik added a comment - I'd like to close it for now as 'Later' until Common has enough tests to separate a smoke suite. Right not it doesn't make much sense.
          Hide
          Jakob Homan added a comment -

          Updated patch-on-the-patch looks good. Thanks. +1 for 20S.

          Show
          Jakob Homan added a comment - Updated patch-on-the-patch looks good. Thanks. +1 for 20S.
          Hide
          Konstantin Boudnik added a comment -

          Addressing Jacob offline comment about name of a variable. This patch should go on top of smoke-test.patch

          Show
          Konstantin Boudnik added a comment - Addressing Jacob offline comment about name of a variable. This patch should go on top of smoke-test.patch
          Hide
          Konstantin Boudnik added a comment -

          This patch addresses the issue with test duplication entries. Please review.

          Show
          Konstantin Boudnik added a comment - This patch addresses the issue with test duplication entries. Please review.
          Hide
          Jakob Homan added a comment -

          Updated patch looks good. I notice line 12 is indented a couple characters too far. I also just noticed that the smoke test list contains duplicates of from the commit list, meaning those tests will be run twice even with this change.

          Show
          Jakob Homan added a comment - Updated patch looks good. I notice line 12 is indented a couple characters too far. I also just noticed that the smoke test list contains duplicates of from the commit list, meaning those tests will be run twice even with this change.
          Hide
          Konstantin Boudnik added a comment -

          This patch goes on top of the previous one in order to have a separate target which runs test-core test set excluding that of commit and smoke tests. test-core behavior left intact and new target is added for the purpose of eliminating test run dups.
          A separate combined patch for HDFS/MR trunks is coming

          Show
          Konstantin Boudnik added a comment - This patch goes on top of the previous one in order to have a separate target which runs test-core test set excluding that of commit and smoke tests. test-core behavior left intact and new target is added for the purpose of eliminating test run dups. A separate combined patch for HDFS/MR trunks is coming
          Hide
          Jakob Homan added a comment -

          +1 for Y20.

          Show
          Jakob Homan added a comment - +1 for Y20.
          Hide
          Konstantin Boudnik added a comment -

          Addressing Jacob's comment: core-test behavior is unchanged in this version. However, the overlap of running test-smoke followed by core-test will be equal to that of test-smoke tests.

          Show
          Konstantin Boudnik added a comment - Addressing Jacob's comment: core-test behavior is unchanged in this version. However, the overlap of running test-smoke followed by core-test will be equal to that of test-smoke tests.
          Hide
          Jakob Homan added a comment -

          I tend to agree with Boris. With the current patch, if I run test-core, the behavior changes so that those tests in commit and smoke will not be run, correct? That would be a big change from what users have grown used to. I had assumed that commit tests are a subset of smoke tests are a subset of core-tests are a subset of test. I had made previous comments that if you run, say ant commit-tests test, you would end up running the members of commit-tests twice, which was suboptimal. But I'm not sure the other side of that - three separate disjoint sets - is also optimal.

          Except for this behavior change, the patch looks good.

          Show
          Jakob Homan added a comment - I tend to agree with Boris. With the current patch, if I run test-core, the behavior changes so that those tests in commit and smoke will not be run, correct? That would be a big change from what users have grown used to. I had assumed that commit tests are a subset of smoke tests are a subset of core-tests are a subset of test. I had made previous comments that if you run, say ant commit-tests test, you would end up running the members of commit-tests twice, which was suboptimal. But I'm not sure the other side of that - three separate disjoint sets - is also optimal. Except for this behavior change, the patch looks good.
          Hide
          Konstantin Boudnik added a comment -

          I think I have made it all confusing by setting 0.20 as the target. The attached patch is for internal Yahoo 20 release and trunk patch will be add here shortly. Trunk versions for HDFS/MR will follow shortly as well.

          Show
          Konstantin Boudnik added a comment - I think I have made it all confusing by setting 0.20 as the target. The attached patch is for internal Yahoo 20 release and trunk patch will be add here shortly. Trunk versions for HDFS/MR will follow shortly as well.
          Hide
          Konstantin Boudnik added a comment -

          Have spoken with Boris offline and he expresses the concern that excluding commit and smoke tests from test-core target. While this is a valid concern because the set of core tests is changing by the patch without it test-core execution will increase by about 40 minutes (13 minutes from commit tests, which we already have in place, and about 30 minutes from smokes).

          Any thoughts?

          Show
          Konstantin Boudnik added a comment - Have spoken with Boris offline and he expresses the concern that excluding commit and smoke tests from test-core target. While this is a valid concern because the set of core tests is changing by the patch without it test-core execution will increase by about 40 minutes (13 minutes from commit tests, which we already have in place, and about 30 minutes from smokes). Any thoughts?
          Hide
          Konstantin Boudnik added a comment -

          Adding comments to the test list to separate test specific to common from hdfs

          Show
          Konstantin Boudnik added a comment - Adding comments to the test list to separate test specific to common from hdfs
          Hide
          Konstantin Boudnik added a comment -

          Now the list of smoke tests is formed pretty much to its final shape. Ready for a review, I suppose.

          Show
          Konstantin Boudnik added a comment - Now the list of smoke tests is formed pretty much to its final shape. Ready for a review, I suppose.
          Hide
          Konstantin Boudnik added a comment -

          This version of the patch excludes commit and smoke tests from test-core to eliminate the duplication. Instead test-commit and test-smoke are included to the test along with test-core

          This patch is for 0.20.3. As 0.20.3 release might not happen I am including this patch in case someone needs it eventually.

          Show
          Konstantin Boudnik added a comment - This version of the patch excludes commit and smoke tests from test-core to eliminate the duplication. Instead test-commit and test-smoke are included to the test along with test-core This patch is for 0.20.3. As 0.20.3 release might not happen I am including this patch in case someone needs it eventually.
          Hide
          Konstantin Boudnik added a comment -

          To be clean, the above patch is for 0.20.3 branch.

          Show
          Konstantin Boudnik added a comment - To be clean, the above patch is for 0.20.3 branch.
          Hide
          Konstantin Boudnik added a comment -

          Initial patch with preliminary list of tests for MR. Core/HDFS test list to be added.

          Show
          Konstantin Boudnik added a comment - Initial patch with preliminary list of tests for MR. Core/HDFS test list to be added.
          Hide
          Konstantin Boudnik added a comment -

          It is different for 0.20 branch where all tests are mixed together.
          For HDFS/MR we'll need a separate JIRAs.

          Show
          Konstantin Boudnik added a comment - It is different for 0.20 branch where all tests are mixed together. For HDFS/MR we'll need a separate JIRAs.
          Hide
          Jakob Homan added a comment -

          How would this be different than the current commit (run-commit-test) test suite? That suite may need to be updated, it's been a while. The Common tests already take less than 10 minutes to run...

          Show
          Jakob Homan added a comment - How would this be different than the current commit (run-commit-test) test suite? That suite may need to be updated, it's been a while. The Common tests already take less than 10 minutes to run...

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development