Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5628

Create target for 10 minute patch test build

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: test
    • Labels:
      None

      Description

      I think we should create an ant target that performs a smoke test on the patched system to enable developers to have faster turn around time on developing patches than the 3 hour unit tests that we currently have.

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          Closing this as stale.

          test-patch.sh is now very smart about what it does. The problem is now that a lot of the tests are out of control in how long they run (hadoop-hdfs, for example, is usually 2 hours on its own...)

          Show
          aw Allen Wittenauer added a comment - Closing this as stale. test-patch.sh is now very smart about what it does. The problem is now that a lot of the tests are out of control in how long they run (hadoop-hdfs, for example, is usually 2 hours on its own...)
          Hide
          aw Allen Wittenauer added a comment -

          ping!

          I'm tempted to close this one for a variety of reasons:

          • we no longer use ant
          • it's possible to just do unit tests on a sub project
          • it blocks a closed jira...

          On the flip side, doing a full test still takes forever. But that might be a different jira than this one.

          Show
          aw Allen Wittenauer added a comment - ping! I'm tempted to close this one for a variety of reasons: we no longer use ant it's possible to just do unit tests on a sub project it blocks a closed jira... On the flip side, doing a full test still takes forever. But that might be a different jira than this one.
          Hide
          cos Konstantin Boudnik added a comment -

          In the current state of the project the 100% of Common's tests are running under 4 minutes time. I think we can safely include all current tests into the 10 minutes list and update it later as needed.

          Show
          cos Konstantin Boudnik added a comment - In the current state of the project the 100% of Common's tests are running under 4 minutes time. I think we can safely include all current tests into the 10 minutes list and update it later as needed.
          Hide
          tomwhite Tom White added a comment -

          The following ant argument will suppress all deprecated warnings, while keeping the other Xlint warnings:

          -Djavac.args="-Xlint -Xlint:-deprecation -Xmaxwarns 1000"
          

          I don't think you can suppress deprecated warnings on a per-package basis. While we have so many deprecated classes in the old mapred package, I think that blanket suppression is the most pragmatic approach for this new patch target.

          I think this will also require we categorize our unit tests into fast and slow.

          Or perhaps into unit test vs. functional or integration test.

          Show
          tomwhite Tom White added a comment - The following ant argument will suppress all deprecated warnings, while keeping the other Xlint warnings: -Djavac.args="-Xlint -Xlint:-deprecation -Xmaxwarns 1000" I don't think you can suppress deprecated warnings on a per-package basis. While we have so many deprecated classes in the old mapred package, I think that blanket suppression is the most pragmatic approach for this new patch target. I think this will also require we categorize our unit tests into fast and slow. Or perhaps into unit test vs. functional or integration test.
          Hide
          jothipn Jothi Padmanabhan added a comment -

          The current trunk has 1000 deprecated warnings from the mapred package. All of these warnings are because of deprecation of old-api related classes in favor of the new-api.
          Is there a way by which we can do a blanket suppression of deprecated warnings, possibly from javac command line? What is a good way to handle this?

          Show
          jothipn Jothi Padmanabhan added a comment - The current trunk has 1000 deprecated warnings from the mapred package. All of these warnings are because of deprecation of old-api related classes in favor of the new-api. Is there a way by which we can do a blanket suppression of deprecated warnings, possibly from javac command line? What is a good way to handle this?
          Hide
          nidaley Nigel Daley added a comment -

          Ya, all the warnings need to go to zero (like javadoc). +1 for that, as long as we evaluate them carefully before suppressing them.

          I think this will also require we categorize our unit tests into fast and slow. This will require TestNG or Junit 4.5 (my vote is for TestNG). See HADOOP-4901. +1 for that too.

          Show
          nidaley Nigel Daley added a comment - Ya, all the warnings need to go to zero (like javadoc). +1 for that, as long as we evaluate them carefully before suppressing them. I think this will also require we categorize our unit tests into fast and slow. This will require TestNG or Junit 4.5 (my vote is for TestNG). See HADOOP-4901 . +1 for that too.
          Hide
          owen.omalley Owen O'Malley added a comment -

          I propose that we take the test-patch target name and have it:
          1. Run javac, javadoc, findbugs, and release audit tool and report any new violations. Since this is for developers, we shouldn't force them to make a patch and make a clean work directory, but assume the patch has been applied to the work directory. That means we need to either clean up or suppress all of the warnings, especially from javac and findbugs.
          2. Run some unit tests that aim for breadth of coverage.

          Try and keep the whole thing under 10 minutes so that developers that are trying to test their patches aren't spending hours in the edit-test patch that is currently happening.

          The other tests should be kept and run as often as possible.

          Show
          owen.omalley Owen O'Malley added a comment - I propose that we take the test-patch target name and have it: 1. Run javac, javadoc, findbugs, and release audit tool and report any new violations. Since this is for developers, we shouldn't force them to make a patch and make a clean work directory, but assume the patch has been applied to the work directory. That means we need to either clean up or suppress all of the warnings, especially from javac and findbugs. 2. Run some unit tests that aim for breadth of coverage. Try and keep the whole thing under 10 minutes so that developers that are trying to test their patches aren't spending hours in the edit-test patch that is currently happening. The other tests should be kept and run as often as possible.

            People

            • Assignee:
              Unassigned
              Reporter:
              owen.omalley Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development