Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-2771

Tests in nightly builds: speed up long runners

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.16
    • Fix Version/s: 1.19
    • Component/s: build, test
    • Labels:
      None

      Description

      The Nutch tests run by "ant test" or "ant nightly") take rather long to run. Although all tests are implemented as JUnit tests, some tests are more integration tests, eg. launching a Jetty web server and fetching documents from it. It's nice to have also higher level tests, and they are expected to long runner than a simple unit test. However, some of the test classes take really long to run (times taken from https://builds.apache.org/job/Nutch-trunk/3663/consoleText):

          [junit] Running org.apache.nutch.segment.TestSegmentMergerCrawlDatums
          [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 133.898 sec
      
          [junit] Running org.apache.nutch.segment.TestSegmentMerger
          [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 101.026 sec
      
          [junit] Running org.apache.nutch.crawl.TestGenerator
          [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.03 sec
      
          [junit] Running org.apache.nutch.fetcher.TestFetcher
          [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.805 sec
      
          [junit] Running org.apache.nutch.urlfilter.fast.TestFastURLFilter
          [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.36 sec
      
          [junit] Running org.apache.nutch.parse.tika.TestPdfParser
          [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.974 sec
      
          [junit] Running org.apache.nutch.parse.tika.TestImageMetadata
          [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.113 sec
      
          [junit] Running org.apache.nutch.parse.feed.TestFeedParser
          [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.369 sec
      
          [junit] Running org.apache.nutch.crawl.TestInjector
          [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.15 sec
      

      We could try to speed up at least some of these long-running tests.

        Attachments

          Activity

            People

            • Assignee:
              snagel Sebastian Nagel
              Reporter:
              snagel Sebastian Nagel
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: