Uploaded image for project: 'Crunch'
  1. Crunch
  2. CRUNCH-542

Wider tolerance for flaky scrunch PCollectionTest

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.11.0, 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: Scrunch
    • Labels:
      None

      Description

      One of the Scrunch tests uses an unseeded version of the sample() function that verifies that it works correctly by ensuring that an actual sampling of elements is within ~ 3 standard deviations of the expected value. Given this, we expect the test to fail about once every 370 times it is run, or once a year if the tests were run every day.

      My issue is that we test about a dozen versions of Crunch automatically in Jenkins every day, and so I'm having this test fail on at least some version about once every month. I'd like to bump the control limit up to a little over 5 standard deviations so that the test fails around once every millennium and/or get rid of the test entirely and only rely on the seeded versions of the test.

        Attachments

        1. CRUNCH-542b.patch
          2 kB
          Josh Wills
        2. CRUNCH-542.patch
          1 kB
          Josh Wills

          Activity

            People

            • Assignee:
              jwills Josh Wills
              Reporter:
              jwills Josh Wills
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: