Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13716

Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: test
    • Labels:
      None

      Description

      To make our tests robust against timing problems and eventual consistent stores, we need to do more spin & wait for state.

      We have some code in GenericTestUtils.waitFor to await a condition being met, but the predicate it calls doesn't throw exceptions, there's no way for a probe to throw an exception, and all you get is the eventual "timed out" message.

      We can do better, and in closure-ready languages (scala & scalatest, groovy and some slider code) we've examples to follow. Some of that work has been reimplemented slightly in S3ATestUtils.eventually

      I propose adding a class in the test tree, Eventually to be a successor/replacement for these.

      1. has an eventually/waitfor operation taking a predicate that throws an exception
      2. has an "evaluate" exception which tries to evaluate an answer until the operation stops raising an exception. (again, from scalatest)
      3. plugin backoff strategies (from Scalatest; lets you do exponential as well as linear)
      4. option of adding a special handler to generate the failure exception (e.g. run more detailed diagnostics for the exception text, etc).
      5. be Java 8 lambda expression friendly
      6. be testable and tested itself.

        Attachments

        1. HADOOP-13716-branch-2-004.patch
          31 kB
          Steve Loughran
        2. HADOOP-13716-006.patch
          40 kB
          Steve Loughran
        3. HADOOP-13716-005.patch
          40 kB
          Steve Loughran
        4. HADOOP-13716-003.patch
          35 kB
          Steve Loughran
        5. HADOOP-13716-002.patch
          30 kB
          Steve Loughran
        6. HADOOP-13716-001.patch
          18 kB
          Steve Loughran

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: