Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1213

TestHadoopExamples smokes: orderering of tests; other improvements

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.8.0
    • Component/s: tests
    • Labels:
      None

      Description

      According to the JDK contract for a Map, the order of iteration is not gauranteed.

      But in TestHadoopExamples.groovy, we define a map and iterate through it.

      The tests should run in the defined order,

      1) both for determinism, as well

      2) so that TeraGen always runs before TeraSort

       static Map examples =
          [
              pi :'5 10',
              wordcount :"$EXAMPLES/text $EXAMPLES_OUT/wordcount",
              teragen :"${terasort_rows} teragen${terasortid}",
              terasort :"teragen${terasortid} terasort${terasortid}",
              teravalidate :"terasort${terasortid} tervalidate${terasortid}",
              ... 
          ];
      

      examples.each

      { k, v -> res[k] = [k.toString(), v.toString()] as Object[]; }
      
      

      While implementing this JIRA, we can also add in some other minor improvements :

      • Parameterization of calculate pi so that it can run fast on VMS
      • Remove "sleep" , as its not in newer versions of YARN ( i think, need to confirm)

      TL;DR, Im proposing a fix to add concreted deterministic JDK-independant ordering to the Map so tests always run in same order + a few minor upgrades to this class while were at it.

        Attachments

        1. BIGTOP-1213.patch
          7 kB
          Dasha Boudnik
        2. BIGTOP-1213.patch
          2 kB
          Dasha Boudnik

          Activity

            People

            • Assignee:
              dasha.boudnik Dasha Boudnik
              Reporter:
              jayunit100 jay vyas
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: