Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-8352

Enable windowing.q for spark [Spark Branch]

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: Spark
    • Labels:
      None

      Description

      We should enable windowing.q for basic windowing coverage. After checking out the spark branch, we would build:

      $ mvn clean install -DskipTests -Phadoop-2
      $ cd itests/
      $ mvn clean install -DskipTests -Phadoop-2
      

      Then generate the windowing.q.out file:

      $ cd qtest-spark/
      $ mvn test -Dtest=TestSparkCliDriver -Dqfile=windowing.q -Phadoop-2 -Dtest.output.overwrite=true
      

      Compare the output against MapReduce:

      $ diff -y -W 150 ../../ql/src/test/results/clientpositive/spark/windowing.q.out ../../ql/src/test/results/clientpositive/windowing.q.out| less
      

      And if everything looks good, add it to spark.query.files in ./itests/src/test/resources/testconfiguration.properties

      then submit the patch including the .q file

        Attachments

        1. hive-8385.patch
          148 kB
          Jimmy Xiang
        2. HIVE-8352.1-spark.patch
          148 kB
          Jimmy Xiang
        3. HIVE-8352.1-spark.patch
          148 kB
          Brock Noland

          Activity

            People

            • Assignee:
              jxiang Jimmy Xiang
              Reporter:
              brocknoland Brock Noland
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: