Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5072

test_recover_many_partitions fails on S3

    Details

    • Epic Color:
      ghx-label-5

      Description

      See the following error:

      07:53:27  TestRecoverPartitions.test_recover_many_partitions[exec_option: {'batch_size': 0, 'num_nodes': 0, 'sync_ddl': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none] 
      07:53:27 [gw2] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-umbrella-build-and-test-s3/repos/Impala/bin/../infra/python/env/bin/python
      07:53:27 metadata/test_recover_partitions.py:199: in test_recover_many_partitions
      07:53:27     assert self.has_value(PART_DIR, result.data)
      07:53:27 E   assert <bound method TestRecoverPartitions.has_value of <test_recover_partitions.TestRecoverPartitions object at 0x3ffd890>>('part1\t', ['Total\t-1\t0\t0B\t0B\t\t\t\t'])
      07:53:27 E    +  where <bound method TestRecoverPartitions.has_value of <test_recover_partitions.TestRecoverPartitions object at 0x3ffd890>> = <test_recover_partitions.TestRecoverPartitions object at 0x3ffd890>.has_value
      07:53:27 E    +  and   ['Total\t-1\t0\t0B\t0B\t\t\t\t'] = <tests.beeswax.impala_beeswax.ImpalaBeeswaxResult object at 0x3f2e150>.data
      

        Activity

        Hide
        mikesbrown Michael Brown added a comment -

        Zach Amsden want to mark this resolved? People typically copy/paste the commit message or link to the Gerrit review.

        Show
        mikesbrown Michael Brown added a comment - Zach Amsden want to mark this resolved? People typically copy/paste the commit message or link to the Gerrit review.
        Hide
        zamsden Zach Amsden added a comment -

        IMPALA-5072: Fix test_recover_partitions on S3

        On S3, directory creation is a nop, so we have to actually
        create values to materialize the partitions.

        Was able to merge this against my fork and successfully ran
        the metadata/test_recover_partitions test on S3. Also ran
        locally to verify there is no slowdown for HDFS.

        Change-Id: I30d86e49b000db925a778ff53479b61a075dc88c
        Reviewed-on: http://gerrit.cloudera.org:8080/6408
        Reviewed-by: Michael Brown <mikeb@cloudera.com>
        Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
        Tested-by: Impala Public Jenkins
        Author
        Zach Amsden <zamsden@cloudera.com>
        Mar 15, 2017 2:36 PM
        Committer
        Impala Public Jenkins <impala-public-jenkins@gerrit.cloudera.org>
        Mar 20, 2017 7:29 PM
        Commit
        bccbf9ebdaed9a03d9a5ecf2a39aad044f178ee1
        Parent(s)
        5ecb97598ac0b33d99fdb84a624727952a5b1623
        Change-Id
        I30d86e49b000db925a778ff53479b61a075dc88c

        Show
        zamsden Zach Amsden added a comment - IMPALA-5072 : Fix test_recover_partitions on S3 On S3, directory creation is a nop, so we have to actually create values to materialize the partitions. Was able to merge this against my fork and successfully ran the metadata/test_recover_partitions test on S3. Also ran locally to verify there is no slowdown for HDFS. Change-Id: I30d86e49b000db925a778ff53479b61a075dc88c Reviewed-on: http://gerrit.cloudera.org:8080/6408 Reviewed-by: Michael Brown <mikeb@cloudera.com> Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com> Tested-by: Impala Public Jenkins Author Zach Amsden <zamsden@cloudera.com> Mar 15, 2017 2:36 PM Committer Impala Public Jenkins <impala-public-jenkins@gerrit.cloudera.org> Mar 20, 2017 7:29 PM Commit bccbf9ebdaed9a03d9a5ecf2a39aad044f178ee1 Parent(s) 5ecb97598ac0b33d99fdb84a624727952a5b1623 Change-Id I30d86e49b000db925a778ff53479b61a075dc88c

          People

          • Assignee:
            zamsden Zach Amsden
            Reporter:
            dtsirogiannis Dimitris Tsirogiannis
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development