Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2518

DROP DATABASE CASCADE does not remove cache directives of tables

    Details

      Description

      Executing a DROP DATABASE statement with the CASCADE option does not clear the cache directives associated with the cached tables that are contained in the dropped database.

      To reproduce:

      impala> create database foo;
      impala> use foo;
      impala>create table t1 (a int) cached in 'testPool' with replication = 8;
      shell> hdfs cacheadmin -listDirectives
      impala> use default;
      impala> drop database foo cascade;
      shell> hdfs cachedmin -listDirectives <-- the output contains the directive associated with the path of table t1
      

      This has been breaking impala-cdh5.5.x-repeated-runs (https://issues.cloudera.org/browse/IMPALA-2510)

        Activity

        Hide
        dhecht Dan Hecht added a comment -

        Dimitris Tsirogiannis, do you know if this is a regression? Any idea why is it just showing up in the tests now?

        Show
        dhecht Dan Hecht added a comment - Dimitris Tsirogiannis , do you know if this is a regression? Any idea why is it just showing up in the tests now?
        Hide
        dtsirogiannis Dimitris Tsirogiannis added a comment -

        Dan Hecht, I don't think it ever worked correctly in this case and we didn't have tests to verify. I believe something changed in the way we run the repeated tests that triggered this. I noticed we no longer restart impala after each iteration. I am not sure how this affects any other cleanup operations performed but the timing of this change seems very suspicious.

        Show
        dtsirogiannis Dimitris Tsirogiannis added a comment - Dan Hecht , I don't think it ever worked correctly in this case and we didn't have tests to verify. I believe something changed in the way we run the repeated tests that triggered this. I noticed we no longer restart impala after each iteration. I am not sure how this affects any other cleanup operations performed but the timing of this change seems very suspicious.
        Hide
        dhecht Dan Hecht added a comment -

        I think you're right. cc: Harrison Sheinblatt

        Show
        dhecht Dan Hecht added a comment - I think you're right. cc: Harrison Sheinblatt
        Hide
        dhecht Dan Hecht added a comment -

        Actually, did the repeated runs change go in yet? It's still here:
        http://gerrit.cloudera.org:8080/#/c/1119/

        Show
        dhecht Dan Hecht added a comment - Actually, did the repeated runs change go in yet? It's still here: http://gerrit.cloudera.org:8080/#/c/1119/
        Hide
        dtsirogiannis Dimitris Tsirogiannis added a comment -

        Anticipating the bug . I probably got confused with the builds that Harrison launched that contained that change.

        Show
        dtsirogiannis Dimitris Tsirogiannis added a comment - Anticipating the bug . I probably got confused with the builds that Harrison launched that contained that change.
        Hide
        dtsirogiannis Dimitris Tsirogiannis added a comment -

        Change-Id: I83ef5a33e06728c2b3f833a0309d9da64dce7b88
        Reviewed-on: http://gerrit.cloudera.org:8080/5815
        Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
        Tested-by: Impala Public Jenkins

        M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
        M fe/src/main/java/org/apache/impala/util/HdfsCachingUtil.java
        M tests/query_test/test_hdfs_caching.py
        3 files changed, 66 insertions, 34 deletions

        Approvals:
        Impala Public Jenkins: Verified
        Dimitris Tsirogiannis: Looks good to me, approved

        Show
        dtsirogiannis Dimitris Tsirogiannis added a comment - Change-Id: I83ef5a33e06728c2b3f833a0309d9da64dce7b88 Reviewed-on: http://gerrit.cloudera.org:8080/5815 Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com> Tested-by: Impala Public Jenkins — M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/HdfsCachingUtil.java M tests/query_test/test_hdfs_caching.py 3 files changed, 66 insertions , 34 deletions Approvals: Impala Public Jenkins: Verified Dimitris Tsirogiannis: Looks good to me, approved
        Hide
        jbapple Jim Apple added a comment -

        Are you sure this was resolved by 2.8?

        https://github.com/apache/incubator-impala/tree/2.8.0

        Show
        jbapple Jim Apple added a comment - Are you sure this was resolved by 2.8? https://github.com/apache/incubator-impala/tree/2.8.0

          People

          • Assignee:
            dtsirogiannis Dimitris Tsirogiannis
            Reporter:
            dtsirogiannis Dimitris Tsirogiannis
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development