Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4221

test_insert_inherit_permission fails with isilon

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.2
    • Fix Version/s: Impala 2.2
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      The test starts an Impala cluster with "--insert_inherit_permissions=true" as an additional commandline parameter.

      It then executes the following sequence of steps:

      self.hdfs_client.chmod("test-warehouse/%s/p1=1/" % TEST_TBL, "777")
      
            # 1. INSERT that creates two new directories gets permissions from parent
            self.execute_query_expect_success(client, "INSERT INTO %s"
                                              " PARTITION(p1=1, p2=2, p3=2) VALUES(1)" % TEST_TBL)
            self._check_partition_perms("p1=1/p2=2/", "777")
            self._check_partition_perms("p1=1/p2=2/p3=2/", "777")
      
            # 2. INSERT that creates one new directory gets permissions from parent
            self.execute_query_expect_success(client, "INSERT INTO %s"
                                              " PARTITION(p1=1, p2=2, p3=3) VALUES(1)" % TEST_TBL)
            self._check_partition_perms("p1=1/p2=2/p3=3/", "777")
      
            # 3. INSERT that creates no new directories keeps standard permissions
            self.hdfs_client.chmod("test-warehouse/%s/p1=1/p2=2" % TEST_TBL, "644")
            self.execute_query_expect_success(client, "INSERT INTO %s"
                                              " PARTITION(p1=1, p2=2, p3=3) VALUES(1)" % TEST_TBL)
            self._check_partition_perms("p1=1/p2=2/", "644")
            self._check_partition_perms("p1=1/p2=2/p3=3/", "777")
      

      However, when the test tries to clean up its state by dropping the test table, the following error is reported:

      beeswax/impala_beeswax.py:347: in wait_for_completion
      >         raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
      E         ImpalaBeeswaxException: ImpalaBeeswaxException:
      E          Query aborted:Error(s) moving partition files. First error (of 1) was: Hdfs op (RENAME hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/_impala_insert_staging/5246910d4fdaa977_f5b66cceda02798c//.5246910d4fdaa977-f5b66cceda02798d_1731215075_dir/p1=1/p2=2/p3=3/5246910d4fdaa977-f5b66cceda02798d_1197264558_data.0. TO hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1/p2=2/p3=3/5246910d4fdaa977-f5b66cceda02798d_1197264558_data.0.) failed, error was: Error(13): Permission denied
      

      p3 does not exist, the following are the permissions of the sub-directories.

      ishaan@viper-desktop:~/Impala/tests$ hdfs dfs -ls hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission
      Found 1 items
      drwxrwxrwx   - ishaan 1514         22 2015-05-11 09:58 hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1
      ishaan@viper-desktop:~/Impala/tests$ hdfs dfs -ls hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1
      Found 1 items
      drw-r--r--   - ishaan 1514         44 2015-05-11 09:58 hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1/p2=2
      ishaan@viper-desktop:~/Impala/tests$ hdfs dfs -ls hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1/p2=2
      

      I believe the problem is in RENAME - I am able to delete the test table location using the hdfs commandline with the same user name as Impala/

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              lv Lars Volker
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: