[IMPALA-4221] test_insert_inherit_permission fails with isilon

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.2
• Fix Version/s: Impala 2.2
• Component/s: Infrastructure
• Assignee: Unassigned
• Reporter: Lars Volker
• Created: 28/Sep/16 21:10
• Updated: 28/Sep/16 21:11
• Resolved: 28/Sep/16 21:10

Description

The test starts an Impala cluster with "--insert_inherit_permissions=true" as an additional commandline parameter.

It then executes the following sequence of steps:

self.hdfs_client.chmod("test-warehouse/%s/p1=1/" % TEST_TBL, "777")

      # 1. INSERT that creates two new directories gets permissions from parent
      self.execute_query_expect_success(client, "INSERT INTO %s"
                                        " PARTITION(p1=1, p2=2, p3=2) VALUES(1)" % TEST_TBL)
      self._check_partition_perms("p1=1/p2=2/", "777")
      self._check_partition_perms("p1=1/p2=2/p3=2/", "777")

      # 2. INSERT that creates one new directory gets permissions from parent
      self.execute_query_expect_success(client, "INSERT INTO %s"
                                        " PARTITION(p1=1, p2=2, p3=3) VALUES(1)" % TEST_TBL)
      self._check_partition_perms("p1=1/p2=2/p3=3/", "777")

      # 3. INSERT that creates no new directories keeps standard permissions
      self.hdfs_client.chmod("test-warehouse/%s/p1=1/p2=2" % TEST_TBL, "644")
      self.execute_query_expect_success(client, "INSERT INTO %s"
                                        " PARTITION(p1=1, p2=2, p3=3) VALUES(1)" % TEST_TBL)
      self._check_partition_perms("p1=1/p2=2/", "644")
      self._check_partition_perms("p1=1/p2=2/p3=3/", "777")

However, when the test tries to clean up its state by dropping the test table, the following error is reported:

beeswax/impala_beeswax.py:347: in wait_for_completion
>         raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E         ImpalaBeeswaxException: ImpalaBeeswaxException:
E          Query aborted:Error(s) moving partition files. First error (of 1) was: Hdfs op (RENAME hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/_impala_insert_staging/5246910d4fdaa977_f5b66cceda02798c//.5246910d4fdaa977-f5b66cceda02798d_1731215075_dir/p1=1/p2=2/p3=3/5246910d4fdaa977-f5b66cceda02798d_1197264558_data.0. TO hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1/p2=2/p3=3/5246910d4fdaa977-f5b66cceda02798d_1197264558_data.0.) failed, error was: Error(13): Permission denied

p3 does not exist, the following are the permissions of the sub-directories.

ishaan@viper-desktop:~/Impala/tests$ hdfs dfs -ls hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission
Found 1 items
drwxrwxrwx   - ishaan 1514         22 2015-05-11 09:58 hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1
ishaan@viper-desktop:~/Impala/tests$ hdfs dfs -ls hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1
Found 1 items
drw-r--r--   - ishaan 1514         44 2015-05-11 09:58 hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1/p2=2
ishaan@viper-desktop:~/Impala/tests$ hdfs dfs -ls hdfs://10.20.166.39:8020/test-warehouse/insert_inherit_permission/p1=1/p2=2

I believe the problem is in RENAME - I am able to delete the test table location using the hdfs commandline with the same user name as Impala/