Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5968

load_nested.py fails on secure cluster: no write access to /user on HDFS

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Later
    • Impala 2.10.0
    • Not Applicable
    • Infrastructure
    • None

    Description

      testdata/bin/load_nested.py \
    -s tpch_100_parquet \
    -t tpch_100_nested_parquet \
    --cm-host=vc0332.test \
    -p 34 --use-kerberos --use-ssl
Output: 2017-09-20 14:18:06,174 INFO:db_connection[234]:Creating database tpch_100_nested_parquet
2017-09-20 14:18:06,737 INFO:load_nested[98]:Creating temp orders (chunk 1 of 34)
2017-09-20 14:18:31,420 INFO:load_nested[98]:Creating temp orders (chunk 2 of 34)
2017-09-20 14:18:52,146 INFO:load_nested[98]:Creating temp orders (chunk 3 of 34)
2017-09-20 14:19:12,398 INFO:load_nested[98]:Creating temp orders (chunk 4 of 34)
2017-09-20 14:19:32,109 INFO:load_nested[98]:Creating temp orders (chunk 5 of 34)
2017-09-20 14:19:51,823 INFO:load_nested[98]:Creating temp orders (chunk 6 of 34)
2017-09-20 14:20:12,521 INFO:load_nested[98]:Creating temp orders (chunk 7 of 34)
2017-09-20 14:20:33,220 INFO:load_nested[98]:Creating temp orders (chunk 8 of 34)
2017-09-20 14:20:53,417 INFO:load_nested[98]:Creating temp orders (chunk 9 of 34)
2017-09-20 14:21:13,143 INFO:load_nested[98]:Creating temp orders (chunk 10 of 34)
2017-09-20 14:21:34,361 INFO:load_nested[98]:Creating temp orders (chunk 11 of 34)
2017-09-20 14:21:53,048 INFO:load_nested[98]:Creating temp orders (chunk 12 of 34)
2017-09-20 14:22:13,743 INFO:load_nested[98]:Creating temp orders (chunk 13 of 34)
2017-09-20 14:22:34,439 INFO:load_nested[98]:Creating temp orders (chunk 14 of 34)
2017-09-20 14:22:55,160 INFO:load_nested[98]:Creating temp orders (chunk 15 of 34)
2017-09-20 14:23:15,868 INFO:load_nested[98]:Creating temp orders (chunk 16 of 34)
2017-09-20 14:23:37,075 INFO:load_nested[98]:Creating temp orders (chunk 17 of 34)
2017-09-20 14:23:57,260 INFO:load_nested[98]:Creating temp orders (chunk 18 of 34)
2017-09-20 14:24:18,967 INFO:load_nested[98]:Creating temp orders (chunk 19 of 34)
2017-09-20 14:24:39,680 INFO:load_nested[98]:Creating temp orders (chunk 20 of 34)
2017-09-20 14:24:59,364 INFO:load_nested[98]:Creating temp orders (chunk 21 of 34)
2017-09-20 14:25:20,556 INFO:load_nested[98]:Creating temp orders (chunk 22 of 34)
2017-09-20 14:25:40,240 INFO:load_nested[98]:Creating temp orders (chunk 23 of 34)
2017-09-20 14:26:00,949 INFO:load_nested[98]:Creating temp orders (chunk 24 of 34)
2017-09-20 14:26:21,146 INFO:load_nested[98]:Creating temp orders (chunk 25 of 34)
2017-09-20 14:26:41,841 INFO:load_nested[98]:Creating temp orders (chunk 26 of 34)
2017-09-20 14:27:01,530 INFO:load_nested[98]:Creating temp orders (chunk 27 of 34)
2017-09-20 14:27:21,219 INFO:load_nested[98]:Creating temp orders (chunk 28 of 34)
2017-09-20 14:27:42,921 INFO:load_nested[98]:Creating temp orders (chunk 29 of 34)
2017-09-20 14:28:04,617 INFO:load_nested[98]:Creating temp orders (chunk 30 of 34)
2017-09-20 14:28:25,808 INFO:load_nested[98]:Creating temp orders (chunk 31 of 34)
2017-09-20 14:28:48,031 INFO:load_nested[98]:Creating temp orders (chunk 32 of 34)
2017-09-20 14:29:08,213 INFO:load_nested[98]:Creating temp orders (chunk 33 of 34)
2017-09-20 14:29:29,918 INFO:load_nested[98]:Creating temp orders (chunk 34 of 34)
2017-09-20 14:29:50,610 INFO:load_nested[128]:Creating temp customers (chunk 1 of 34)
2017-09-20 14:30:18,747 INFO:load_nested[128]:Creating temp customers (chunk 2 of 34)
2017-09-20 14:30:44,058 INFO:load_nested[128]:Creating temp customers (chunk 3 of 34)
2017-09-20 14:31:07,818 INFO:load_nested[128]:Creating temp customers (chunk 4 of 34)
2017-09-20 14:31:31,565 INFO:load_nested[128]:Creating temp customers (chunk 5 of 34)
2017-09-20 14:31:54,799 INFO:load_nested[128]:Creating temp customers (chunk 6 of 34)
2017-09-20 14:32:18,529 INFO:load_nested[128]:Creating temp customers (chunk 7 of 34)
2017-09-20 14:32:41,762 INFO:load_nested[128]:Creating temp customers (chunk 8 of 34)
2017-09-20 14:33:04,974 INFO:load_nested[128]:Creating temp customers (chunk 9 of 34)
2017-09-20 14:33:28,713 INFO:load_nested[128]:Creating temp customers (chunk 10 of 34)
2017-09-20 14:33:51,923 INFO:load_nested[128]:Creating temp customers (chunk 11 of 34)
2017-09-20 14:34:15,656 INFO:load_nested[128]:Creating temp customers (chunk 12 of 34)
2017-09-20 14:34:38,900 INFO:load_nested[128]:Creating temp customers (chunk 13 of 34)
2017-09-20 14:35:02,628 INFO:load_nested[128]:Creating temp customers (chunk 14 of 34)
2017-09-20 14:35:26,347 INFO:load_nested[128]:Creating temp customers (chunk 15 of 34)
2017-09-20 14:35:51,079 INFO:load_nested[128]:Creating temp customers (chunk 16 of 34)
2017-09-20 14:36:14,804 INFO:load_nested[128]:Creating temp customers (chunk 17 of 34)
2017-09-20 14:36:38,535 INFO:load_nested[128]:Creating temp customers (chunk 18 of 34)
2017-09-20 14:37:01,765 INFO:load_nested[128]:Creating temp customers (chunk 19 of 34)
2017-09-20 14:37:25,512 INFO:load_nested[128]:Creating temp customers (chunk 20 of 34)
2017-09-20 14:37:49,745 INFO:load_nested[128]:Creating temp customers (chunk 21 of 34)
2017-09-20 14:38:13,980 INFO:load_nested[128]:Creating temp customers (chunk 22 of 34)
2017-09-20 14:38:37,706 INFO:load_nested[128]:Creating temp customers (chunk 23 of 34)
2017-09-20 14:39:01,428 INFO:load_nested[128]:Creating temp customers (chunk 24 of 34)
2017-09-20 14:39:25,652 INFO:load_nested[128]:Creating temp customers (chunk 25 of 34)
2017-09-20 14:39:49,368 INFO:load_nested[128]:Creating temp customers (chunk 26 of 34)
2017-09-20 14:40:13,600 INFO:load_nested[128]:Creating temp customers (chunk 27 of 34)
2017-09-20 14:40:38,902 INFO:load_nested[128]:Creating temp customers (chunk 28 of 34)
2017-09-20 14:41:03,117 INFO:load_nested[128]:Creating temp customers (chunk 29 of 34)
2017-09-20 14:41:26,847 INFO:load_nested[128]:Creating temp customers (chunk 30 of 34)
2017-09-20 14:41:51,068 INFO:load_nested[128]:Creating temp customers (chunk 31 of 34)
2017-09-20 14:42:15,300 INFO:load_nested[128]:Creating temp customers (chunk 32 of 34)
2017-09-20 14:42:39,527 INFO:load_nested[128]:Creating temp customers (chunk 33 of 34)
2017-09-20 14:43:03,753 INFO:load_nested[128]:Creating temp customers (chunk 34 of 34)
2017-09-20 14:43:28,187 INFO:load_nested[178]:Creating temp regions
2017-09-20 14:43:38,690 INFO:load_nested[208]:Creating temp suppliers
2017-09-20 14:44:51,084 INFO:load_nested[244]:Creating parts
2017-09-20 14:44:59,144 INFO:cluster[422]:kerberos module not found; attempting to install it...
2017-09-20 14:45:00,056 INFO:cluster[429]:kerberos installation complete.
/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
2017-09-20 14:45:21,562 ERROR:kerberos_[226]:handle_other(): Mutual authentication unavailable on 403 response
Traceback (most recent call last):
  File "testdata/bin/load_nested.py", line 320, in <module>
    load()
  File "testdata/bin/load_nested.py", line 253, in load
    cluster.hdfs.ensure_home_dir()
  File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/tests/comparison/cluster.py", line 402, in ensure_home_dir
    client.makedirs(hdfs_dir)
  File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py", line 866, in makedirs
    self._mkdirs(hdfs_path, permission=permission)
  File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py", line 90, in api_handler
    **self.kwargs
  File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/ext/kerberos.py", line 98, in _request
    return super(KerberosClient, self)._request(method, url, **kwargs)
  File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py", line 182, in _request
    return _on_error(response)
  File "/var/lib/jenkins/workspace/Impala-Stress-Test-Physical/Impala/infra/python/env/local/lib/python2.7/site-packages/hdfs/client.py", line 44, in _on_error
    raise HdfsError(message)
hdfs.util.HdfsError: Permission denied: user=systest, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

      Attachments

        Activity

          People

            Unassigned Unassigned
            mmulder Matthew Mulder
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: