Hive
  1. Hive
  2. HIVE-7001

fs.permissions.umask-mode is getting unset when Session is started

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.0
    • Fix Version/s: 0.14.0, 0.13.1
    • Component/s: None
    • Labels:
      None

      Description

      hive> set fs.permissions.umask-mode;
      fs.permissions.umask-mode=022
      hive> show tables;
      OK
      t1
      Time taken: 0.301 seconds, Fetched: 1 row(s)
      hive> set fs.permissions.umask-mode;
      fs.permissions.umask-mode is undefined
      
      
      1. HIVE-7001.1.patch
        2 kB
        Thejas M Nair
      2. HIVE-7001.2.patch
        4 kB
        Thejas M Nair
      3. HIVE-7001.3.patch
        4 kB
        Thejas M Nair
      4. TestUMask.patch
        3 kB
        Venki Korukanti

        Issue Links

          Activity

          Hide
          Venki Korukanti added a comment -

          Thanks Thejas M Nair. Created HIVE-7807 and attached patch. Please review.

          Show
          Venki Korukanti added a comment - Thanks Thejas M Nair . Created HIVE-7807 and attached patch. Please review.
          Hide
          Thejas M Nair added a comment -

          Using FsPermission.UMASK_LABEL sounds good to me .
          Please open a new jira.

          Show
          Thejas M Nair added a comment - Using FsPermission.UMASK_LABEL sounds good to me . Please open a new jira.
          Hide
          Venki Korukanti added a comment -

          Hi Thejas M Nair, One question regarding the "fs.permissions.umask-mode". Looks like "fs.permissions.umask-mode" doesn't exist in Hadoop 1.x and property "dfs.umaskmode" is used instead in 1.x for the same purpose. Also "dfs.umaskmode" was not deprecated in 1.x according to HADOOP-8727. Should we use FsPermission.UMASK_LABEL instead of "fs.permissions.umask-mode" which always points to proper property in latest Hadoop in each version (0.23.x, 1.x, 2.x)?

          Attached a testcase to illustrate the problem. Test passes fine with -Phadoop-2, but not with -Phadoop-1.

          Show
          Venki Korukanti added a comment - Hi Thejas M Nair , One question regarding the "fs.permissions.umask-mode". Looks like "fs.permissions.umask-mode" doesn't exist in Hadoop 1.x and property "dfs.umaskmode" is used instead in 1.x for the same purpose. Also "dfs.umaskmode" was not deprecated in 1.x according to HADOOP-8727 . Should we use FsPermission.UMASK_LABEL instead of "fs.permissions.umask-mode" which always points to proper property in latest Hadoop in each version (0.23.x, 1.x, 2.x)? Attached a testcase to illustrate the problem. Test passes fine with -Phadoop-2, but not with -Phadoop-1.
          Hide
          Thejas M Nair added a comment -

          This jira has been fixed as part of 0.13.1 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Thejas M Nair added a comment - This jira has been fixed as part of 0.13.1 release. If you find further issues, please create a new jira and link it to this one.
          Hide
          Thejas M Nair added a comment -

          Patch committed to trunk.
          Thanks for the reviews Ashutosh & Vikram!

          Show
          Thejas M Nair added a comment - Patch committed to trunk. Thanks for the reviews Ashutosh & Vikram!
          Hide
          Thejas M Nair added a comment -

          Ran parquet_ctas locally and it passed, rest of the test failures are unrelated.

          Show
          Thejas M Nair added a comment - Ran parquet_ctas locally and it passed, rest of the test failures are unrelated.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12642930/HIVE-7001.3.patch

          ERROR: -1 due to 7 failed/errored test(s), 5429 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_ctas
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/100/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/100/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 7 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12642930

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12642930/HIVE-7001.3.patch ERROR: -1 due to 7 failed/errored test(s), 5429 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_ctas org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/100/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/100/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed This message is automatically generated. ATTACHMENT ID: 12642930
          Hide
          Vikram Dixit K added a comment -

          +1 LGTM.

          Show
          Vikram Dixit K added a comment - +1 LGTM.
          Hide
          Thejas M Nair added a comment -

          Sushanth Sowmyan Apart from being a critical bug fix, this also enables a workaround for pre-hadoop 1.2 versions. See HIVE-6962 . That is added value including this in hive 0.13.1.

          Show
          Thejas M Nair added a comment - Sushanth Sowmyan Apart from being a critical bug fix, this also enables a workaround for pre-hadoop 1.2 versions. See HIVE-6962 . That is added value including this in hive 0.13.1.
          Hide
          Thejas M Nair added a comment -

          HIVE-7001.3.patch - better 'unit' test

          Show
          Thejas M Nair added a comment - HIVE-7001 .3.patch - better 'unit' test
          Hide
          Thejas M Nair added a comment -

          HIVE-7001.2.patch - also includes a test. Checks if value is preserved.

          Show
          Thejas M Nair added a comment - HIVE-7001 .2.patch - also includes a test. Checks if value is preserved.
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1

            People

            • Assignee:
              Thejas M Nair
              Reporter:
              Thejas M Nair
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development