Hive
  1. Hive
  2. HIVE-6891

Alter rename partition Perm inheritance and general partition/table group inheritance

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      Found this issue while looking at the method mentioned by HIVE-6648.

      'alter table .. partition .. rename to ..' and other commands calling Warehouse.mkdirs() doesn't inherit permission on the partition directories and consequently the data, when "hive.warehouse.subdir.inherit.perms" is set.

      Also, in these scenarios of directory creation, group is not being inherited. Data files are already inheriting group by HIVE-3756.

      1. HIVE-6891.patch
        21 kB
        Szehon Ho
      2. HIVE-6891.2.patch
        21 kB
        Szehon Ho
      3. HIVE-6891.3.patch
        22 kB
        Szehon Ho
      4. HIVE-6891.4.patch
        22 kB
        Szehon Ho

        Issue Links

          Activity

          Hide
          Szehon Ho added a comment -

          Addressing review comments.

          Show
          Szehon Ho added a comment - Addressing review comments.
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12639888/HIVE-6891.2.patch

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2229/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2229/console

          Messages:

          **** This message was trimmed, see log for full details ****
          [INFO] Tests are skipped.
          [INFO] 
          [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hcatalog-it-unit ---
          [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
          [INFO] 
          [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-hcatalog-it-unit ---
          [INFO] 
          [INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-hcatalog-it-unit ---
          [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
          [INFO] 
          [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-hcatalog-it-unit ---
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.pom
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
          [INFO]                                                                         
          [INFO] ------------------------------------------------------------------------
          [INFO] Building Hive Integration - Testing Utilities 0.14.0-SNAPSHOT
          [INFO] ------------------------------------------------------------------------
          [INFO] 
          [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-util ---
          [INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/util (includes = [datanucleus.log, derby.log], excludes = [])
          [INFO] 
          [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-util ---
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-util ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-util ---
          [INFO] Executing tasks
          
          main:
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-util ---
          [INFO] Compiling 45 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/classes
          [WARNING] Note: Some input files use or override a deprecated API.
          [WARNING] Note: Recompile with -Xlint:deprecation for details.
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-util ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/test/resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-util ---
          [INFO] Executing tasks
          
          main:
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/warehouse
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf
               [copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-util ---
          [INFO] No sources to compile
          [INFO] 
          [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-it-util ---
          [INFO] Tests are skipped.
          [INFO] 
          [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-it-util ---
          [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar
          [INFO] 
          [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-it-util ---
          [INFO] 
          [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-it-util ---
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.jar
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.pom
          [INFO]                                                                         
          [INFO] ------------------------------------------------------------------------
          [INFO] Building Hive Integration - Unit Tests 0.14.0-SNAPSHOT
          [INFO] ------------------------------------------------------------------------
          [INFO] 
          [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-unit ---
          [INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit (includes = [datanucleus.log, derby.log], excludes = [])
          [INFO] 
          [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-unit ---
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-unit ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/main/resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-unit ---
          [INFO] Executing tasks
          
          main:
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-unit ---
          [INFO] Compiling 2 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/classes
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-unit ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-unit ---
          [INFO] Executing tasks
          
          main:
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/warehouse
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf
               [copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (setup-metastore-scripts) @ hive-it-unit ---
          [INFO] Executing tasks
          
          main:
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore
               [copy] Copying 156 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-unit ---
          [INFO] Compiling 56 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/test-classes
          [INFO] -------------------------------------------------------------
          [WARNING] COMPILATION WARNING : 
          [INFO] -------------------------------------------------------------
          [WARNING] Note: Some input files use or override a deprecated API.
          [WARNING] Note: Recompile with -Xlint:deprecation for details.
          [WARNING] Note: Some input files use unchecked or unsafe operations.
          [WARNING] Note: Recompile with -Xlint:unchecked for details.
          [INFO] 4 warnings 
          [INFO] -------------------------------------------------------------
          [INFO] -------------------------------------------------------------
          [ERROR] COMPILATION ERROR : 
          [INFO] -------------------------------------------------------------
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[28,28] cannot find symbol
          symbol  : class LocatedFileStatus
          location: package org.apache.hadoop.fs
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[30,28] cannot find symbol
          symbol  : class RemoteIterator
          location: package org.apache.hadoop.fs
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,5] cannot find symbol
          symbol  : class RemoteIterator
          location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,20] cannot find symbol
          symbol  : class LocatedFileStatus
          location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,56] cannot find symbol
          symbol  : method listFiles(org.apache.hadoop.fs.Path,boolean)
          location: class org.apache.hadoop.fs.FileSystem
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[233,7] cannot find symbol
          symbol  : class LocatedFileStatus
          location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:[396,16] mkdirs(org.apache.hadoop.fs.Path,boolean) in org.apache.hadoop.hive.metastore.Warehouse cannot be applied to (org.apache.hadoop.fs.Path)
          [INFO] 7 errors 
          [INFO] -------------------------------------------------------------
          [INFO] ------------------------------------------------------------------------
          [INFO] Reactor Summary:
          [INFO] 
          [INFO] Hive Integration - Parent ......................... SUCCESS [9.419s]
          [INFO] Hive Integration - Custom Serde ................... SUCCESS [11.176s]
          [INFO] Hive Integration - HCatalog Unit Tests ............ SUCCESS [7.733s]
          [INFO] Hive Integration - Testing Utilities .............. SUCCESS [9.561s]
          [INFO] Hive Integration - Unit Tests ..................... FAILURE [10.414s]
          [INFO] Hive Integration - Test Serde ..................... SKIPPED
          [INFO] Hive Integration - QFile Tests .................... SKIPPED
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD FAILURE
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 50.065s
          [INFO] Finished at: Fri Apr 11 22:49:18 EDT 2014
          [INFO] Final Memory: 38M/108M
          [INFO] ------------------------------------------------------------------------
          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project hive-it-unit: Compilation failure: Compilation failure:
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[28,28] cannot find symbol
          [ERROR] symbol  : class LocatedFileStatus
          [ERROR] location: package org.apache.hadoop.fs
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[30,28] cannot find symbol
          [ERROR] symbol  : class RemoteIterator
          [ERROR] location: package org.apache.hadoop.fs
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,5] cannot find symbol
          [ERROR] symbol  : class RemoteIterator
          [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,20] cannot find symbol
          [ERROR] symbol  : class LocatedFileStatus
          [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,56] cannot find symbol
          [ERROR] symbol  : method listFiles(org.apache.hadoop.fs.Path,boolean)
          [ERROR] location: class org.apache.hadoop.fs.FileSystem
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[233,7] cannot find symbol
          [ERROR] symbol  : class LocatedFileStatus
          [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:[396,16] mkdirs(org.apache.hadoop.fs.Path,boolean) in org.apache.hadoop.hive.metastore.Warehouse cannot be applied to (org.apache.hadoop.fs.Path)
          [ERROR] -> [Help 1]
          [ERROR] 
          [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
          [ERROR] Re-run Maven using the -X switch to enable full debug logging.
          [ERROR] 
          [ERROR] For more information about the errors and possible solutions, please read the following articles:
          [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
          [ERROR] 
          [ERROR] After correcting the problems, you can resume the build with the command
          [ERROR]   mvn <goals> -rf :hive-it-unit
          + exit 1
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12639888

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12639888/HIVE-6891.2.patch Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2229/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2229/console Messages: **** This message was trimmed, see log for full details **** [INFO] Tests are skipped. [INFO] [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hcatalog-it-unit --- [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-hcatalog-it-unit --- [INFO] [INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-hcatalog-it-unit --- [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-hcatalog-it-unit --- [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.pom [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Hive Integration - Testing Utilities 0.14.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-util --- [INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/util (includes = [datanucleus.log, derby.log], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-util --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-util --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-util --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-util --- [INFO] Compiling 45 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/classes [WARNING] Note: Some input files use or override a deprecated API. [WARNING] Note: Recompile with -Xlint:deprecation for details. [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-util --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/test/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-util --- [INFO] Executing tasks main: [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/warehouse [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf [copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-util --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-it-util --- [INFO] Tests are skipped. [INFO] [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-it-util --- [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-it-util --- [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-it-util --- [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.pom [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Hive Integration - Unit Tests 0.14.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-unit --- [INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit (includes = [datanucleus.log, derby.log], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-unit --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-unit --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-unit --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-unit --- [INFO] Compiling 2 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/classes [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-unit --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-unit --- [INFO] Executing tasks main: [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/warehouse [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf [copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf [INFO] Executed tasks [INFO] [INFO] --- maven-antrun-plugin:1.7:run (setup-metastore-scripts) @ hive-it-unit --- [INFO] Executing tasks main: [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore [copy] Copying 156 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-unit --- [INFO] Compiling 56 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/test-classes [INFO] ------------------------------------------------------------- [WARNING] COMPILATION WARNING : [INFO] ------------------------------------------------------------- [WARNING] Note: Some input files use or override a deprecated API. [WARNING] Note: Recompile with -Xlint:deprecation for details. [WARNING] Note: Some input files use unchecked or unsafe operations. [WARNING] Note: Recompile with -Xlint:unchecked for details. [INFO] 4 warnings [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[28,28] cannot find symbol symbol : class LocatedFileStatus location: package org.apache.hadoop.fs [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[30,28] cannot find symbol symbol : class RemoteIterator location: package org.apache.hadoop.fs [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,5] cannot find symbol symbol : class RemoteIterator location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,20] cannot find symbol symbol : class LocatedFileStatus location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,56] cannot find symbol symbol : method listFiles(org.apache.hadoop.fs.Path,boolean) location: class org.apache.hadoop.fs.FileSystem [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[233,7] cannot find symbol symbol : class LocatedFileStatus location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:[396,16] mkdirs(org.apache.hadoop.fs.Path,boolean) in org.apache.hadoop.hive.metastore.Warehouse cannot be applied to (org.apache.hadoop.fs.Path) [INFO] 7 errors [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Hive Integration - Parent ......................... SUCCESS [9.419s] [INFO] Hive Integration - Custom Serde ................... SUCCESS [11.176s] [INFO] Hive Integration - HCatalog Unit Tests ............ SUCCESS [7.733s] [INFO] Hive Integration - Testing Utilities .............. SUCCESS [9.561s] [INFO] Hive Integration - Unit Tests ..................... FAILURE [10.414s] [INFO] Hive Integration - Test Serde ..................... SKIPPED [INFO] Hive Integration - QFile Tests .................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 50.065s [INFO] Finished at: Fri Apr 11 22:49:18 EDT 2014 [INFO] Final Memory: 38M/108M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project hive-it-unit: Compilation failure: Compilation failure: [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[28,28] cannot find symbol [ERROR] symbol : class LocatedFileStatus [ERROR] location: package org.apache.hadoop.fs [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[30,28] cannot find symbol [ERROR] symbol : class RemoteIterator [ERROR] location: package org.apache.hadoop.fs [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,5] cannot find symbol [ERROR] symbol : class RemoteIterator [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,20] cannot find symbol [ERROR] symbol : class LocatedFileStatus [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,56] cannot find symbol [ERROR] symbol : method listFiles(org.apache.hadoop.fs.Path,boolean) [ERROR] location: class org.apache.hadoop.fs.FileSystem [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[233,7] cannot find symbol [ERROR] symbol : class LocatedFileStatus [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:[396,16] mkdirs(org.apache.hadoop.fs.Path,boolean) in org.apache.hadoop.hive.metastore.Warehouse cannot be applied to (org.apache.hadoop.fs.Path) [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hive-it-unit + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12639888
          Hide
          Szehon Ho added a comment -

          Fixing TestHiveMetaStore

          Show
          Szehon Ho added a comment - Fixing TestHiveMetaStore
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12639923/HIVE-6891.3.patch

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2235/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2235/console

          Messages:

          **** This message was trimmed, see log for full details ****
          [INFO] 
          [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-hcatalog-it-unit ---
          [INFO] Tests are skipped.
          [INFO] 
          [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hcatalog-it-unit ---
          [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
          [INFO] 
          [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-hcatalog-it-unit ---
          [INFO] 
          [INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-hcatalog-it-unit ---
          [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
          [INFO] 
          [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-hcatalog-it-unit ---
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.pom
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar
          [INFO]                                                                         
          [INFO] ------------------------------------------------------------------------
          [INFO] Building Hive Integration - Testing Utilities 0.14.0-SNAPSHOT
          [INFO] ------------------------------------------------------------------------
          [INFO] 
          [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-util ---
          [INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/util (includes = [datanucleus.log, derby.log], excludes = [])
          [INFO] 
          [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-util ---
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-util ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-util ---
          [INFO] Executing tasks
          
          main:
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-util ---
          [INFO] Compiling 45 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/classes
          [WARNING] Note: Some input files use or override a deprecated API.
          [WARNING] Note: Recompile with -Xlint:deprecation for details.
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-util ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/test/resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-util ---
          [INFO] Executing tasks
          
          main:
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/warehouse
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf
               [copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-util ---
          [INFO] No sources to compile
          [INFO] 
          [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-it-util ---
          [INFO] Tests are skipped.
          [INFO] 
          [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-it-util ---
          [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar
          [INFO] 
          [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-it-util ---
          [INFO] 
          [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-it-util ---
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.jar
          [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.pom
          [INFO]                                                                         
          [INFO] ------------------------------------------------------------------------
          [INFO] Building Hive Integration - Unit Tests 0.14.0-SNAPSHOT
          [INFO] ------------------------------------------------------------------------
          [INFO] 
          [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-unit ---
          [INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit (includes = [datanucleus.log, derby.log], excludes = [])
          [INFO] 
          [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-unit ---
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-unit ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/main/resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-unit ---
          [INFO] Executing tasks
          
          main:
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-unit ---
          [INFO] Compiling 2 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/classes
          [INFO] 
          [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-unit ---
          [INFO] Using 'UTF-8' encoding to copy filtered resources.
          [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/resources
          [INFO] Copying 3 resources
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-unit ---
          [INFO] Executing tasks
          
          main:
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/warehouse
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf
               [copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-antrun-plugin:1.7:run (setup-metastore-scripts) @ hive-it-unit ---
          [INFO] Executing tasks
          
          main:
              [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore
               [copy] Copying 156 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore
          [INFO] Executed tasks
          [INFO] 
          [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-unit ---
          [INFO] Compiling 56 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/test-classes
          [INFO] -------------------------------------------------------------
          [WARNING] COMPILATION WARNING : 
          [INFO] -------------------------------------------------------------
          [WARNING] Note: Some input files use or override a deprecated API.
          [WARNING] Note: Recompile with -Xlint:deprecation for details.
          [WARNING] Note: Some input files use unchecked or unsafe operations.
          [WARNING] Note: Recompile with -Xlint:unchecked for details.
          [INFO] 4 warnings 
          [INFO] -------------------------------------------------------------
          [INFO] -------------------------------------------------------------
          [ERROR] COMPILATION ERROR : 
          [INFO] -------------------------------------------------------------
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[28,28] cannot find symbol
          symbol  : class LocatedFileStatus
          location: package org.apache.hadoop.fs
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[30,28] cannot find symbol
          symbol  : class RemoteIterator
          location: package org.apache.hadoop.fs
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,5] cannot find symbol
          symbol  : class RemoteIterator
          location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,20] cannot find symbol
          symbol  : class LocatedFileStatus
          location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,56] cannot find symbol
          symbol  : method listFiles(org.apache.hadoop.fs.Path,boolean)
          location: class org.apache.hadoop.fs.FileSystem
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[233,7] cannot find symbol
          symbol  : class LocatedFileStatus
          location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [INFO] 6 errors 
          [INFO] -------------------------------------------------------------
          [INFO] ------------------------------------------------------------------------
          [INFO] Reactor Summary:
          [INFO] 
          [INFO] Hive Integration - Parent ......................... SUCCESS [9.652s]
          [INFO] Hive Integration - Custom Serde ................... SUCCESS [12.170s]
          [INFO] Hive Integration - HCatalog Unit Tests ............ SUCCESS [8.083s]
          [INFO] Hive Integration - Testing Utilities .............. SUCCESS [9.127s]
          [INFO] Hive Integration - Unit Tests ..................... FAILURE [10.498s]
          [INFO] Hive Integration - Test Serde ..................... SKIPPED
          [INFO] Hive Integration - QFile Tests .................... SKIPPED
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD FAILURE
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 51.341s
          [INFO] Finished at: Sat Apr 12 08:57:22 EDT 2014
          [INFO] Final Memory: 38M/96M
          [INFO] ------------------------------------------------------------------------
          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project hive-it-unit: Compilation failure: Compilation failure:
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[28,28] cannot find symbol
          [ERROR] symbol  : class LocatedFileStatus
          [ERROR] location: package org.apache.hadoop.fs
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[30,28] cannot find symbol
          [ERROR] symbol  : class RemoteIterator
          [ERROR] location: package org.apache.hadoop.fs
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,5] cannot find symbol
          [ERROR] symbol  : class RemoteIterator
          [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,20] cannot find symbol
          [ERROR] symbol  : class LocatedFileStatus
          [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,56] cannot find symbol
          [ERROR] symbol  : method listFiles(org.apache.hadoop.fs.Path,boolean)
          [ERROR] location: class org.apache.hadoop.fs.FileSystem
          [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[233,7] cannot find symbol
          [ERROR] symbol  : class LocatedFileStatus
          [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions
          [ERROR] -> [Help 1]
          [ERROR] 
          [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
          [ERROR] Re-run Maven using the -X switch to enable full debug logging.
          [ERROR] 
          [ERROR] For more information about the errors and possible solutions, please read the following articles:
          [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
          [ERROR] 
          [ERROR] After correcting the problems, you can resume the build with the command
          [ERROR]   mvn <goals> -rf :hive-it-unit
          + exit 1
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12639923

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12639923/HIVE-6891.3.patch Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2235/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2235/console Messages: **** This message was trimmed, see log for full details **** [INFO] [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-hcatalog-it-unit --- [INFO] Tests are skipped. [INFO] [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-hcatalog-it-unit --- [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-hcatalog-it-unit --- [INFO] [INFO] --- maven-jar-plugin:2.2:test-jar (default) @ hive-hcatalog-it-unit --- [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-hcatalog-it-unit --- [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT.pom [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/hcatalog-unit/target/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-hcatalog-it-unit/0.14.0-SNAPSHOT/hive-hcatalog-it-unit-0.14.0-SNAPSHOT-tests.jar [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Hive Integration - Testing Utilities 0.14.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-util --- [INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/util (includes = [datanucleus.log, derby.log], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-util --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-util --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-util --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-util --- [INFO] Compiling 45 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/classes [WARNING] Note: Some input files use or override a deprecated API. [WARNING] Note: Recompile with -Xlint:deprecation for details. [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-util --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/util/src/test/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-util --- [INFO] Executing tasks main: [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/warehouse [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf [copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/tmp/conf [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-util --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-it-util --- [INFO] Tests are skipped. [INFO] [INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ hive-it-util --- [INFO] Building jar: /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar [INFO] [INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ hive-it-util --- [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-it-util --- [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/target/hive-it-util-0.14.0-SNAPSHOT.jar to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.jar [INFO] Installing /data/hive-ptest/working/apache-svn-trunk-source/itests/util/pom.xml to /data/hive-ptest/working/maven/org/apache/hive/hive-it-util/0.14.0-SNAPSHOT/hive-it-util-0.14.0-SNAPSHOT.pom [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Hive Integration - Unit Tests 0.14.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-it-unit --- [INFO] Deleting /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit (includes = [datanucleus.log, derby.log], excludes = []) [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-unit --- [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-it-unit --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-unit --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-unit --- [INFO] Compiling 2 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/classes [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hive-it-unit --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-unit --- [INFO] Executing tasks main: [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/warehouse [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf [copy] Copying 5 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/conf [INFO] Executed tasks [INFO] [INFO] --- maven-antrun-plugin:1.7:run (setup-metastore-scripts) @ hive-it-unit --- [INFO] Executing tasks main: [mkdir] Created dir: /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore [copy] Copying 156 files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/tmp/scripts/metastore [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-unit --- [INFO] Compiling 56 source files to /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/target/test-classes [INFO] ------------------------------------------------------------- [WARNING] COMPILATION WARNING : [INFO] ------------------------------------------------------------- [WARNING] Note: Some input files use or override a deprecated API. [WARNING] Note: Recompile with -Xlint:deprecation for details. [WARNING] Note: Some input files use unchecked or unsafe operations. [WARNING] Note: Recompile with -Xlint:unchecked for details. [INFO] 4 warnings [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[28,28] cannot find symbol symbol : class LocatedFileStatus location: package org.apache.hadoop.fs [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[30,28] cannot find symbol symbol : class RemoteIterator location: package org.apache.hadoop.fs [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,5] cannot find symbol symbol : class RemoteIterator location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,20] cannot find symbol symbol : class LocatedFileStatus location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,56] cannot find symbol symbol : method listFiles(org.apache.hadoop.fs.Path,boolean) location: class org.apache.hadoop.fs.FileSystem [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[233,7] cannot find symbol symbol : class LocatedFileStatus location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [INFO] 6 errors [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Hive Integration - Parent ......................... SUCCESS [9.652s] [INFO] Hive Integration - Custom Serde ................... SUCCESS [12.170s] [INFO] Hive Integration - HCatalog Unit Tests ............ SUCCESS [8.083s] [INFO] Hive Integration - Testing Utilities .............. SUCCESS [9.127s] [INFO] Hive Integration - Unit Tests ..................... FAILURE [10.498s] [INFO] Hive Integration - Test Serde ..................... SKIPPED [INFO] Hive Integration - QFile Tests .................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 51.341s [INFO] Finished at: Sat Apr 12 08:57:22 EDT 2014 [INFO] Final Memory: 38M/96M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project hive-it-unit: Compilation failure: Compilation failure: [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[28,28] cannot find symbol [ERROR] symbol : class LocatedFileStatus [ERROR] location: package org.apache.hadoop.fs [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[30,28] cannot find symbol [ERROR] symbol : class RemoteIterator [ERROR] location: package org.apache.hadoop.fs [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,5] cannot find symbol [ERROR] symbol : class RemoteIterator [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,20] cannot find symbol [ERROR] symbol : class LocatedFileStatus [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[231,56] cannot find symbol [ERROR] symbol : method listFiles(org.apache.hadoop.fs.Path,boolean) [ERROR] location: class org.apache.hadoop.fs.FileSystem [ERROR] /data/hive-ptest/working/apache-svn-trunk-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java:[233,7] cannot find symbol [ERROR] symbol : class LocatedFileStatus [ERROR] location: class org.apache.hadoop.hive.ql.security.TestFolderPermissions [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hive-it-unit + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12639923
          Hide
          Szehon Ho added a comment -

          I see, was using hadoop-2 profile to build the test, and pre commit test is using hadoop-1..

          Show
          Szehon Ho added a comment - I see, was using hadoop-2 profile to build the test, and pre commit test is using hadoop-1..
          Hide
          Szehon Ho added a comment -

          Using HadoopShims for listing the files works in both profiles works for me, lets see how far it goes.

          Show
          Szehon Ho added a comment - Using HadoopShims for listing the files works in both profiles works for me, lets see how far it goes.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12639948/HIVE-6891.4.patch

          ERROR: -1 due to 2 failed/errored test(s), 5619 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2237/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2237/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12639948

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12639948/HIVE-6891.4.patch ERROR: -1 due to 2 failed/errored test(s), 5619 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2237/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2237/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12639948
          Hide
          Szehon Ho added a comment -

          That seems to work, fixing a minor whitespace missed in previous patch and uploading to rb.

          Show
          Szehon Ho added a comment - That seems to work, fixing a minor whitespace missed in previous patch and uploading to rb.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12639963/HIVE-6891.4.patch

          ERROR: -1 due to 4 failed/errored test(s), 5619 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_disable_merge_for_bucketing
          org.apache.hadoop.hive.thrift.TestDBTokenStore.testDBTokenStore
          org.apache.hive.service.cli.thrift.TestThriftBinaryCLIService.testExecuteStatementAsync
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2238/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2238/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 4 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12639963

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12639963/HIVE-6891.4.patch ERROR: -1 due to 4 failed/errored test(s), 5619 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_disable_merge_for_bucketing org.apache.hadoop.hive.thrift.TestDBTokenStore.testDBTokenStore org.apache.hive.service.cli.thrift.TestThriftBinaryCLIService.testExecuteStatementAsync Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2238/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2238/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed This message is automatically generated. ATTACHMENT ID: 12639963
          Hide
          Szehon Ho added a comment -

          Brock Noland Thanks for the review, these test failures dont look like related to this patch.

          I ran and they passed locally. TestDBTokenStore looks like a transient derby error, and the other looks like a transient error with the test itself.

          Show
          Szehon Ho added a comment - Brock Noland Thanks for the review, these test failures dont look like related to this patch. I ran and they passed locally. TestDBTokenStore looks like a transient derby error, and the other looks like a transient error with the test itself.
          Hide
          Brock Noland added a comment -

          +1

          Show
          Brock Noland added a comment - +1
          Hide
          Brock Noland added a comment -

          I'd like to commit this, but I am locked out of my account. I am working on this but if someone else has the ability to commit this, please go ahead.

          Show
          Brock Noland added a comment - I'd like to commit this, but I am locked out of my account. I am working on this but if someone else has the ability to commit this, please go ahead.
          Hide
          Prasad Mujumdar added a comment -

          Patch committed to trunk.
          Thanks Szehon Ho for the contribution, thanks Brock Noland for the review.

          Show
          Prasad Mujumdar added a comment - Patch committed to trunk. Thanks Szehon Ho for the contribution, thanks Brock Noland for the review.
          Hide
          Thejas M Nair added a comment -

          This has been fixed in 0.14 release. Please open new jira if you see any issues.

          Show
          Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.

            People

            • Assignee:
              Szehon Ho
              Reporter:
              Szehon Ho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development