Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24200

MSCK repair table is not working

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 3.1.0
    • Fix Version/s: None
    • Component/s: CLI, Hive, HiveServer2
    • Labels:
      None

      Description

      steps to reproduce :

      create external table test_sync_part (name string) partitioned by (id int) location '/projects/PTEST/dev/hive/test_sync_part';

      insert into table test_sync_part values ('nom1',1),('nom2',2);

      delete the sub-folder of one partition on the folder /projects/PTEST/dev/hive/test_sync_part

      msck repair table test_sync_part drop partitions;

      2020-09-24T14:45:57,419 INFO  [HiveServer2-Background-Pool: Thread-208]: metastore.Msck (:()) - Tables not in metastore: []
      2020-09-24T14:45:57,419 INFO  [HiveServer2-Background-Pool: Thread-208]: metastore.Msck (:()) - Tables missing on filesystem: []
      2020-09-24T14:45:57,419 INFO  [HiveServer2-Background-Pool: Thread-208]: metastore.Msck (:()) - Partitions not in metastore: []
      2020-09-24T14:45:57,419 INFO  [HiveServer2-Background-Pool: Thread-208]: metastore.Msck (:()) - Partitions missing from filesystem: [test_sync_part:id=2]
      2020-09-24T14:45:57,419 INFO  [HiveServer2-Background-Pool: Thread-208]: metastore.Msck (:()) - Expired partitions: []
      2020-09-24T14:45:57,420 INFO  [HiveServer2-Background-Pool: Thread-208]: metastore.HiveMetaStoreClient (:()) - Closed a connection to metastore, current connections: 8
      2020-09-24T14:45:57,420 INFO  [HiveServer2-Background-Pool: Thread-208]: reexec.ReOptimizePlugin (:()) - ReOptimization: retryPossible: false
      2020-09-24T14:45:57,420 INFO  [HiveServer2-Background-Pool: Thread-208]: hooks.HiveProtoLoggingHook (:()) - Received post-hook notification for: hive_20200924144557_3e164203-720a-4e4a-bbdd-b65f53901e15
      2020-09-24T14:45:57,421 ERROR [HiveServer2-Background-Pool: Thread-208]: ql.Driver (:()) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
      2020-09-24T14:45:57,421 INFO  [HiveServer2-Background-Pool: Thread-208]: ql.Driver (:()) - Completed executing command(queryId=hive_20200924144557_3e164203-720a-4e4a-bbdd-b65f53901e15); Time taken: 0.289 seconds
      2020-09-24T14:45:57,421 INFO  [HiveServer2-Background-Pool: Thread-208]: lockmgr.DbTxnManager (:()) - Stopped heartbeat for query: hive_20200924144557_3e164203-720a-4e4a-bbdd-b65f53901e15
      2020-09-24T14:45:57,458 ERROR [HiveServer2-Background-Pool: Thread-208]: operation.Operation (:()) - Error running hive query:
      org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
              at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:348) ~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
              at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:228) ~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
              at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87) ~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
              at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:324) ~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
              at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
              at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) ~[hadoop-common-3.1.1.3.1.4.0-315.jar:?]
              at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342) ~[hive-service-3.1.0.3.1.4.0-315.jar:3.1.0.3.1.4.0-315]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_112]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_112]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_112]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_112]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_112]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_112]
              at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
      

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              stephbat stephbat
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Due:
                Created:
                Updated: