Uploaded image for project: 'Sentry'
  1. Sentry
  2. SENTRY-2127

Fix unstable unit test TestColumnEndToEnd.testCrossDbTableOperations

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.1.0
    • Component/s: Sentry
    • Labels:
      None

      Description

      Occasionally, the test TestColumnEndToEnd.testCrossDbTableOperations fails at statement
      statement.execute("CREATE table db_1.t1 as select tb1.id, tb3.val, tb2.num from db_1.tb1,db_2.tb3,db_2.tb2");
      We should make it stable to avoid blocking other updates.

      More info:

      1) Failed run with the following statement. The reason it fails is not related to sentry processing. It is due to the map reduce task fails with the more complicated join in the unit test

      2018-01-23 19:46:28,766 (5c4ce662-7587-40a5-8b24-b4c31f006b53 HiveServer2-Handler-Pool: Thread-365) [INFO - org.apache.hadoop.hive.ql.Driver.compile(Driver.java:460)] Compiling command(queryId=jenkins_20180123194628_73ef2fc1-7819-4329-9d1c-96775dcc839c): CREATE table db_1.t1 as select tb1.id, tb3.val, tb2.num from db_1.tb1,db_2.tb3,db_2.tb2
      ...
      2018-01-23 19:46:29,266 (HiveServer2-Background-Pool: Thread-373) [INFO - org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1713)] Executing command(queryId=jenkins_20180123194628_73ef2fc1-7819-4329-9d1c-96775dcc839c): CREATE table db_1.t1 as select tb1.id, tb3.val, tb2.num from db_1.tb1,db_2.tb3,db_2.tb2
      ...
      2018-01-23 19:49:29,229 (HiveServer2-Background-Pool: Thread-373) [INFO - org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2021)] Completed executing command(queryId=jenkins_20180123194628_73ef2fc1-7819-4329-9d1c-96775dcc839c); Time taken: 179.962 seconds
      2018-01-23 19:49:29,235 (HiveServer2-Background-Pool: Thread-373) [ERROR - org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:345)] Error running hive query: 
      org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
      	at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:330)
      	at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:254)
      	at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:91)
      	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:342)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
      	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:354)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

      2) Success run with following statement in unit test 

      statement.execute("CREATE table db_1.t2 as select tb1.id from db_1.tb1");
      
      2018-01-24 10:29:01,071 (c042d8d8-423b-4ddf-a50b-63f1dd3e3f50 HiveServer2-Handler-Pool: Thread-365) [INFO - org.apache.hadoop.hive.ql.Driver.compile(Driver.java:460)] Compiling command(queryId=jenkins_20180124102901_6376f6e1-0e74-4b05-b0a1-ab3ca5512e94): CREATE table db_1.t2 as select tb1.id from db_1.tb1
      2018-01-24 10:29:01,513 (HiveServer2-Background-Pool: Thread-372) [INFO - org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1713)] Executing command(queryId=jenkins_20180124102901_6376f6e1-0e74-4b05-b0a1-ab3ca5512e94): CREATE table db_1.t2 as select tb1.id from db_1.tb1
      2018-01-24 10:29:03,636 (HiveServer2-Background-Pool: Thread-372) [INFO - org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2021)] Completed executing command(queryId=jenkins_20180124102901_6376f6e1-0e74-4b05-b0a1-ab3ca5512e94); Time taken: 2.122 seconds
      OK

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                LinaAtAustin Na Li
                Reporter:
                LinaAtAustin Na Li
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: