Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7256

Query over empty Hive tables fails, we will need to print heap usagePercent details in error message

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.15.0
    • Fix Version/s: None
    • Component/s: Execution - Flow
    • Labels:
      None

      Description

      The below query from Drill's web UI on Hive tables failed due to not enough heap memory to run this query.
      It fails intermittently from Drill web UI, and note that the two Hive tables used in the query are empty, meaning they have no data in them. The query does not fail when run from sqlline.

      The error message does not provide information about the usagePercent of heap.
      It will be useful to provide heap usagePercent information as part of the error message in QueryWrapper.java when usagePercent > HEAP_MEMORY_FAILURE_THRESHOLD

      Drill 1.15.0

      Failing query.

      SELECT a.event_id
       FROM hive.cust_bhsf_ce_blob a, hive.t_fct_clinical_event b
       where 
       a.event_id=b.event_id
       and a.blob_contents not like '%dd:contenttype="TESTS"%'
       and b.EVENT_RELATIONSHIP_CD='B'
      and b.EVENT_CLASS_CD in ('DOC')
      and b.entry_mode_cd='Web'
      and b.RECORD_STATUS_CD='Active'
      and b.RESULT_STATUS_CD ='Auth (Verified)'
      and substring(b.valid_until_dt_tm,1,10) >='2017-12-30'
      and substring(b.event_end_date,1,10) >='2018-01-01'
      

      Stack trace from drillbit.log

      2019-05-09 16:25:58,472 [qtp1934687-790] ERROR o.a.d.e.server.rest.QueryResources - Query from Web UI Failed
      org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: There is not enough heap memory to run this query using the web interface.
      
      Please try a query with fewer columns or with a filter or limit condition to limit the data returned.
      You can also try an ODBC/JDBC client.
      
      [Error Id: 91668f42-d88e-426b-b1fe-c0d042700500 ]
       at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) ~[drill-common-1.15.0.5-mapr.jar:1.15.0.5-mapr]
       at org.apache.drill.exec.server.rest.QueryWrapper.run(QueryWrapper.java:103) ~[drill-java-exec-1.15.0.5-mapr.jar:1.15.0.5-mapr]
       at org.apache.drill.exec.server.rest.QueryResources.submitQueryJSON(QueryResources.java:72) ~[drill-java-exec-1.15.0.5-mapr.jar:1.15.0.5-mapr]
       at org.apache.drill.exec.server.rest.QueryResources.submitQuery(QueryResources.java:87) ~[drill-java-exec-1.15.0.5-mapr.jar:1.15.0.5-mapr]
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_151]
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_151]
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_151]
       at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_151]
       at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:195) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jersey-common-2.8.jar:na]
       at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jersey-common-2.8.jar:na]
       at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jersey-common-2.8.jar:na]
       at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jersey-common-2.8.jar:na]
       at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jersey-common-2.8.jar:na]
       at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297) [jersey-common-2.8.jar:na]
       at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023) [jersey-server-2.8.jar:na]
       at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:372) [jersey-container-servlet-core-2.8.jar:na]
       at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:382) [jersey-container-servlet-core-2.8.jar:na]
       at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:345) [jersey-container-servlet-core-2.8.jar:na]
       at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:220) [jersey-container-servlet-core-2.8.jar:na]
       at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551) [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:544) [jetty-security-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.apache.drill.exec.server.rest.auth.DrillHttpSecurityHandlerProvider.handle(DrillHttpSecurityHandlerProvider.java:151) [drill-java-exec-1.15.0.5-mapr.jar:1.15.0.5-mapr]
       at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478) [jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.Server.handle(Server.java:462) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232) [jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534) [jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) [jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505]
       at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) [jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505]
       at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
      

        Attachments

          Activity

            People

            • Assignee:
              khfaraaz Khurram Faraaz
              Reporter:
              khfaraaz Khurram Faraaz
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: