Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-1787

Properly deal with limit clause in CubeHBaseEndpointRPC (SELECT * problem)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v1.5.2
    • Fix Version/s: v1.5.3
    • Component/s: None
    • Labels:

      Description

      what is the difference between "SELECT * FROM
      MEDIA_SNS.T_MAIL_POP3_LOGIN as T_MAIL_POP3_LOGIN limit 10“ and "SELECT
      count FROM MEDIA_SNS.T_MAIL_POP3_LOGIN as T_MAIL_POP3_LOGIN limit 10"?

      the second one run success quickly,and the first one failed with:
      2016-06-15 00:01:02,364 INFO [BadQueryDetector]
      service.BadQueryDetector:104 : Slow query has been running 10398.763 seconds
      (project:media_sns_mail, thread: 0x34) – SELECT * FROM
      MEDIA_SNS.T_MAIL_POP3_LOGIN as T_MAIL_POP3_LOGIN limit 10

      the regionserver showes error info like:
      2016-06-15 10:37:08,093 WARN
      [B.DefaultRpcServer.handler=54,queue=4,port=60020] ipc.RpcServer:
      (responseTooLarge):

      {"processingtimems":152140,"call":"ExecService(org. apache.hadoop.hbase.protobuf.generated.ClientProtos$CoprocessorServiceRequest)","client":"XXX.XXX.XXX.XXX:45264","starttimems":1465958075952,"queuetimems":0,"class":"HRe gionServer","responsesize":*494803440*,"method":"ExecService"}

      2016-06-15 10:37:08,645 WARN
      [B.DefaultRpcServer.handler=54,queue=4,port=60020] ipc.RpcServer:
      RpcServer.respondercallId: 93695 service: ClientService methodName: Exe
      cService size: 913 connection: XXX.XXX.XXX.XXX:45264: output error
      2016-06-15 10:37:08,648 WARN
      [B.DefaultRpcServer.handler=54,queue=4,port=60020] ipc.RpcServer:
      B.DefaultRpcServer.handler=54,queue=4,port=60020: caught a ClosedChanne
      lException, this means that the server XXX.XXX.XXX.XXX/XXX.XXX.XXX.XXX:60020
      was processing a request but the client went away. The error message was:
      null

      pleasse notice infomation about “responseTooLarge“ and
      "responsesize:494803440",but in my enviroment,i set(in kylin.properties):
      kylin.hbase.scan.cache_rows=10
      kylin.hbase.scan.max_result_size=102400

      my hbase is 0.98.16.1 and i notice HBASE-13527;and i also notice some
      difference between CubeHBaseScanRPC and CubeHBaseEndpointRPC

      CubeHBaseScanRPC has done something like buildScan(which call
      setMaxResultSize),while CubeHBaseEndpointRPC does not do things like this?

      may it be a problem?

      more to be added,the regionserver related with the second failing query
      will be down soon after dealing with this query,jvm gc happen frequentyl

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                liyang.gmt8@gmail.com liyang
                Reporter:
                mahongbin Hongbin Ma
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: