[KYLIN-1787] Properly deal with limit clause in CubeHBaseEndpointRPC (SELECT * problem)

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v1.5.2
• Fix Version/s: v1.5.3
• Labels: newbie
• Assignee: liyang
• Reporter: Hongbin Ma
• Created: 15/Jun/16 13:18
• Updated: 03/Aug/16 04:45
• Resolved: 24/Jun/16 03:22

Description

what is the difference between "SELECT * FROM
MEDIA_SNS.T_MAIL_POP3_LOGIN as T_MAIL_POP3_LOGIN limit 10“ and "SELECT
count FROM MEDIA_SNS.T_MAIL_POP3_LOGIN as T_MAIL_POP3_LOGIN limit 10"?

the second one run success quickly,and the first one failed with：
2016-06-15 00:01:02,364 INFO [BadQueryDetector]
service.BadQueryDetector:104 : Slow query has been running 10398.763 seconds
(project:media_sns_mail, thread: 0x34) – SELECT * FROM
MEDIA_SNS.T_MAIL_POP3_LOGIN as T_MAIL_POP3_LOGIN limit 10

the regionserver showes error info like:
2016-06-15 10:37:08,093 WARN
[B.DefaultRpcServer.handler=54,queue=4,port=60020] ipc.RpcServer:
(responseTooLarge):

{"processingtimems":152140,"call":"ExecService(org. apache.hadoop.hbase.protobuf.generated.ClientProtos$CoprocessorServiceRequest)","client":"XXX.XXX.XXX.XXX:45264","starttimems":1465958075952,"queuetimems":0,"class":"HRe gionServer","responsesize":*494803440*,"method":"ExecService"}

2016-06-15 10:37:08,645 WARN
[B.DefaultRpcServer.handler=54,queue=4,port=60020] ipc.RpcServer:
RpcServer.respondercallId: 93695 service: ClientService methodName: Exe
cService size: 913 connection: XXX.XXX.XXX.XXX:45264: output error
2016-06-15 10:37:08,648 WARN
[B.DefaultRpcServer.handler=54,queue=4,port=60020] ipc.RpcServer:
B.DefaultRpcServer.handler=54,queue=4,port=60020: caught a ClosedChanne
lException, this means that the server XXX.XXX.XXX.XXX/XXX.XXX.XXX.XXX:60020
was processing a request but the client went away. The error message was:
null

pleasse notice infomation about “responseTooLarge“ and
"responsesize:494803440",but in my enviroment,i set（in kylin.properties）：
kylin.hbase.scan.cache_rows=10
kylin.hbase.scan.max_result_size=102400

my hbase is 0.98.16.1 and i notice HBASE-13527;and i also notice some
difference between CubeHBaseScanRPC and CubeHBaseEndpointRPC

CubeHBaseScanRPC has done something like buildScan(which call
setMaxResultSize),while CubeHBaseEndpointRPC does not do things like this?

may it be a problem?

more to be added,the regionserver related with the second failing query
will be down soon after dealing with this query,jvm gc happen frequentyl

Issue Links

• contains: KYLIN-1936 Improve enable limit logic (exactAggregation is too strict)
• is related to: KYLIN-1792 behaviours for non-aggregated queries