【问题】Thecoprocessorthreadsto

Kylin执行查询语句的时候报错如下:

成都创新互联是一家专业提供浦东企业网站建设,专注与做网站、成都网站设计、H5开发、小程序制作等业务。10年已为浦东众多企业、政府机构等服务。创新互联专业网站建设公司优惠进行中。

Error while executing SQL "select t.hotel_id_m,t.live_dt, d.day_of_week,sum(rns) from tableT t join TableD d on t.live_dt = d.daY_no group by t.hotel_id_m,t.live_dt, d.day_of_week LIMIT 50000": <sub-thread for Query ac580b70-96f2-403a-a64a-0557e599d35f GTScanRequest 143ae1ba>The coprocessor thread stopped itself due to scan timeout or scan threshold(check region server log), failing current query...

查看regionserver日志

2017-03-20 11:10:05,436 INFO  [Query dc7017bb-fefc-4177-a2c9-5842625beb89-109] endpoint.CubeVisitService: Scanned 9999001 rows from HBase.

2017-03-20 11:10:05,454 INFO  [Query dc7017bb-fefc-4177-a2c9-5842625beb89-109] endpoint.CubeVisitService: The cube visit did not finish normally because scan num exceeds threshold

org.apache.kylin.gridtable.GTScanExceedThresholdException: Exceed scan threshold at 10000001

at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService$1.hasNext(CubeVisitService.java:258)

at org.apache.kylin.storage.hbase.cube.v2.HBaseReadonlyStore$1$1.hasNext(HBaseReadonlyStore.java:111)

at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:290)

at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4117)

at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797)

at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982)

at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964)

at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652)

at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)

at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)

at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)

at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)

2017-03-20 11:10:05,459 WARN  [RpcServer.FifoWFPBQ.default.handler=59,queue=5,port=60020] ipc.RpcServer: (responseTooSlow): {"call":"ExecService(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$CoprocessorServiceRequest)","starttimems":1489979376495,"responsesize":359,"method":"ExecService","processingtimems":28964,"client":"10.10.16.102:58720","queuetimems":1,"class":"HRegionServer"}

分析解决:

在${KYLIN_HOME}/conf/kylin.property配置文件中有一个配置项

kylin.query.scan.threshold=10000000

这个配置项限制了Hbase scan的行数,当scan超过kylin.query.scan.threshold行的时候还是不能满足查询需求,则kylin取消hbase端的查询。

目前暂时有两种方式解决

1、增加kylin.query.scan.threshold值,这相应也会增加hbase压力,在Hbase能够顶住查询压力的情况下,可以增大该值

2、此次查询语句中有LIMIT 50000,Hbase在scan了kylin.query.scan.threshold行之后,让不能扫描出50000行。可以减小LIMIT的值,在Hbase在scan到达kylin.query.scan.threshold之前,就能满足查询要求。则不会出现这个问题

本文题目:【问题】Thecoprocessorthreadsto
网页链接:https://www.cdcxhl.com/article0/jocsio.html

成都网站建设公司_创新互联,为您提供网页设计公司电子商务企业建站外贸建站网站制作用户体验

广告

声明:本网站发布的内容(图片、视频和文字)以用户投稿、用户转载内容为主,如果涉及侵权请尽快告知,我们将会在第一时间删除。文章观点不代表本网站立场,如需处理请联系客服。电话:028-86922220;邮箱:631063699@qq.com。内容未经允许不得转载,或转载时需注明来源: 创新互联

绵阳服务器托管