Apache Drill - First start time is high

↘锁芯ラ 提交于 2019-12-11 15:42:54

问题


I am running SQL on MongoDB backend using Drill. I am getting response time ~500ms. But most of that time is spent on "First start" phase. Actual processing in drill is taking much less time(~50ms). Why does "First start" take so much time? I would like to know what drill in doing in that phase and if possible optimise that.

Fragment profile

Operator profile


回答1:


After the first query Drill creates a lot of cache objects to improve the further work, see Generated Code Cache [1], [2], [3] for details, StoragePluginRegistry Cache [4]. Also MongoSchemaFactory creates cache for your database and tableName [5], which expires after 1 minute.

Most likely you will get the similar time, if you try to execute your query, when the cache expires.

[1] https://github.com/paul-rogers/drill/wiki/CG-Code-Cache
[2] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/compile/ByteCodeLoader.java#L36
[3] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/compile/CodeCompiler.java#L145
[4] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistryImpl.java#L90
[5] https://github.com/apache/drill/blob/master/contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/schema/MongoSchemaFactory.java#L66



来源:https://stackoverflow.com/questions/52759650/apache-drill-first-start-time-is-high

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!