Execute MLCP Content Load Command as a schedule task in Marklogic

大城市里の小女人 提交于 2019-12-24 04:35:10

问题


Is there any possible way to bulk load data using MLCP as a scheduled task in Marklogic


回答1:


You can't invoke mlcp via a scheduled task; I recommend trying something like Apache Camel for this.

Camel has a Timer component and a Quartz component, either of which can be used for scheduling.

And here's an example Camel file with a route (commented out, but still operable) that is initiated by a Timer which then writes a file to disk and ingests it via mlcp - https://github.com/rjrudin/ml-camel-client/blob/master/src/main/resources/META-INF/camel-routes.xml .

I've had good success with doing all kinds of processing/scheduling in Camel and then ultimately ingesting content via mlcp. I think it's a good fit for your use case here so you can leverage what mlcp does best - get content into MarkLogic as fast as possible.




回答2:


Scheduled tasks inside MarkLogic can call external services (using HTTP), but they don't have a way to run an external command. You do have some options:

  • schedule the MLCP job externally, using cron on Linux or something along those lines;
  • restructure your load using JavaScript or XQuery; you can retrieve data from a file system, run it through some transforms, and insert it into the database using modules running in MarkLogic;
  • set up a Java app server, have your scheduled task make an HTTP request to that server and have the Java app server call MLCP

I think I'd start with the first option, but which one is best depends on your use case.



来源:https://stackoverflow.com/questions/29696689/execute-mlcp-content-load-command-as-a-schedule-task-in-marklogic

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!