Timeout for a loop in XQuery MarkLogic

Question

I have a big collection of about 1 million documents, in MarkLogic (version 9), with below structure:

<File>
    <Id></Id>
    <ModifiedAt></ModifiedAt>
    <Author></Author>
    <Title></Title>
</File>

And I need to iterate through entire collection and to replace space from ModifiedAt with T for all documents

Example of document:

     <File>
         <Id>12121</Id>
         <ModifiedAt>2011-06-08 14:29:29.000</ModifiedAt>
         <Author>Test</Author>
         <Title>Test</Title>
    </File>

Field ModifiedAt should become: 2011-06-08T14:29:29.000

The code looks like:

for $doc in fn:collection('File')
    return xdmp:node-replace($doc/File/ModifiedAt,<ModifiedAt>{fn:replace($doc/File/ModifiedAt,' ','T')}</ModifiedAt>)

The issue is that this code returns timeout.

I assume there is a more elegant way to make this modification on entire collection and maybe someone has a hint.

Thank you!

Answer 1

There are various external tools out there, like Corb2, and MLCP that can be used for this, but you can also do adhoc or less adhoc work from inside MarkLogic. All you essentially need to do is do your processing in batches. Taskbot is very useful for that:

https://github.com/mblakele/taskbot

HTH!