Concurrent read/write access to XML

Question

I am trying to implement information updates in an XML file, by multiple processes running on multiple machines somewhat concurrently. My thinking is to loop for 10 minutes, try

Answer 1

On a general note: files aren't optimized for concurrent access the way databases, are, so if you need concurrent access with some sophistication, you'll need to roll your own.

This answer to a closely related question demonstrates use of a separate lock file (sentinel file) for managing concurrency with minimal disruption.

However, you can simplify the approach and obviate the need for a lock file if you're willing to put an exclusive lock on the file for the entire duration of reading it, modifying it, and saving the modifications.

By contrast, the lock-file approach allows reading and preparing modifications concurrently with other processes reading the file, and only requires the exclusive lock for the actual act of rewriting / replacing the file.

With both approaches, however, a period of exclusive locking of the file is required, so as to prevent the unpredictability of readers reading from a file while it is being rewritten.

That said, you still need cooperation from all processes involved:

• Writers need to deal with the (temporary) inability to open the file exclusively, namely while other processes (readers or writers) are using it.

• Similarly, readers must be prepared to handle the (temporary) inability to open the file (while it is being updated by a writer).

The key is to:

• Open the file with file-share mode None (i.e., deny other processes use of the same file while you have it open), and to keep it open until updating has completed. This ensures that the operation is atomic from a cross-process perspective.

• Use only the FileStream instance returned by [System.IO.File]::Open() to read from and write to the file (calling cmdlets or .NET methods such as System.Xml.XmlDocument.Save() will fail, because they themselves will try to open the - then exclusively locked - file).

---

Here's a fixed version of your code that implements exclusive locking:

$path = '\\Px\Support\Px Tools\Resources\jobs.xml'
foreach ($i in 1..10) {

    $sleepTime = get-random -minimum:2 -maximum:5

    # Open the file with an exclusive lock so that no other process will be
    # be able to even read it while an update is being performed.
    # Use a RETRY LOOP until exclusive locking succeeds.
    # You'll need a similar loop for *readers*.
    # Note: In production code, you should also implement a TIMEOUT.
    do {  # retry loop
      try {
        $file = [IO.File]::Open($path, 'Open', 'ReadWrite', 'None')
      } catch {
        # Did opening fail due to the file being LOCKED? -> keep trying.
        if ($_.Exception.InnerException -is [System.IO.IOException] -and ($_.Exception.InnerException.HResult -band 0x21) -in 0x21, 0x20) { 
          $host.ui.Write('.') # Some visual feedback
          Start-Sleep -Milliseconds 500 # Sleep a little.
          continue # Try again.
        }
        Throw # Unexpexted error -> rethrow.
      }
      break # Opening with exclusive lock succeeded, proceed below.
    } while ($true)


    # Read the file's content into an XML document (DOM).
    $xml = New-Object xml # xml is a type accelerator for System.XML.XMLDocument
    $xml.Load($file)

    # Modify the XML document.
    $newNode = $xml.createElement('Item')
    $newNode.InnerXml = "$id : $i : $sleepTime : $(Get-Date)"
    $null = $xml.DocumentElement.AppendChild($newNode)

    # Convert the XML document back to a string
    # and write that string back to the file.
    $file.SetLength(0) # truncate existing content first
    $xml.Save($file)

    # Close the file and release the lock.
    $file.Close()
}

---

As for what you tried:

$file = [IO.File]::Open($path, 'Open', 'ReadWrite', 'Read') opens the file in a manner that allows other processes read access, but not write access.

You then call $xml.Save($path) while $file is still open, yet that method call - which itself tries to open the file too - requires write access, which fails.

As shown above, the key is to use the same $file (FileStream instance used to open the file exclusively for updating the file.

Also note that calling $file.Close() just before $xml.Save($path) is not a solution, because that introduces a race condition where another process could open the file in the time between the two statements.