Serialize execution of symstore via Powershell or BATCH

Question

We are working to integrate a step into our continuous integration (CI) server (CruiseControl.NET). We want to register the debug symbols *.pdb generated from

Answer 1

Use a file in the shared directory as semaphore to avoid concurrent executions.

:checkfile
if exist %cidir%\sem.txt goto :wait10secs
echo gotit! >%cidir%\sem.txt
doit
del %cidir%\sem.txt
goto :eof
:wait10secs
ing 192.0.2.2 -n 1 -w 10000 > nul
goto :checkfile

Be prepared for debugging all strange ways your batch can fail and all nasty racing conditions.

Answer 2

In order to lock access to the network drive, you'll need a third-party that both your CI services talk to. This third-party would then handle access to the network drive. This third party could be:

• MSMQ
• Row in a database table
• WCF service on symbol server which runs symstore and your CCNet builds talk to it to trigger symstore
• CCNet on the symbol server with a project/job that can be triggered remotely
• A scheduled job on the symbol server that can be triggered remotely from your CI servers

Answer 3

You can use a locked file as a simple semaphore to serialize events. When you redirect stdout to a file in a batch file, it establishes an exclusive write lock on that file. No other process can open up the same file for write access. The lock will automatically be released when the process finishes, no matter how it ends (clean exit, CTRL-C, Exception failure, etc)

A batch file can attempt to redirect 9 to a lock file, and if it fails, loop back until success. The symstore command is only run while the lock is in place. A non-standard file handle (stream?) is used so that the lock does not interfere with stdin, stdout, or stderr processing.

So you just need to make sure you never call symstore directly. Instead you always call it via a batch script. Something like the following (serializeSymstore.bat):

@echo off
setlocal

:loop

:: Save stderr definition and redirect stderr to nul
:: to hide possible redirection error when establishing lock.
8>&2 2>nul (

  %= Attempt to establish the lock and restore stderr =%
  9>"\\centralServer\somePath\symstore.lock" 2>&8 (

    %= If got here then lock is established throughout all commands =%
    %= in this set of parentheses.                                  =%

    %= Execute your command =%
    symstore %*

    %= Save the return code =%
    call set "rtnCd=%%errorlevel%%"

    %= The next command is a very fast way to clear the ERRORLEVEL. =%
    %= We don't want symstore failure to trigger a loop.            =%
    (call )
  )

) || (
  %= If entered here then failed to establish lock.                 =%
  %= Wait 1 second and then loop back to retry.                     =%
  %= Replace with PING delay if TIMEOUT not universally available.  =%
  timeout 1 /nobreak >nul
  goto loop
)

:: Exit with appropriate return code
exit /b %rtnCd%

Without comments, it becomes a tiny bit of code

@echo off
setlocal

:loop
8>&2 2>nul (
  9>"\\centralServer\somePath\symstore.lock" 2>&8 (
    symstore %*
    call set "rtnCd=%%errorlevel%%"
    (call )
  )
) || (
  timeout 1 /nobreak >nul
  goto loop
)
exit /b %rtnCd%

I have found this primitive and simple strategy to be extremely effective in many projects. I must confess that I have not tested the lock and release characteristics on remote machines. But I believe it should be reliable as long as all machines are Windows.

The only drawback I am aware of is that there is no FIFO queue. If multiple overlapping requests are received, then it's a random luck of the draw as to which process gets to go next. But the processes will be serialized.

EDIT:
I've read splattered bits' original answer prior to editing. He questions whether file locking is reliable on remote machines. I did some quick Google searches and there does appear to be some issues with file locking on UNC paths. If you run into problems, you may have better luck redirecting to a file on a mapped drive letter instead of directly through a UNC path. This is all theory - I've done no testing. Please be sure to do adequate testing before committing to this solution. Note that PUSHD is a convenient way to temporarily assign a drive letter to a UNC path without knowing what drive letters are available. POPD will unmap the drive assignment.