PowerShell Remove-Item return error even if set to SilentlyContinue

Question

I have a PowerShell script to execute a deployment of our application. At one point I want to delete a file to clean stuff up so I have this snippet :

try {