Check the attribute of each folder and file and then do a conditional based deletion. This is just the pseudo code.
If (-not (a readonly file)) {
delete file
}
So, to check if a given file or folder is readonly:
$item = Get-Item C:\Scripts\Test.txt
$item.IsReadOnly
HTH