I've got a few suggestions.
Method 1:
The default Path parameter in cmdlets are known to have problems with special characters. LiteralPath however should support all characters and often solves problems like the one you have.
Get-ChildItem 2012* | % { Remove-Item -LiteralPath $_.FullName -Force }
Method 2:
You can try to use the shortname (8.3 filename) for the folder. This is a cmd.exe approach. You could also wrap the two commands inside cmd /c " YOUR COMMAND " to run them in PowerShell.
D:\> dir /x
Volume in drive D is Storage
Volume Serial Number is *******
Directory of D:\
12.01.2014 12:29 <DIR> APPLEI~1 Apple iOS 7 GM
D:\> rd /s d:\APPLEI~1
d:\applei~1, Are you sure (Y/N)? y
Method 3:
You could also see if a WMI approach works:
#Remember to use \\ instead of \ in the path
$fold = Get-WmiObject -Query "select * from Win32_Directory where Name = 'C:\\holds bad subdir\\20120530-04'"
$fold
If this returns nothing, try adding a space at the end in the filename. If it returns an object, run:
$fold.Delete()
If it doens't return an object both with and without the space at the end, try the following apporach using wildcard (this can take everything from 1-15 minutes to run).
#Remember to use \\ instead of \ in the path
$fold = Get-WmiObject -Query "select * from Win32_Directory where Name like 'C:\\holds bad subdir\\20120530-04%'"
$fold
And delete it if it returns the correct folder:
$fold.Delete()
