I followed this link to sign my exe application.
I installed SDK tool on Windows 7,
run C:\\Program Files\\Microsoft SDKs\\Windows
There are a couple of problems.
First of all you are using self-signed certificate, so you should define it explicitly by adding -r key to makecert command or you'll get an error "The signer's certificate is not valid for signing" at sign step.
Next, at this step
signtool.exe sign /f "App-O.pfx" /p fess "C:\Output\setup.exe"
you are trying to open pfx using password "fess". But you actually didn't set any password for pfx file. To do it you should add -po key to pfx creation command.
After that you can sign your application.
So the correct process will be:
makecert.exe -sv App-O.pvk -n "CN=MY DIGITAL KEY" App-O.cer -r
pvk2pfx.exe -pvk App-O.pvk -spc App-O.cer -pfx App-O.pfx -po fess
signtool.exe sign /f "App-O.pfx" /p fess "C:\Output\setup.exe"
Here is some useful links: