See this question:
C# GUI programming for beginners: Where to start?
Especially this answer there.
I think you should start with Windows Form, WPF is the new thing for Windows Dev and its getting all the lime light, but I would really not recommend starting off with it.
- Get Visual Studio 2010 Express
- See this MSDN page on Windows Forms
Programming Windows Forms by Charles Petzold is nice book (Charles is very cool) , windows forms hasn't changed all too much since Visual 2005, you should learn C# 4, the latest language.
Visual C# 2010 Step by Step is a good book to get a gist of the .NET world and all your options (including WPF)
See these SO questions:
- Beginners book for .NET and C#?
- https://stackoverflow.com/questions/4807574/best-c-programing-books-for-absolute-beginner-or-someone-just-starting-with-net
- https://stackoverflow.com/questions/52926/beginning-c
Hope that helps.
Update
Incase you were thinking of going with C or C++ for windows development, is it not easy. I've done some Win32 API and believe me you'll be writing seriously long/complicated programs for even simple things
You have two MS options, you can just use the C or C++ language and call the Win32 API functions.
This book is the Bible for that.
The other MS option is using MFC, people have some rather strong (bad) views about MFC. I haven't done much of it myself so can't say too much.
See : https://stackoverflow.com/questions/557114/stick-with-mfc-or-go-to-net
There are of course tons of non-MS options, which again I wouldn't recommend.
See : Native Windows Application Development Options
Bottom line, in my opinion, C# is a very well done language, you will get TONs of support here, Visual Studio is one of the best tools around, and you will have fun learning C#/.NET, and the biggest advantage is you can use your C# knowledge to write even Web Apps, Cloud and Mobile apps and lots more.