Edit:
It wasn't quite there last time I evaluated it, but there has been quite a bit of activity, so you should also consider the (free and open source) Math.NET Numerics.
In looking now, it seems they've finished their new version, and have added sparse matrix support, as well as other nice goodies.
If you want more robust support, you unfortunately really need to get into commercial packages for .NET atm.
There are two very feature-rich packages, both of which support matrices very well. Extreme Numerics works great, and has some very nice features. I've also heard very good things about the IMSL Visual Numerics math libraries.
