Here is my take on this question.
Firstly, I'll tackle this bit:
"I recently found out that most of the major websites fail W3C's markup and CSS validation tests. Therefore [...]"
Big companies can often suffer from paralysis, which makes it very hard to get even small changes made. In most cases, the practices of "big companies" are not a good example to follow. They are not "big companies" because they have a poorly constructed website - its the other way around. They become big, and to make a change to their website to fix a validation error becomes a longer and longer process that costs more and more money (with long lists of people who need to sign off the changes!)
Now let's move onto the next bit, without the baggage of big companies to worry about!
"[...] how important is it really to follow web standards?"
This can be answered in two contexts. I think the first paragraph probably applies to you as you are asking on Stack Overflow :)
If you are a professional web developer, you should always follow the standards unless explicitly requested not to do so (see 1)). The reasons you would do this are that being able to validate your mark-up, accessibility and browser-compatibility are part of the craft of creating professional websites.
If you are a subject-enthusiast (for example, an expert on rare insects) - it should be possible for you to publish a web page that contains terrible mark-up and browsers should all treat the errors in the same way - at the very least, everyone should be able to read the information. I mention this because the web is open to anyone to publish a web page - not just expert developers.
1) "explicitly requested not to do so" ? What on Earth does that mean. It means that it is possible to create a website that works in all browsers without following the standards. Google very deliberately use some very strange mark-up on their search page as discussed in this article:
http://www.stevefenton.co.uk/Content/Blog/Date/201008/Blog/Google-Deliberately-Write-Awful-HTML/