The co-founder of Reddit gave a presentation on issues they had while scaling to millions of users. A summary is available here.
What surprised me is point 3:
We worked in a similar problem not long ago, i could say at first it wasn't easy and fun, but after some point that you get used to it, it has it's own benefit, it's like developing another Database withing your tables, in some area it's an overkill task, but when you pass these levels, it provide you with lots of functionality, basically after one point, we didn't create any new table, and we just created dynamic forms for everything's, even for our own programming tasks. as for performance, system didn't get million of rows to be fair comparison, but for daily usage i never noticed any differences. some problems i want to share.
3.you should keep your indexes up to date, but you should skip of building them whenever possible (again nightly service keep index up to date)
we tried hard no to jump in ocean of rows to calc some values by user demand. if we prepared it you can use it, if not then you cant
at the end, there are so many unique challenges to this approach that you should find a way to solve, problems you never faced before in your routine job, but after all of these you earn more flexibility that you can spend.