This goes hand-in-hand with, "Don't use exceptions to control program flow.", and, "Only use exceptions for exceptional circumstances." If these are done, then exceptions should only be occurring when there's a problem. And if there's a problem, you don't want to fail silently. In the rare anomalies where it's not necessary to handle the problem you should at least log the exception, just in case the anomaly becomes no longer an anomaly. The only thing worse than failing is failing silently.
