No one has mentioned anonymous delegates. You can create delegates on the fly, without declaring them:
public void Action(Func func);
...
Action(delegate(int x) { return x*x; });
Which is just a more verbose version of the lambda syntax:
Action(x => x*x);
Also note that the lambda syntax has more aggressive type inference. Another difference is that the lambda notation can be used to declare expression trees:
public void Action(Expression>);
Action(x => x*x);
In which case what you get is not a function but a parse tree that you can examine at runtime. This is how linq queries build their sql, for example.
edit
To more directly answer the question of when to use one or the other:
You rarely need to declare a new delegate type yourself, although it is occasionally helpful. The framework provides several Func<> types, along with Action and Predicate which are usually all that you need.
When creating a function 'on the fly', there is no advantage to using the anonymous delegate syntax instead of the lambda syntax. Since the lambda syntax is more concise and type-inferred, prefer it.
