The linux file command does a very good job in recognising file types and gives very fine-grained results. The diff tool is able to tell binary files f
A quick-and-dirty way is to look for a NUL character (a zero byte) in the first K or two of the file. As long as you're not worried about UTF-16 or UTF-32, no text file should ever contain a NUL.
Update: According to the diff manual, this is exactly what diff does.