The first requirement is intent: a programming language is a language intended for computer programming. Note that this is not quite a tautology; recall that it is possible to program a computer by setting the bits by hand.... Anyway, all the other attributes of a programming language follow from its intent.
I do think a "real" programming language should be Turing-complete. This excludes things like HTML, which direct computers and are called languages, but which are not programming languages.
With any classification, you will get boundary cases; e.g., didn't APL start out as a mathematical notation? However, I think that returning to intent will get you as close as you're likely to get: for those boundary cases, the interesting questions are less relevant to the classification, anyway.
Edit: so, as an aphorism, then:
A programming language is a language
intended or used for programming
computers.
A real programming
language is also Turing-complete.