This may be subjective, I don\'t know: I have this problem, which I\'m kind of equating to the \"what language for this project?\" question, since I can\'t seem to solve it.
You have to weigh the degree to which you can provide a cleaner notation against the fact that nobody will start out knowing the notation you use. In theory, you might be able to follow the notation this group uses in general so well that it's "intuitively obvious" -- but unless they're different from most physicists, that probably won't work well. Many use things like Greek and/or Hebrew characters that aren't present on a typical keyboard.
If you do use a DSL, are you prepared not only to implement it, but spend a good part of your life maintaining it? Do you have the resources to go beyond the language proper, and provide the kind of infrastructure for it to become a useful development tool? If it's going to be used for real development, it'll probably need a debugger. You might also want to consider either a new IDE (if it's really different) or tailoring an existing product to work (e.g. create a specialized mode for emacs or a plug-in for Eclipse).
If you do invent a DSL, how likely is it that people will really use it? If there's already something thoroughly entrenched, displacing it will probably be quite difficult. Creating a new language that fills a vacuum is a lot easier than replacing something that's already in use. On the other hand, if there really is a vacuum, this may indicate that there's just not much market -- your audience may not be particularly interested in becoming software developers.