>>> import matplotlib
>>> print matplotlib.rcsetup.all_backends
[u\'GTK\', u\'GTKAgg\', u\'GTKCairo\', u\'MacOSX\', u\'Qt4Agg\', u\'Qt5Agg\', u\'TkAgg\
The backend mainly matters if you're embedding matplotlib in an application, in which case you need to use a backend (GTK, Qt, TkInter, WxWindows) which matches the toolkit you're using to build your application. If you're also using matplotlib in a simple interactive way, you'll also want to use a backend which matches what is available on your machine (GTK if you're running Gnome, Qt if you're running KDE, etc) (although most libs are already installed on most machines)
The drawing layer part of the backend (Cairo, Agg...) also matters in terms of functionalities: you can choose it depending on what that layer provides compared to what your application needs (anti aliasing, alpha channel, export formats...). So if you develop and test using TkAgg and other people run with e.g. TkCairo, some things might not work. OTOH, running with QtAgg would certainly work in a very similar way as long as you stick to the matplotlib API and don't reach in the wrapped toolkit layer.