Fortran was the first programming language I learned, since then I have also picked up C and some C++. My two cents is that if you need to quickly speed up some matrix processing, definitely go with Fortran. The reasons are:
Fortran is really good at efficiently processing numerical data, especially when it is stored in matrices or arrays. This sort of work is the 'sweet spot' of the language.
Because Fortran has a narrow focus on numerical operations, it has a lower learning curve compared to C and C++. There are fewer language features and quirks to learn and you don't have to deal with pointers. This is a big win if all you want to do is speed up some calculations as quickly as possible and move on with your work.
Multidimensional Arrays and array operations are first-class citizens in the Fortran language. With C or C++ you need to worry about using external libraries or writing functions/macros to provide the same functionality.
On the other hand, C and C++ are decidedly better suited for general purpose programming tasks outside the realm of numerical computation. If you see the possibility for something like lots of string manipulation in your future then you probably want to invest your time in a language other than Fortran.
Update
One other important consideration is how your data is stored and processed on the R side. If you use fortran then you will have to pass your data into the compiled routines in a very basic manner- scalars, vectors, etc. No lists or fancy objects.
Since R is implemented in C, there is a richer interface available that allows you to directly pass arbitrary R objects to C and C++ routines and then return arbitrary R objects. You can also execute callbacks that allow you to execute R functions from within the compiled C code.