does anybody know if there is a good gem for mathematic / geometric operations? I need some of this things:
I can highly recommend the Ruby/GSL gem.
You may need to refer to the GSL manual to work out the name of the function you need to call though, if it's not documented in the Ruby/GSL documentation.
http://linalg.rubyforge.org/ seems to do what you want. It is based on LAPACK, the standard ultra tested 40-year linear algebra framework. Moreover, you can plug in vendor-specific LAPACK libraries for optimized code.
I am not sure if this may help: http://narray.rubyforge.org
NMatrix from the SciRuby project might be what you're looking for.
Linalg https://github.com/wedesoft , I posted instructions how to install it here: http://www.quora.com/Installation-Instructions/How-do-I-install-Ruby-linalg-library-on-Mac