Is there a small library for RSA or DSA without any dependencies like GMP or OpenSSL? (Written in C or Asm)
You may find LibTomCrypt
useful. It's written in C, supports RSA and DSA (along with a host of other algorithms), and is public domain software. You can read about it here.
Okay I found myself one after searching not so common coder sites http://z0mbie.daemonlab.org/ it's under PGP/RSA-RELATED if someones interested. But are there any others?
Take a look at axTLS it's a BSD licensed TLSv1 SSL in C for embedded systems. It's built for POSIX sockets, but is easily ported.
The one thing it's missing is RSA key generation, but it can read X.509 certificates.
It's about 32KB of code space with a small RAM footprint.