The nature of RSA keys, and the similar algorithms which are used to generate key pairs (your private and public keys) makes it impossible to retrieve your keys if you lose them.
The passphrase is used to hash the private key so that its use is protected. Since all the password is used for during decryption is to unhash the private key, it cannot be used to derive the private key when it is lost.
The spec for how that works is in RFC 2437 section 7.1.2.