I trigger an c# application by an custom action:
On failing condition, my application
Reading between the lines here, it appears your custom action launches an EXE. If that is so, there is no way to do what you ask. You could show a message from your EXE before returning a non-zero exit code, but then Windows Installer would still show the Error 1722 message.
If you can instead run a function from a DLL, you have more options. Instead of returning errors, you'd be able to set properties (assuming this is an immediate mode action), and could use those properties to do further things, such as show another dialog, or exit the installation without the Error 1722 message. I don't think all the necessary configuration options are available in the limited edition - you certainly cannot edit dialogs in LE - so to do all of that, you would have to change to a more capable tool (including the Professional edition, or options from other vendors).