Is there any ways to try to guess encryption algorithm used to encrypt the ciphertext?
It depends if you're talking about "raw encrypted data" (in that case you can use methods such as listed by "gs" in the other answer) or an encrypted file in some standard format (the most common are CMS/PKCS#7 and OpenPGP); in the latter case the encryption algorithm is explicitly indicated in the metadata contained in the very file.
For CMS you need an ASN.1 decoder such as command-line dumpasn1 program or my own web-based Javascript decoder while for OpenPGP you can use pgpdump.