How to list all openssl ciphers available in statically linked python releases?

Question

In the python 2.7.8 to 2.7.9 upgrade, the ssl module changed from using

_DEFAULT_CIPHERS = \'DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2\'

to

Answer 1

Jan-Philip Gehrcke's answer requires an as-yet-unreleased version of python to be useful (see the comments), that make it not practical for answering the question about older versions of python. But this paragraph inspired me:

...you cannot call sslsock.shared_ciphers() before the socket is connected. Otherwise, Python's _ssl module does not create a low-level OpenSSL SSL object, which is needed to read the ciphers.

This got me thinking about a possible solution. All in the same python program:

• Create a server socket that accepts any cipher (ciphers='ALL:aNULL:eNULL').
• Connect to the server socket with a client socket configured with the cipher list we want to check (say 'DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2' if we want to test the default from python 2.7.8)
• Once the connection is established, examine the cipher that actually got chosen by the client and print it e.g. 'AES256-GCM-SHA384'. The client will choose the highest priority cipher from its configured cipher list that matches one supplied by the server. The server accepts any cipher and is running in the same python program with the same OpenSSL lib so the server's list is guaranteed to be a superset of the client's list. So the cipher used must be the highest priority one from the expanded list supplied to the client socket. Hooray.
• Now repeat, connecting to the server socket again but this time exclude the cipher that was chosen in the previous round, by appending the negation of it to the client socket's cipher list e.g. 'DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2:!AES256-GCM-SHA384')
• Repeat until the SSL handshake fails, because we've run out of ciphers.

Here is the code (also available as a github gist):

"""An attempt to produce similar output to "openssl ciphers -v", but for
python's built-in ssl.

To answer https://stackoverflow.com/q/28332448/445073
"""
from __future__ import print_function

import argparse
import logging
import multiprocessing
import os
import socket
import ssl
import sys

def server(log_level, queue):
    logging.basicConfig(level=log_level)
    logger = logging.getLogger("server")

    logger.debug("Creating bind socket")
    bind_sock = socket.socket()
    bind_sock.bind(('127.0.0.1', 0))
    bind_sock.listen(5)

    bind_addr = bind_sock.getsockname()
    logger.debug("Listening on %r", bind_addr)
    queue.put(bind_addr)

    while True:
        logger.debug("Waiting for connection")
        conn_sock, fromaddr = bind_sock.accept()
        conn_sock = ssl.wrap_socket(conn_sock,
                                    ssl_version=ssl.PROTOCOL_SSLv23,
                                    server_side=True,
                                    certfile="server.crt",
                                    keyfile="server.key",
                                    ciphers="ALL:aNULL:eNULL")

        data = conn_sock.read()
        logger.debug("Read %r", data)
        conn_sock.close()
    logger.debug("Done")

def parse_args(argv):
    parser = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    parser.add_argument("--verbose", "-v", action="store_true",
                        help="Turn on debug logging")
    parser.add_argument("--ciphers", "-c",
                        default=ssl._DEFAULT_CIPHERS,
                        help="Cipher list to test. Defaults to this python's "
                        "default client list")
    args = parser.parse_args(argv[1:])
    return args

if __name__ == "__main__":
    args = parse_args(sys.argv)

    log_level = logging.DEBUG if args.verbose else logging.INFO

    logging.basicConfig(level=log_level)
    logger = logging.getLogger("client")

    if not os.path.isfile('server.crt') or not os.path.isfile('server.key'):
        print("Must generate server.crt and server.key before running")
        print("Try:")
        print("openssl req -x509 -newkey rsa:2048 -keyout server.key -out server.crt -nodes -days 365  -subj '/CN=127.0.0.1'")
        sys.exit(1)

    queue = multiprocessing.Queue()
    server_proc = multiprocessing.Process(target=server, args=(log_level, queue))
    server_proc.start()
    logger.debug("Waiting for server address")
    server_addr = queue.get()

    chosen_ciphers = []
    try:
        cipher_list = args.ciphers
        while True:
            client_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            client_sock = ssl.wrap_socket(client_sock,
                                          ssl_version=ssl.PROTOCOL_SSLv23,
                                          ciphers=cipher_list)
            logger.debug("Connecting to %r", server_addr)
            client_sock.connect(server_addr)
            logger.debug("Connected")

            chosen_cipher = client_sock.cipher()
            chosen_ciphers.append(chosen_cipher)

            client_sock.write("ping")
            client_sock.close()

            # Exclude the first choice cipher from the list, to see what we get
            # next time.
            cipher_list += ':!' + chosen_cipher[0]
    except ssl.SSLError as err:
        if 'handshake failure' in str(err):
            logger.debug("Handshake failed - no more ciphers to try")
        else:
            logger.exception("Something bad happened")
    except Exception:
        logger.exception("Something bad happened")
    else:
        server_proc.join()
    finally:
        server_proc.terminate()

    print("Python: {}".format(sys.version))
    print("OpenSSL: {}".format(ssl.OPENSSL_VERSION))
    print("Expanding cipher list: {}".format(args.ciphers))
    print("{} ciphers found:".format(len(chosen_ciphers)))
    print("\n".join(repr(cipher) for cipher in chosen_ciphers))

Note how it defaults to testing the default cipher list built-in to python:

day@laptop ~/test
$ python --version
Python 2.7.8

day@laptop ~/test
$ python ssltest.py -h
usage: ssltest.py [-h] [--verbose] [--ciphers CIPHERS]

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Turn on debug logging (default: False)
  --ciphers CIPHERS, -c CIPHERS
                        Cipher list to test. Defaults to this python's default
                        client list (default:
                        DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2)

so we can easily see what the default client cipher list expands to, and how this changed from python 2.7.8 to 2.7.9:

day@laptop ~/test
$ ~/dists/python-2.7.8-with-pywin32-218-x86/python ssltest.py
Python: 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)]
OpenSSL: OpenSSL 1.0.1h 5 Jun 2014
Expanding cipher list: DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2
12 ciphers found:
('AES256-GCM-SHA384', 'TLSv1/SSLv3', 256)
('AES256-SHA256', 'TLSv1/SSLv3', 256)
('AES256-SHA', 'TLSv1/SSLv3', 256)
('CAMELLIA256-SHA', 'TLSv1/SSLv3', 256)
('DES-CBC3-SHA', 'TLSv1/SSLv3', 168)
('AES128-GCM-SHA256', 'TLSv1/SSLv3', 128)
('AES128-SHA256', 'TLSv1/SSLv3', 128)
('AES128-SHA', 'TLSv1/SSLv3', 128)
('SEED-SHA', 'TLSv1/SSLv3', 128)
('CAMELLIA128-SHA', 'TLSv1/SSLv3', 128)
('RC4-SHA', 'TLSv1/SSLv3', 128)
('RC4-MD5', 'TLSv1/SSLv3', 128)

day@laptop ~/test
$ ~/dists/python-2.7.9-with-pywin32-219-x86/python ssltest.py
Python: 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)]
OpenSSL: OpenSSL 1.0.1j 15 Oct 2014
Expanding cipher list: ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:ECDH+RC4:DH+RC4:RSA+RC4:!aNULL:!eNULL:!MD5
18 ciphers found:
('ECDHE-RSA-AES256-GCM-SHA384', 'TLSv1/SSLv3', 256)
('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1/SSLv3', 128)
('ECDHE-RSA-AES256-SHA384', 'TLSv1/SSLv3', 256)
('ECDHE-RSA-AES256-SHA', 'TLSv1/SSLv3', 256)
('ECDHE-RSA-AES128-SHA256', 'TLSv1/SSLv3', 128)
('ECDHE-RSA-AES128-SHA', 'TLSv1/SSLv3', 128)
('ECDHE-RSA-DES-CBC3-SHA', 'TLSv1/SSLv3', 112)
('AES256-GCM-SHA384', 'TLSv1/SSLv3', 256)
('AES128-GCM-SHA256', 'TLSv1/SSLv3', 128)
('AES256-SHA256', 'TLSv1/SSLv3', 256)
('AES256-SHA', 'TLSv1/SSLv3', 256)
('AES128-SHA256', 'TLSv1/SSLv3', 128)
('AES128-SHA', 'TLSv1/SSLv3', 128)
('CAMELLIA256-SHA', 'TLSv1/SSLv3', 256)
('CAMELLIA128-SHA', 'TLSv1/SSLv3', 128)
('DES-CBC3-SHA', 'TLSv1/SSLv3', 112)
('ECDHE-RSA-RC4-SHA', 'TLSv1/SSLv3', 128)
('RC4-SHA', 'TLSv1/SSLv3', 128)

And I think this answers my question. Unless anyone can see a problem with this approach?

Answer 2

You might want to have a look into openssl cipher's source code at https://github.com/openssl/openssl/blob/master/apps/ciphers.c

The crucial steps seem to be:

1. meth = SSLv23_server_method();
2. ctx = SSL_CTX_new(meth);
3. SSL_CTX_set_cipher_list(ctx, ciphers), whereas ciphers is your string
4. ssl = SSL_new(ctx);
5. sk = SSL_get1_supported_ciphers(ssl);
6. for (i = 0; i < sk_SSL_CIPHER_num(sk); i++) { print SSL_CIPHER_get_name(sk_SSL_CIPHER_value(sk, i)); }

The SSL_CTX_set_cipher_list function is called in Python 3.4 in _ssl's set_ciphers method for contexts. You can achieve the same using:

import socket
from ssl import SSLSocket
sslsock = SSLSocket(socket.socket(socket.AF_INET, socket.SOCK_STREAM))
sslsock.context.set_ciphers('DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2')

The next step would be calling SSL_get1_supported_ciphers() which, unfortunately, is not used in Python's _ssl.c. The closest you can get is the shared_ciphers() method of SSLSocket instances. The (current) implementation is

static PyObject *PySSL_shared_ciphers(PySSLSocket *self)
{
    [...]
    ciphers = sess->ciphers;
    res = PyList_New(sk_SSL_CIPHER_num(ciphers));
    for (i = 0; i < sk_SSL_CIPHER_num(ciphers); i++) {
        PyObject *tup = cipher_to_tuple(sk_SSL_CIPHER_value(ciphers, i));
        [...]
        PyList_SET_ITEM(res, i, tup);
    }
    return res;
}

That is, this loop is very similar as in the ciphers.c implementation above, and returns a Python list of ciphers, in the same order as the loop in ciphers.c would.

Continuing with the sslsock = SSLSocket(...) example from above, you cannot call sslsock.shared_ciphers() before the socket is connected. Otherwise, Python's _ssl module does not create a low-level OpenSSL SSL object, which is needed to read the ciphers. That is different from the implementation in ciphers.c, which creates a low level SSL object without requiring a connection.

That is how far I got, I hope that helps, and maybe you can figure out what you need based on these findings.