SqlAlchemy mysql millisecond or microsecond precision

Question

I've been venturing down the odyssey of trying to get fractional time resolution working properly in my database. I use the python method datetime.now() in order to create date objects. I then store these objects in a field which is mapped to a COLUMN(DATETIME(9)) which is from SqlAlchemy's library. Originally, I was getting an error that my data was being truncated. This is because I was using mysql 5.5. I have since updated to 5.6.19 and no longer get the data truncated error.

However, the database still does not actually contain fractional time entries. For example, here is the value from when the datetime.now() object is instantiated:

2015-04-17 16:31:12.804444

The above is exactly what I would expect. The object in memory has microsecond resolution. Now, after it saves this to the mysql database, I see the following value if I open the mysql command line client and return the row using a select statement:

2015-04-17 16:31:13

Obviously, the value is being rounded to the nearest second. This is bad, and I have no idea what is causing it!

In case it is relevant, I'm using mysql-connector-python==2.0.3

UPDATE: I also tried using COLUMN(DATETIME(6)), but got the same behavior.

I am below including the model, in case that information is relevant:

class User(Base):
        __tablename__ = 'Users'

        uid = Column(INT, primary_key=True, autoincrement=True)
        dateCreated = Column(DATETIME(6))

        def __init__(self, *args, **kwargs):
                super(user, self).__init__(*args, **kwargs)
                self.dateCreated = datetime.now()

UPDATE: Pedro's suggestion wasn't the problem, though it definitely helped me make progress, so big thanks. I tried stepping through code in the sql connector until I got to the mysql insert statement. The statement does indeed contain the fractional time value. However, when executed, the value is rounded. When I did a describe on the table, I noticed that the datetime type is just that, datetime when it should really be datetime(6).

I'm generating the database itself using the SA model,which explicitly declares Column(DATETIME(6)) , and Base.metadata.create_all(self.db, checkfirst=True) , so I don't understand why the (6) isn't ending up in the actual table structure. I think I'll figure it out shortly though, and I'll post an update when I do.

UPDATE: The constructor to DATETIME does not accept field length specification. It only takes an argument for timezones. It isn't clear to me how to specify the length of a datetime field, since for types like varchar one would just pass it in to the constructor. The dive continues.

Answer 1

The problem I was having is that the stock SqlAlchemy DATETIME class does not work with the mysql requirement of passing a (6) value into the constructor in order to represent fractional time values. Instead, one needs to use the sqlalchemy.dialects.mysql.DATETIME class. This class allows the use of the fsp parameter (fractional seconds parameter.) So, a column declaration should actually look like:

dateCreated = Column(DATETIME(fsp=6)) 

Thanks to those others who replied. It helped guide my investigation to ultimately stumbling over this esoteric and very confusing distinction.

Answer 2

This seems to be an open issue with MySQLdb, not MySQL or SQLAlchemy.

http://sourceforge.net/p/mysql-python/feature-requests/24/

You can try using another MySQL driver library -- check SQLAlchemy documentation for supported options --, or you can try the monkeypatch suggested in the open issue.

Answer 3

According to the MySQL documentation, it only supports precision up to 6 places (microseconds):

"MySQL 5.6.4 and up expands fractional seconds support for TIME, DATETIME, and TIMESTAMP values, with up to microseconds (6 digits) precision"

I don't know for sure, but specifying 9 digits might be reverting the column to a smaller default precision.