tornado_mysql: round to the nearest full microsecond

[xs]

After we convert the microseconds string to a float and multiply by 1e6, it's possible that we lose a microsecond when casting to an integer again:

Example:

In [1]: usecs = float('0.524226') * 1e6

In [2]: usecs
Out[2]: 524225.99999999994

In [3]: int(usecs)
Out[3]: 524225

Test code:

old_errors = 0
new_errors = 0

for expected in range(1000000):
    usecs_string = '0.' + str(expected).zfill(6)

    usecs = float(usecs_string) * 1e6

    old_actual = int(usecs)
    new_actual = int(round(usecs))

    if old_actual != expected:
        old_errors += 1

    if new_actual != expected:
        new_errors += 1


print 'old_errors: ' + str(old_errors)
print 'new_errors: ' + str(new_errors)

Test output:

old_errors: 11549
new_errors: 0

cc @V0idmain

[dangnvang]

I haven't checked yet, but is usecs a whole number microseconds (before the round bit)? If so, seems like it could be simpler to just use that directly (and/or convert it to int from string)?

[xs]

You may be right. I haven't checked either!

I just wanted to leave the smallest footprint that passes for all million possible microsecond test cases.

[agustinavarela]

@dangnvang

i think that any of the changes should be fine, i added a new scenario and it looks ok as well 🙂

This test code:

new2_errors = 0

for expected in range(1000000):
    usecs_string2 = round(float('0.' + str(expected).zfill(6)) * 1e6)

    if usecs_string2 != expected:
        new2_errors += 1

print 'new2_errors: ' + str(new2_errors)

Throws this output

new2_errors: 0

So i think both cases are ok, what do you think?

[dangnvang]

@agustinavarela:
I was thinking more like usecs = int(usecs) instead of usecs = round(float('0.' + usecs) * 1e6)

Looking at the rest of the code, it looks like this is the conversion from MySQL datetimes to Python. The MySQL datetime format, when returned with microseconds (microseconds is the only fractional second supported it looks like), always has 6 numbers after the decimal point, so I think we're safe to go with using the more simple int(usec) conversion method.

Regardless of which one we go with, this change should be tested with an actual DB where you write a datetime and read it back, ensuring that the microseconds portion of it is correctly converted in and out of MySQL.

[dangnvang]

So I did some testing and determined that the MySQL documentation is a little misleading in some places.

In actuality, datetimes do have a fractional precision that you can specify for the number of decimal places to use. Thus, with testing actual use of the library, the int(usec) approach will not work because it's not always 6 characters as I previously inferred from other parts of the documentation.

Keeping it as round(float('0.' + usecs) * 1e6) should be ok!

Sorry for the confusion!

[agustinavarela]

@dangnvang thanks so much for checking that! :)