Strange intermittent UUID bug

[Jerry Vinokurov]

Hi all,

We have a project that uses UUIDs as a primary key for some objects. Our
project is deployed on AWS via Elastic Beanstalk. After the initial deploy,
everything works fine, but at some point, we encounter the following error,
deep within Django:

ValidationError: ["'7c6eee47-53d0-48f6-a8b7-8aff82bc47c3' is not a valid
UUID."]

Now, that certainly is a valid UUID; just pass it as a parameter to
uuid.UUID to verify that. So this is definitely odd. We use Sentry for our
error logging, and I dove into the stack trace, which I'll post a picture
of:

[image: stack_trace.png]


As can be seen in the stack trace, the following thing seems to happen:

1. Line 2315 is supposed to check for whether the value is already a UUID.
That check fails (the purple line above indicates that the execution path
has reached that line). Note that below, Sentry gives value as a UUID
object.
2. Line 2316 is therefore invoked, calling self.to_python(value)

Here's the to_python function in its entirety (it can be found in
django/db/models/fields/__init__.py):

2322: def to_python(self, value):
2323: if value is not None and not isinstance(value, uuid.UUID):
2324: try:
2325: return uuid.UUID(value)
2326: except (AttributeError, ValueError):
2327: raise exceptions.ValidationError(
2328: self.error_messages['invalid'],
2329: code='invalid',
2330: params={'value': value},
2331: )
2332: return value

3. The isinstance check at 2323 also fails, and so uuid.UUID(value) is
invoked, which throws the ValidationError shown above, since the value
passed into the constructor is not a string.

Now, the really, really weird thing about this problem is that *it goes
away when we redeploy the code*. We then seem to operate normally for some
time (as short as half a day, as long as several weeks) and then the
problem crops up again. And on top of all of this, we can't seem to at all
reproduce it locally; it only shows up in the production environment on AWS.

This is bedeviling our entire team and I would love to know if anyone has
encountered a something like this or can provide any insight into just what
is happening here. It seems that it should not be possible and yet it's
happening.

[Jerry Vinokurov]

Addendum: I forgot to say that our version of Django is 2.1.3.

[Ryan Nowakowski]

It is suspicious that value passed in to get_db_prep_value is failing isinstance since it looks like value is already a UUID. Are you sure the UUID type of value is uuid.UUID?

Post by Jerry Vinokurov
Hi all,
We have a project that uses UUIDs as a primary key for some objects. Our
project is deployed on AWS via Elastic Beanstalk. After the initial deploy,
everything works fine, but at some point, we encounter the following error,
ValidationError: ["'7c6eee47-53d0-48f6-a8b7-8aff82bc47c3' is not a valid
UUID."]
Now, that certainly is a valid UUID; just pass it as a parameter to
uuid.UUID to verify that. So this is definitely odd. We use Sentry for our
error logging, and I dove into the stack trace, which I'll post a picture
[image: stack_trace.png]
1. Line 2315 is supposed to check for whether the value is already a UUID.
That check fails (the purple line above indicates that the execution path
has reached that line). Note that below, Sentry gives value as a UUID
object.
2. Line 2316 is therefore invoked, calling self.to_python(value)
Here's the to_python function in its entirety (it can be found in
2325: return uuid.UUID(value)
2327: raise exceptions.ValidationError(
2328: self.error_messages['invalid'],
2329: code='invalid',
2330: params={'value': value},
2331: )
2332: return value
3. The isinstance check at 2323 also fails, and so uuid.UUID(value) is
invoked, which throws the ValidationError shown above, since the value
passed into the constructor is not a string.
Now, the really, really weird thing about this problem is that *it goes
away when we redeploy the code*. We then seem to operate normally for some
time (as short as half a day, as long as several weeks) and then the
problem crops up again. And on top of all of this, we can't seem to at all
reproduce it locally; it only shows up in the production environment on AWS.
This is bedeviling our entire team and I would love to know if anyone has
encountered a something like this or can provide any insight into just what
is happening here. It seems that it should not be possible and yet it's
happening.
--
You received this message because you are subscribed to the Google
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/7f720fd9-8c4f-43a0-b084-e41f39d2305a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Zach]

You could try modifying to_python with something like this:

import logging


def to_python(self, value):
if value is not None and not isinstance(value, uuid.UUID):
try:
value = uuid.UUID(value)
except Exception as e:
logging.info('class: %s', e.__class__)
logging.info('module: %s', e.__module__)
logging.info('type(value): %s', type(value))
logging.info('value: %s', value)
raise exceptions.ValidationError(
self.error_messages['invalid'],
code='invalid',
params={'value': value},
)
return value

Post by Jerry Vinokurov
Hi all,
We have a project that uses UUIDs as a primary key for some objects. Our
project is deployed on AWS via Elastic Beanstalk. After the initial deploy,
everything works fine, but at some point, we encounter the following error,
ValidationError: ["'7c6eee47-53d0-48f6-a8b7-8aff82bc47c3' is not a valid
UUID."]
Now, that certainly is a valid UUID; just pass it as a parameter to
uuid.UUID to verify that. So this is definitely odd. We use Sentry for
our error logging, and I dove into the stack trace, which I'll post a
[image: stack_trace.png]
1. Line 2315 is supposed to check for whether the value is already a UUID.
That check fails (the purple line above indicates that the execution path
has reached that line). Note that below, Sentry gives value as a UUID
object.
2. Line 2316 is therefore invoked, calling self.to_python(value)
Here's the to_python function in its entirety (it can be found in
2325: return uuid.UUID(value)
2327: raise exceptions.ValidationError(
2328: self.error_messages['invalid'],
2329: code='invalid',
2330: params={'value': value},
2331: )
2332: return value
3. The isinstance check at 2323 also fails, and so uuid.UUID(value) is
invoked, which throws the ValidationError shown above, since the value
passed into the constructor is not a string.
Now, the really, really weird thing about this problem is that *it goes
away when we redeploy the code*. We then seem to operate normally for
some time (as short as half a day, as long as several weeks) and then the
problem crops up again. And on top of all of this, we can't seem to at all
reproduce it locally; it only shows up in the production environment on AWS.
This is bedeviling our entire team and I would love to know if anyone has
encountered a something like this or can provide any insight into just what
is happening here. It seems that it should not be possible and yet it's
happening.

[Tim Graham]

See if
https://groups.google.com/d/msg/django-users/ZGb8ofw1Ux8/TnMdqsj4AgAJ helps.

Post by Zach
import logging
value = uuid.UUID(value)
logging.info('class: %s', e.__class__)
logging.info('module: %s', e.__module__)
logging.info('type(value): %s', type(value))
logging.info('value: %s', value)
raise exceptions.ValidationError(
self.error_messages['invalid'],
code='invalid',
params={'value': value},
)
return value

Post by Jerry Vinokurov
Hi all,
We have a project that uses UUIDs as a primary key for some objects. Our
project is deployed on AWS via Elastic Beanstalk. After the initial deploy,
everything works fine, but at some point, we encounter the following error,
ValidationError: ["'7c6eee47-53d0-48f6-a8b7-8aff82bc47c3' is not a valid
UUID."]
Now, that certainly is a valid UUID; just pass it as a parameter to
uuid.UUID to verify that. So this is definitely odd. We use Sentry for
our error logging, and I dove into the stack trace, which I'll post a
[image: stack_trace.png]
1. Line 2315 is supposed to check for whether the value is already a
UUID. That check fails (the purple line above indicates that the execution
path has reached that line). Note that below, Sentry gives value as a
UUID object.
2. Line 2316 is therefore invoked, calling self.to_python(value)
Here's the to_python function in its entirety (it can be found in
2325: return uuid.UUID(value)
2327: raise exceptions.ValidationError(
2328: self.error_messages['invalid'],
2329: code='invalid',
2330: params={'value': value},
2331: )
2332: return value
3. The isinstance check at 2323 also fails, and so uuid.UUID(value) is
invoked, which throws the ValidationError shown above, since the value
passed into the constructor is not a string.
Now, the really, really weird thing about this problem is that *it goes
away when we redeploy the code*. We then seem to operate normally for
some time (as short as half a day, as long as several weeks) and then the
problem crops up again. And on top of all of this, we can't seem to at all
reproduce it locally; it only shows up in the production environment on AWS.
This is bedeviling our entire team and I would love to know if anyone has
encountered a something like this or can provide any insight into just what
is happening here. It seems that it should not be possible and yet it's
happening.