Useful Sticky Notes

Wednesday, May 10, 2017

Work-around for exec() differences between Python 2 and Python 3

Recently I've had to navigate a Python 2 vs 3 compatibility issue with regard to the changes in semantics (?) for the exec() call. I'll note here that I do not yet fully comprehend all of the deep issues and subtleties involved here, so if anyone could enlighten me please do so in the comments. I am also not certain that what I am implementing in order to achieve my goals is in fact a well-established (or even reasonable) coding idiom.

Anyway in a project I am working on with the OpenWorm Foundation, I encountered a scenario where it was beneficial to parameterize a oft-repeated section of code. In a GET request from Django, I'd loop through all of our supported field types, and invoke a function to process each field type if that field's signature shows up in the GET request.

The call to filter on a field in Django requires that the field type be a part of the code text - something like - filter(__icontains=searchString). The searchString term is not a problem since it refers to a variable name. myField however is part of the code text, and cannot be a variable string. So to achieve the parameterization I desired, I had to encapsulate the code in an exec() call like so - exec('filter(' + myField + '__icontains=searchString'));

A quick caveat - the above pseudo-code fragment will work just fine in both Python 2 and 3. It was used to build the context for my motivations for writing code of this nature. The real problem arises when I attempt to assign code of that nature to a local variable in a loop within the function, something akin to an accumulation operation. In Python 2's case, the code will work just fine as intended - the exec() call is treated as a in-place statement. In the case of Python 3, exec() is a function but there are some rules governing the way scoping works that I do not yet fully understand. Because of those rules, direct assignment to local function variables will not work.

To illustrate here is a code fragment I wrote which more or less captures the nature of what I was trying to achieve in the production code:

In this case, the output looks like this:


There is a weird bug where the number doesn't come out right in the "correct" Python 2 case when "*" was supplied to the code, but I don't think that should distract us from the main problem here. Python 2 will report the expected result, but Python 3 won't.

The workaround appears to be to assign into a construct like a list. Somehow exec() will allow mutable variables to be modified while properly scoped and referenced like in the following code:

Now Python 2 and Python 3 agrees on the output and behavior:

No comments:

Post a Comment