Published on

Helpful Python Tricks

Authors

You may already know the bare essentials of Python. However, one of the benefits of the language is that it has an enormous toolkit that we'll explore below.

Advanced Data Structures

Odds are, you know about Python lists and dicts. But there's a couple other really useful ones out there.

Tuples

Most of you have heard of this. Tuples are immutable collections - once you make them, you can't change them.

a = (1,2,3)
a[0] = 4

TypeError: 'tuple' object does not support item assignment

Why are tuples useful? Two reasons:

  1. Immutability is good sometimes

  2. It saves space.

Sets

Sets are kind of like dictionaries without values. They work the same way under the hood (with a hashmap) but keys don't store anything.

It's very useful to keep track of unique things that happen.

a = set([1, 2, 1])
print a
b = {2, 3}  # shorthand for set initialization
print a & b
print a | b

set([1,2])
set([2])
set([1,2,3])

You can also use frozenset, which is immutable like a tuple.

collections

The collections library has a bunch of useful ones:

  • namedtuple() - make a tuple with predefined named fields. Great for things like SQLAlchemy rows.
  • deque - a simple Python queue
  • Counter - counts things. Kind of like a set for counting.
  • OrderedDict - dict that remembers order
  • defaultdict - you can define a default thing to get in your dictionary

Generators

They're pretty great:

def fibonacci():
    a = 0
    b = 1
    while True:
        yield b
        b = a + b
        a = b - a

for num in fibonacci():
    if num > 5:
        break
    print(num)

1
1
2
3
5

Generators start again at the previous place when their called again. If the generator function actually returns, it raises a StopIteration and the generator is done.

They're good because they don't require much memory. That's why they recommend you use xrange instead of range.

Variable swapping and decoupling

This is cool:

a, b = 1, 2
b, a = a, b

This is cooler:

a, b, c = [1,2,3]

This is coolest:

for i, number in enumerate(range(10)):
    print i, number

Neato chained comparisons

(1 < 2 and 2 < 3) == (1 < 2 < 3)

Comprehensions

List/dict/tuple/set comprehensions are amazing. Want to make a simple list?

print [x * 2 for x in xrange(5)]  # list
print (x * 2 for x in xrange(5))  # generator
print {x * 2 for x in xrange(5)}  # set
print {x: x * 2 for x in xrange(5)}  # dict

[0, 2, 4, 6, 8]
<generator object <genexpr> at 0x10df2dcd0>
set([0, 8, 2, 4, 6])
{0: 0, 1: 2, 2: 4, 3: 6, 4: 8}

Easy. And you can add filters:

[x for x in xrange(10) if x % 2 == 0]  # Even numbers

And do double loops!

[x for x in range(5) for y in range(5)]

Neat-o gang. It saves a lot of LOC. I use it frequently in API calls to generate JSON blobs.

q = session.query(Creatives)
return [
    {
        'md5': c.md5,
        'url': generate_cloudfront_url(c),
        'brand_id': c.brand().brand_id
    } for c in q
]

See? Now they don't look so scary.

Common Python gotchas

Credit for most of this section goes to the Python Guide.

Mutable default arguments

This is a classic Python interview question by pedantic people. What do you think will happen when I run this?

def a(b, c=[]):
    c.append(b)
    return c

print a(1)
print a(2)
print a(3)

Huh?

[1]
[1,2]
[1,2,3]

Not what you expected right? Remember that functions are made before the code gets run. So the same list stays.

Instead of that, you should always do:

def a(b, c=None):
    if c is None:
        c = []
    c.append(b)
    return c

Don't use mutable objects as default values (dicts, lists, etc). You can use strings and ints because they aren't mutable.

Closures are weird

Here, we make lambdas in the comprehension:

def create_multipliers():
    return [lambda x : i * x for i in range(5)]

print [multiplier(2) for multiplier in create_multipliers()]

You'd expect the result to be 0,2,4,6,8 but you actually get 8,8,8,8,8! Why? Because the closure is generated at run time and we leave our loop with i as 8.

After all, you'd expect this:

a = 1
def print_a():
    print a
a = 2
print_a()

To print 2, correct?

Instead of this, use default values:

def create_multipliers():
    return [lambda x, i=i : i * x for i in range(5)]

Classes inheritance is weird

This one comes from Martin Chikillian.

class A(object):
    x = 1
class B(A):
    pass
class C(A):
    pass
print A.x, B.x, C.x

1 1 1

Sure.

B.x = 2
print A.x, B.x, C.x

1 2 1

K.

A.x = 3
print A.x, B.x, C.x

3 2 3

Wat?

When you reference an object's attribute, it'll first try to get it from its own definition. After that, it'll go up and get it from its parents. This is called "Method Resolution Order (MRO)".

Local/global variables are weird

a = 0
def b():
    a += 1
b()

UnboundLocalError: local variable 'a' referenced before assignment

The moment you assign something, it creates it in the local scope. So a += 1 turns into a = a + 1 and it doesn't know what the second a is.

To do this properly:

a = 0
def b():
    global a
    a += 1
b()

is vs ==

Remember: is will only be true if two things are the same object. == will call the object's __eq__ method, which you can override:

class EverythingIsTrue(object):
    def __eq__(self, other):
        return True
print EverythingIsTrue() == False  # Will return true

There are a ton of other magic functions like __le__, __lte__, __add__ that affect other operators like <, <=, +.

iPython is great

It remembers your history, does tab completion, give information about objects, has magic pasting.

It even has magic functions like %timeit:

%timeit range(1000)
100000 loops, best of 3: 7.76 us per loop

Or %run, which runs external scripts.

Context Managers

Context managers are cool if you need to make sure something gets cleaned up:

class MyContextManager(object):
    def __enter__(self):
        self.b = 2
        return self
    def __exit__(self, type, value, traceback):
        self.b = None
with MyContextManager() as mc:
    print mc.b
print mc.b

2
None

You can do other cool things with contextlib:

from contextlib import contextmanager

@contextmanager
def tag(name):
    print "<%s>" % name
    yield
    print "</%s>" % name

with tag('Lol'):
    print 'hi'

<Lol>
hi
</Lol>

List slicing and dicing

Python's slice notation is the bomb. Get the last element by using -1: mylist[-1].

You can also get slices this way: mylist[0:5] or mylist[:5] or mylist[5:].

You can even define the "jumps": myrange(10)[::2] gets even numbers. A good trick to reverse lists is to do myrange(10)[::-1] (although you should probably use reversed() instead!).

args and kwargs and splat

* is called the "splat" operator. Use it for making functions that take multiple things:

def a(b, c=None, *args, **kwargs):
    print('b', b)
    print('c', c)
    print('my args', args)
    print('my kwargs', kwargs)
a('pikachu', 1, 2, d=3, e=4)

Or use it for the opposite:

def b(a, b, c=None, d=None):
    print 'here'
b(*[1,2], **{'c': 1, 'd': 2})

itertools.chain

I love itertools. Very handy library. Combine iterables:

import itertools
list1 = [1,2]
list2 = [3,4]
list(itertools.chain(list1, list2)) == [1,2,3,4]

Use group by:

a = [[1,2], [1,3], [1,4], [2,5]]
[(k, len(rows)) for k, rows in itertools.groupby(a, key=lambda l:l[0])]

pokemon_lineups = [
    ['nidoran', 'nidrino', 'nidoking'],
    ['nidoran', 'nidrina', 'nidoqueen'],
    ['magikarp', 'gyarados']
]
for base_pokemon, evolution_chain in itertools.groupby(pokemon_lineups, key=lambda l:l[0]):
    print 'base_pokemon {} has {} chains'.format(base_pokemon, len(list(evolution_chain)))

Combinations and permutations (great for tests):

import itertools
import random
my_lineup = ['pikachu', 'bulbasaur', 'squirtle', 'charizard', 'jigglypuff', 'gyarados']
def chance_against_gary(ordered_lineup):
    return random.randint(0, 100)
max(
    [
        (l, chance_against_gary(l))
        for l in itertools.permutations(my_lineup, 6)
    ],
    key=lambda t:t[1]
)

zip

Handy for combining lists for iterating:

pokemon = ['charizard', 'bulbasaur', 'pikachu', 'lickitung']
awesomeness = [10, 4, 8, 1]
for p, a in zip(pokemon, awesomeness):
    print '{} is this awesome: {}'.format(p, a)

charizard is this awesome: 10
bulbasaur is this awesome: 4
pikachu is this awesome: 8
lickitung is this awesome: 1

Decorators are great

Wrap your functions! Here's a decorator that makes sure a view is only useable if the user is logged in:

def must_be_logged_in(view):
    def wrapper(request):
        if not request.user:
            raise HTTPForbidden
        return view(request)
    return wrapper

@must_be_logged_in
def my_view(request):
    pass

Classmethods and staticmethods

class A(object):
    def my_method(self):
        print self

    @classmethod
    def class_method(cls):
        print cls

    @staticmethod
    def static_method():
        print 'static'
a = A()
a.my_method()
A.class_method()
a.class_method()
A.static_method()

Best Python -m tools:

A lot of handy tools out there. -m runs the main function of a module. Here's a good list. Some highlights:

SimpleHTTPServer

Handy for making a quick HTTPServer: python -m SimpleHTTPServer 8000

json.tool

Handy for pretty printing: echo '{"greeting": "hello", "name": "world"}' | python -m json.tool

Gzip

python -m gzip [file] # compress python -m gzip -d [file] # decompress

antigravity

python -m antigravity

this

python -m this

Discuss on Twitter