List comprehension without [ ] in Python

Question

Joining a list:

>>> ''.join([ str(_) for _ in xrange(10) ])
'0123456789'

join must take an iterable.

Apparently, join's argument is [ str(_) for _ in xrange(10) ], and it's a list comprehension.

Look at this:

>>>''.join( str(_) for _ in xrange(10) )
'0123456789'

Now, join's argument is just str(_) for _ in xrange(10), no [], but the result is the same.

Why? Does str(_) for _ in xrange(10) also produce a list or an iterable?

Answer 1

The other respondents were correct in answering that you had discovered a generator expression (which has a notation similar to list comprehensions but without the surrounding square brackets).

In general, genexps (as they are affectionately known) are more memory efficient and faster than list comprehensions.

HOWEVER, it the case of ''.join(), a list comprehension is both faster and more memory efficient. The reason is that join needs to make two passes over the data, so it actually needs a real list. If you give it one, it can start its work immediately. If you give it a genexp instead, it cannot start work until it builds-up a new list in memory by running the genexp to exhaustion:

~ $ python -m timeit '"".join(str(n) for n in xrange(1000))'
1000 loops, best of 3: 335 usec per loop
~ $ python -m timeit '"".join([str(n) for n in xrange(1000)])'
1000 loops, best of 3: 288 usec per loop

The same result holds when comparing itertools.imap versus map:

~ $ python -m timeit -s'from itertools import imap' '"".join(imap(str, xrange(1000)))'
1000 loops, best of 3: 220 usec per loop
~ $ python -m timeit '"".join(map(str, xrange(1000)))'
1000 loops, best of 3: 212 usec per loop