What you'll learn

  1. Quick answer
  2. Everything itertools returns is a one-shot iterator
  3. product, permutations and combinations
  4. groupby only groups items that are already adjacent
  5. chain, islice and accumulate
  6. When itertools is the wrong answer
  7. FAQ

Quick Answer

itertools is a standard library module of iterator building blocks. product, permutations and combinations generate arrangements, chain joins iterables, islice takes a slice lazily, accumulate produces running totals, and groupby groups adjacent items. Two traps dominate. Everything returned is a one-shot iterator, so consuming it twice gives an empty result the second time, and groupby only groups consecutive items, so you must sort by the same key first.

Everything itertools returns is a one-shot iterator

Before any of the individual functions make sense, you need the rule that governs all of them. itertools functions do not return lists. They return iterators, which produce values on demand and remember how far they have got. Once an iterator is exhausted it stays exhausted forever.

from itertools import combinations

pairs = combinations(['a', 'b', 'c'], 2)

print(len(list(pairs)))   # 3
print(list(pairs))        # [] - the same iterator, now empty

Nothing raises here. The second line is simply wrong and silently returns nothing. This is the single most common itertools bug, and it usually appears as "my function worked when I tested it, then returned an empty list when I added a length check above it".

The laziness is the whole point of the module though. Building the full list of arrangements is often impossible. Choosing 10 items from 20 gives 184756 combinations, and the numbers grow far faster than that as the inputs grow, so materialising them is exactly what you want to avoid.

There are two safe patterns. Either consume the iterator once, in a single for loop, or convert it to a list immediately and use the list everywhere after that:

pairs = list(combinations(['a', 'b', 'c'], 2))
print(len(pairs))   # 3
print(pairs)        # [('a', 'b'), ('a', 'c'), ('b', 'c')]

Converting to a list is fine when the result is small. It is a memory disaster when it is not, so treat list() as a decision you make deliberately rather than a reflex. And never call list() on an infinite iterator such as itertools.count() or itertools.cycle(). Your program will not error, it will just consume memory until the process is killed.

product, permutations and combinations

These three answer "generate every arrangement", and the difference between them is whether order matters and whether repeats are allowed. Getting the wrong one is a correctness bug, not a style choice.

product is the nested loop. product(A, B) gives every pairing of one item from A with one from B, in the order the nested loops would produce them:

from itertools import product

sizes = ['S', 'M']
colours = ['red', 'blue']

for combo in product(sizes, colours):
    print(combo)
# ('S', 'red')
# ('S', 'blue')
# ('M', 'red')
# ('M', 'blue')

The repeat argument replaces "the same loop written n times", which is how you generate every four digit PIN or every cell of an n dimensional grid:

from itertools import product

print(len(list(product(range(10), repeat=4))))   # 10000

permutations treats order as meaningful and never reuses a position. combinations ignores order, so it only produces each set once, and it always emits items in the order they appear in the input. combinations_with_replacement allows an item to be picked more than once:

from itertools import permutations, combinations, combinations_with_replacement

print(list(permutations('abc', 2)))
# [('a','b'), ('a','c'), ('b','a'), ('b','c'), ('c','a'), ('c','b')]

print(list(combinations('abc', 2)))
# [('a','b'), ('a','c'), ('b','c')]

print(list(combinations_with_replacement('abc', 2)))
# [('a','a'), ('a','b'), ('a','c'), ('b','b'), ('b','c'), ('c','c')]

Two behaviours to keep in mind. First, none of them look at values, only positions, so a duplicate item in the input produces duplicate output tuples that look identical; deduplicate the input first if that matters. Second, product has to keep all of its input arguments in memory in order to loop over them repeatedly, so passing it a huge generator does not save you anything.

groupby only groups items that are already adjacent

If you come from SQL, the name groupby is actively misleading. SQL's GROUP BY collects every matching row wherever it sits in the table. itertools.groupby walks the input once and starts a new group every time the key changes from one item to the next.

from itertools import groupby

rows = [('Pune', 88), ('Delhi', 71), ('Pune', 94)]

for city, group in groupby(rows, key=lambda r: r[0]):
    print(city, [r[1] for r in group])
# Pune [88]
# Delhi [71]
# Pune [94]

Pune appears twice. No warning, no error, and if you were writing those groups into a dict you would have overwritten the first Pune entry with the second. The fix is to sort by the same key you group by, and to use exactly the same key function for both so they cannot drift apart:

from itertools import groupby

rows = [('Pune', 88), ('Delhi', 71), ('Pune', 94)]
by_city = lambda r: r[0]

rows.sort(key=by_city)
for city, group in groupby(rows, key=by_city):
    print(city, [r[1] for r in group])
# Delhi [71]
# Pune [88, 94]

There is a second trap. The group handed to you is itself an iterator that shares the underlying input, and it is invalidated as soon as you move to the next group. So storing the groups for later gives you empty results:

# rows is already sorted and by_city is the same key function as above
groups = list(groupby(rows, key=by_city))
for city, group in groups:
    print(city, list(group))
# Delhi []
# Pune []

You must consume each group inside the loop, usually with list(group) or sum(x for x in group). Given both traps, ask yourself whether you need groupby at all. If the data is not already sorted and the sort is only there to enable grouping, a defaultdict(list) does the job in one pass with no ordering requirement and no invalidation rule to remember. groupby earns its place when the input is genuinely sequential, such as a sorted log file or lines already ordered by date, where you want to stream through without loading everything.

chain, islice and accumulate

chain treats several iterables as one continuous sequence without building a combined list. chain.from_iterable is the same thing when the iterables arrive as one nested structure, and it doubles as a readable way to flatten one level:

from itertools import chain

a = [1, 2]
b = [3, 4]
print(list(chain(a, b)))                          # [1, 2, 3, 4]
print(list(chain.from_iterable([[1, 2], [3, 4]])))  # [1, 2, 3, 4]

islice is slicing for things you cannot slice. A file object, a generator and a database cursor all refuse [:5], and islice gives you the first n items without reading the rest:

from itertools import islice

with open('big.log', encoding='utf-8') as f:
    for line in islice(f, 5):
        print(line.rstrip())

Two differences from normal slicing. islice does not accept negative indices, because it cannot know the length in advance, so islice(it, -3) raises ValueError. And skipping is not free: islice(it, 100, 110) genuinely pulls and discards the first hundred items, and those items are gone from the underlying iterator afterwards.

accumulate produces running results. With no function it adds, which gives you a running total in one line. Pass any two argument function and you get running products, running maxima, or anything else:

from itertools import accumulate
import operator

sales = [1200, 800, 1500]
print(list(accumulate(sales)))                       # [1200, 2000, 3500]

print(list(accumulate([1, 2, 3, 4], operator.mul)))  # [1, 2, 6, 24]
print(list(accumulate([3, 1, 4, 1, 5], max)))        # [3, 3, 4, 4, 5]

Note the shape of the output: accumulate returns as many values as it received, starting with the first input unchanged. If you expected a leading zero, it is not there unless you pass the initial argument, which was added in Python 3.8. This is the prefix sum array that shows up in half of all subarray problems, so it is worth recognising.

When itertools is the wrong answer

The module is elegant enough that people reach for it when a plain loop would be clearer. A few honest limits.

If a list comprehension expresses the same thing, use the comprehension. [x for xs in nested for x in xs] is understood by every Python developer; chain.from_iterable is only clearly better when you need laziness or when the nesting is coming from somewhere else. Similarly, sum(xs) beats list(accumulate(xs))[-1] every time.

If you need the results more than once, stop using an iterator. Either build the list or write a function that returns a fresh iterator on each call. itertools.tee looks like the answer for "iterate twice", but it works by buffering everything one copy has seen and the other has not, so if you fully consume one copy before touching the second, you have stored the entire sequence in memory anyway.

from itertools import tee

it = (n * n for n in range(1000000))
a, b = tee(it)
squares = list(a)     # b is now buffering a million values

If you are combinatorially enumerating a search space, check the size before you loop. Permutations of a list of 12 items already run into the hundreds of millions, and a brute force permutations loop that looks fine on a sample of 6 will hang on the real input. In interviews this is exactly the difference between the brute force answer and the expected one, so say out loud that you know the enumeration is factorial before you write it.

And if the code needs a comment to explain which itertools function is doing what, a five line for loop is the better submission. The standard library is there to make intent obvious. When it stops doing that, it is not helping.

Frequently Asked Questions

Why does my itertools result come out empty the second time I use it? Because itertools returns iterators, not lists, and an iterator can only be walked once. Calling list() or len(list(...)) on it consumes it, so every later use sees an exhausted iterator and produces nothing. Nothing raises an error, which is why this is hard to spot. Either consume it exactly once in a single loop, or convert it to a list up front and use that list everywhere afterwards.
What is the difference between permutations and combinations? permutations cares about order, so ('a','b') and ('b','a') are both produced. combinations does not, so only ('a','b') appears, following the order of the input. Use permutations for arrangements such as seatings or passwords where position changes the meaning, and combinations for selections such as picking a team of three from a class, where the group is the same regardless of who was named first.
Why does groupby give me the same key more than once? groupby starts a new group whenever the key changes between consecutive items, so unsorted input produces one group per run of adjacent equal keys. Sort the data with the same key function before grouping. Also remember that each group is a live iterator tied to the input, so it must be consumed inside the loop; collecting the groups first and reading them later gives you empty groups.
Is itertools faster than writing my own loop? On CPython the functions are implemented in C, so the looping machinery itself is faster than the equivalent Python bytecode, but that is usually a small part of the total. The real gain is memory. islice on a large file reads only what you asked for, and product with repeat generates arrangements one at a time instead of building a list that may not fit in memory. Choose it for laziness first and speed second.
When should I use accumulate instead of a running total variable? Use accumulate when you want the whole sequence of intermediate results, such as a prefix sum array for subarray queries, a cumulative revenue chart, or a running maximum. If you only need the final value, a plain sum() or a single accumulator variable is clearer. Remember accumulate emits the first input unchanged as its first value, so the output has the same length as the input and does not begin with zero.