Pandas sum of all word counts in column

Question

I have a pandas column that contains strings. I want to get a word count of all of the words in the entire column. What\'s the best way of doing that without looping through eac

Answer 1

df.a.str.extractall('(\w+)').count()[0]

This extracts all words (matches the regex (\w+)) in a each cell in a and puts them in a new frame that looks something like:

             0
  match       
0 0       some
  1      words
1 0       lots
  1       more
  2      words
2 0         hi 

You can then just do a count on the rows to get the number of words.

Note that you can always change the regex if you want. For example, if some words might contain punctuation characters you can define a words as any series of non-whitespace characters and do something like:

df.a.str.extractall('(\S+)').count()[0]

instead

EDIT

If you care about speed at all, use DSM's solution instead:

Basic time test using ipython's %timeit:

%timeit df.a.str.extractall('(\S+)').count()[0] 
1000 loops, best of 3: 1.28 ms per loop

%timeit df["a"].str.split().str.len().sum()
1000 loops, best of 3: 447 µs per loop