If you are using numpy in your code (which might be a good choice for larger amounts of data), check out numpy.unique:
>>> import numpy as np
>>> wordsList = [u'nowplaying', u'PBS', u'PBS', u'nowplaying', u'job', u'debate', u'thenandnow']
>>> np.unique(wordsList)
array([u'PBS', u'debate', u'job', u'nowplaying', u'thenandnow'],
dtype='
(http://docs.scipy.org/doc/numpy/reference/generated/numpy.unique.html)
As you can see, numpy supports not only numeric data, string arrays are also possible. Of course, the result is a numpy array, but it doesn't matter a lot, because it still behaves like a sequence:
>>> for word in np.unique(wordsList):
... print word
...
PBS
debate
job
nowplaying
thenandnow
If you really want to have a vanilla python list back, you can always call list().
However, the result is automatically sorted, as you can see from the above code fragments. Check out numpy unique without sort if retaining list order is required.
