How to remove these duplicates in a list (python)

Question

biglist = 

[ 

    {\'title\':\'U2 Band\',\'link\':\'u2.com\'}, 
    {\'title\':\'ABC Station\',\'link\':\'abc.com\'}, 
    {\'title\':\'Live Concert by U2\',\'link         


        

Answer 1

You can use defaultdict to group items by link, then removed duplicates if you want to.

from collections import defaultdict

nodupes = defaultdict(list)
for d in biglist:
    nodupes[d['url']].append(d['title']

This will give you:

defaultdict(, {'abc.com': ['ABC Station'], 'u2.com': ['U2 Band', 
'Live Concert by U2']})