Finding matching keys in two large dictionaries and doing it fast

Question

I am trying to find corresponding keys in two different dictionaries. Each has about 600k entries.

Say for example:

    myRDP = { \'Actinobacter\':          


        

Answer 1

in python 3 you can just do

myNames.keys() & myRDP.keys()

Answer 2

def combine_two_json(json_request, json_request2):
intersect = {}

for item in json_request.keys():
    if item in json_request2.keys():
        intersect[item]=json_request2.get(item)
return intersect

Answer 3

Use sets, because they have a built-in intersection method which ought to be quick:

myRDP = { 'Actinobacter': 'GATCGA...TCA', 'subtilus sp.': 'ATCGATT...ACT' }
myNames = { 'Actinobacter': '8924342' }

rdpSet = set(myRDP)
namesSet = set(myNames)

for name in rdpSet.intersection(namesSet):
    print name, myNames[name]

# Prints: Actinobacter 8924342

Answer 4

You can simply write this code and it will save the common key in a list. common = [i for i in myRDP.keys() if i in myNames.keys()]

Answer 5

for key in myRDP:
    name = myNames.get(key, None)
    if name:
        print key, name

dict.get returns the default value you give it (in this case, None) if the key doesn't exist.

Answer 6

You could start by finding the common keys and then iterating over them. Set operations should be fast because they are implemented in C, at least in modern versions of Python.

common_keys = set(myRDP).intersection(myNames)
for key in common_keys:
    print key, myNames[key]