How to efficiently count the occurrences of mac values by distinct interface value when these are inside list of dictionaries?

问题

In other words given the list of dictionaries how could I efficiently end up with a list of interfaces which have multiple mac addresses tied to them? In the example below we would output Te1/1/1 since there are three mac addresses tied to this interface, it's possible the result could contain multiple interfaces. Blank values for interface key should not be counted. The input list could be long, so I'm wondering what is the fastest way to approach this problem?

{
   "mac_address_table": [
      {
         "active": true,
         "interface": "",
         "last_move": -1,
         "mac": "01:00:0C:CC:CC:CC",
         "moves": -1,
         "static": true,
         "vlan": 0
      },
      {
         "active": true,
         "interface": "",
         "last_move": -1,
         "mac": "01:00:0C:CC:CC:CD",
         "moves": -1,
         "static": true,
         "vlan": 0
      },
      {
         "active": true,
         "interface": "Gi1/0/3",
         "last_move": -1,
         "mac": "78:72:5D:2C:F7:B4",
         "moves": -1,
         "static": false,
         "vlan": 124
      },
      {
         "active": true,
         "interface": "Te1/1/1",
         "last_move": -1,
         "mac": "B0:90:7E:8E:EE:01",
         "moves": -1,
         "static": false,
         "vlan": 124
      },
      {
         "active": true,
         "interface": "Te1/1/1",
         "last_move": -1,
         "mac": "BC:26:C7:7F:B2:DD",
         "moves": -1,
         "static": false,
         "vlan": 124
      },
      {
         "active": true,
         "interface": "Te1/1/1",
         "last_move": -1,
         "mac": "BC:26:C7:7F:B2:DE",
         "moves": -1,
         "static": false,
         "vlan": 124
      }
   ]
}

回答1:

You could use a defaultdict to keep track of how many times a particular interface name is seen.

Then you would need to filter out the interfaces that are seen only once.

interfaces = defaultdict(int)
for iface in data['mac_address_table']:
    if not iface['interface']:
        continue
    interfaces[iface['interface']] += 1

multi_mac_interfaces = [iface for iface, count in interfaces.items() if count > 1]
print(multi_mac_interfaces)

回答2:

If your input is large and you can't use json parser (RAM limitation), you should process it line by line. Fastest way to check for duplicates will be using set also using set as container for result you don't need to think about duplicates (duplicate entries won't be added):

text = "\"interface\": \""
text_len = len(text)
delimiter = "\""

with open("FILENAME") as in_f:
    unique_keys = set()
    result = set()
    value = ""
    for line in in_f:
        if value and "}" in line: # detecting end of object
            if value in unique_keys:
                result.add(value)
            else:
                unique_keys.add(value)
            value = ""
        else:
            idx = line.find(text) + text_len
            if idx >= text_len:
                value = line[idx: line.find(delimiter, idx)]

for i in result:
    print(i)

来源：https://stackoverflow.com/questions/61176815/how-to-efficiently-count-the-occurrences-of-mac-values-by-distinct-interface-val

标签

python

dictionary