how do you find and save duplicated rows in a numpy array?

问题

I have an array e.g.

Array = [[1,1,1],[2,2,2],[3,3,3],[4,4,4],[5,5,5],[1,1,1],[2,2,2]]

And i would like something that would output the following:

Repeated = [[1,1,1],[2,2,2]]

Preserving the number of repeated rows would work too, e.g.

Repeated = [[1,1,1],[1,1,1],[2,2,2],[2,2,2]]

I thought the solution might include numpy.unique, but i can't get it to work, is there a native python / numpy function?

回答1:

Using the new axis functionality of np.unique alongwith return_counts=True that gives us the unique rows and the corresponding counts for each of those rows, we can mask out the rows with counts > 1 and thus have our desired output, like so -

In [688]: a = np.array([[1,1,1],[2,2,2],[3,3,3],[4,4,4],[5,5,5],[1,1,1],[2,2,2]])

In [689]: unq, count = np.unique(a, axis=0, return_counts=True)

In [690]: unq[count>1]
Out[690]: 
array([[1, 1, 1],
       [2, 2, 2]])

回答2:

If you need to get indices of the repeated rows

import numpy as np

a = np.array([[1,1,1],[2,2,2],[3,3,3],[4,4,4],[5,5,5],[1,1,1],[2,2,2]])
unq, count = np.unique(a, axis=0, return_counts=True)
repeated_groups = unq[count > 1]

for repeated_group in repeated_groups:
    repeated_idx = np.argwhere(np.all(a == repeated_group, axis=1))
    print(repeated_idx.ravel())

# [0 5]
# [1 6]

回答3:

You could use something like Repeated = list(set(map(tuple, Array))) if you didn't necessarily need order preserved. The advantage of this is you don't need additional dependencies like numpy. Depending on what you're doing next, you could probably get away with Repeated = set(map(tuple, Array)) and avoid a type conversion if you would like.

来源：https://stackoverflow.com/questions/48099479/how-do-you-find-and-save-duplicated-rows-in-a-numpy-array