awk + How do I find duplicates in a column?

Question

How do I find duplicates in a column?

$ head countries_lat_long_int_code3.csv | cat -n
     1  country,latitude,longitude,name,code
     2  AD,42.546245,1.601554         


        

Answer 1

This will give you the duplicated codes

awk -F, 'a[$5]++{print $5}'

if you're only interested in count of duplicate codes

awk -F, 'a[$5]++{count++} END{print count}'

To print duplicated rows try this

awk -F, '$5 in a{print a[$5]; print} {a[$5]=$0}'

This will print the whole row with duplicates found in col $5:

awk -F, 'a[$5]++{print $0}'