uniq

How to do uniq -d without presorting (or something similar)

由 …衆ロ難τιáo~ 提交于 2021-01-29 03:54:13
问题 I am aware that I can remove duplicated lines without presorting, for example: awk '!x[$0]++' file However, my goal is to only print lines which are duplicated and only once. If it were not for the presorting problem sort | uniq -d would be perfect. BUT the order is of great importance to me. Is there a way to do this with awk, grep or something similar? I am looking for a one liner which does not require writing a script if possible. 回答1: Just check the value of x[$0] : awk 'x[$0]++ == 1'

“isn't numeric” error in “sort” after “uniq”

由 拈花ヽ惹草 提交于 2020-05-14 17:22:40
问题 use List::MoreUtils 'uniq'; print join ", ", sort uniq ("b", "a", "a"); results in Argument "a" isn't numeric in sort at ... print join ", ", uniq sort ("b", "a", "a"); works as expected. print join ", ", sort {$a cmp $b} uniq ("b", "a", "a"); works too - but what is the problem with the first example? 回答1: That code falls under the sort invocation of sort SUBNAME LIST ... If SUBNAME is specified, it gives the name of a subroutine that returns an integer less than, equal to, or greater than 0

“isn't numeric” error in “sort” after “uniq”

由 孤人 提交于 2020-05-14 17:21:55
问题 use List::MoreUtils 'uniq'; print join ", ", sort uniq ("b", "a", "a"); results in Argument "a" isn't numeric in sort at ... print join ", ", uniq sort ("b", "a", "a"); works as expected. print join ", ", sort {$a cmp $b} uniq ("b", "a", "a"); works too - but what is the problem with the first example? 回答1: That code falls under the sort invocation of sort SUBNAME LIST ... If SUBNAME is specified, it gives the name of a subroutine that returns an integer less than, equal to, or greater than 0

“isn't numeric” error in “sort” after “uniq”

由 做~自己de王妃 提交于 2020-05-14 17:20:22
问题 use List::MoreUtils 'uniq'; print join ", ", sort uniq ("b", "a", "a"); results in Argument "a" isn't numeric in sort at ... print join ", ", uniq sort ("b", "a", "a"); works as expected. print join ", ", sort {$a cmp $b} uniq ("b", "a", "a"); works too - but what is the problem with the first example? 回答1: That code falls under the sort invocation of sort SUBNAME LIST ... If SUBNAME is specified, it gives the name of a subroutine that returns an integer less than, equal to, or greater than 0

Why uniq -c output with space instead of \t?

由 主宰稳场 提交于 2020-04-08 10:09:11
问题 I use uniq -c some text file. Its output like this: 123(space)first word(tab)other things 2(space)second word(tab)other things .... So I need extract total number(like 123 and 2 above), but I can't figure out how to, because if I split this line by space, it will like this ['123', 'first', 'word(tab)other', 'things'] . I want to know why doesn't it output with tab? And how to extract total number in shell? ( I finally extract it with python, WTF) Update : Sorry, I didn't describe my question

你知道你自己最经常使用的是哪几个Linux命令呢?

由 谁说胖子不能爱 提交于 2020-04-05 15:17:11
不知道大家自接触 Linux 以来，都使用过哪些命令，其中最常用的命令是什么? 我最常用的命令之一是 sudo ，因为我每天都在使用它在 Linux 上安装、更新、删除软件包以及其它各种需要超级用户权限的操作。 那么你知道你自己最经常使用的是哪几个命令呢?今天我们就来查看一下自己在使用 Linux 过程中最经常用的命令吧。 在 Linux 上查看你自己最经常使用的命令 在 Linux 中，历史文件 ~/.bash_history 会记录你在终端中输入过的所有命令，所以，我们可以通过这个文件来寻找你最经常使用的命令。 具体做法如下： $ history | awk '{print $2}' | sort | uniq -c | sort -nr | head -5 上面这条命令的结果将显示用户在 Linux 上最经常使用的 5 个命令。 输出结果如下： 153 sudo118 ls33 cd30 ssh29 git 这个结果非常清爽直观! 那么，上面这条命令各个部分你都知道它的具体作用吗?现在，我们来详细讲解。 首先我们来看下 history 命令的输出结果： alvin@alvin-pc:~$ history 743 sudo apt-get update 744 sudo apt-get upgrade 745 ls 747 ls 748 git status 这是在

elasticsearch去重计数

由 帅比萌擦擦* 提交于 2020-03-28 12:43:31
如果需要针对ES索引统计某个字段上出现的不同值的个数时，可以使用cardinality聚合查询完成: Post /urlAttributes/_search?search_type=count { "aggs": { "uniq_attr": { "cardinality": { "field": "domain" } } } } 注释：如上，是查询索引urlAttributes中domain字段上的不同值个数， "uniq_attr"是用户自己取的bucket名字。 得到的响应如下所示： { "took": 28, "timed_out": false, "_shards": { "total": 6, "successful": 6, "failed": 0 }, "hits": { "total": 689562, "max_score": 0, "hits": [] }, "aggregations": { "uniq_attr": { "value": 121618 } } } 返回结果表示该字段出现过121618个不同的domain名字。 来源： https://www.cnblogs.com/suntp/p/9562130.html

linux下查看机器是cpu是几核

由 ﹥＞﹥吖頭↗ 提交于 2020-03-23 09:56:59
几个cpu more /proc/cpuinfo |grep "physical id"|uniq|wc -l 每个cpu是几核（假设cpu配置相同） more /proc/cpuinfo |grep "physical id"|grep "0"|wc -l cat /proc/cpuinfo | grep processor 1. 查看物理CPU的个数 #cat /proc/cpuinfo |grep "physical id"|sort |uniq|wc -l 2. 查看逻辑CPU的个数 #cat /proc/cpuinfo |grep "processor"|wc -l 3. 查看CPU是几核 #cat /proc/cpuinfo |grep "cores"|uniq 4. 查看CPU的主频 #cat /proc/cpuinfo |grep MHz|uniq # uname -a Linux euis1 2.6.9-55.ELsmp #1 SMP Fri Apr 20 17:03:35 EDT 2007 i686 i686 i386 GNU/Linux (查看当前操作系统内核信息) # cat /etc/issue | grep Linux Red Hat Enterprise Linux AS release 4 (Nahant Update 5)

shell编程题（九）

由 做~自己de王妃 提交于 2020-03-03 07:28:51
文本分析，取出/etc/password中shell出现的次数 答案1： cat /etc/passwd | awk -F: '{if ($7!="") print $7}' | sort | uniq -c /etc/passwd 中的数据是以:隔开的 sort 是将文件中的每一行作为一个单位，互相比较。 比较的原则是从首字母向后，依次按照ASCII码值进行比较，最后将它们按照升序输出。 参数： -u 去除重复行 -r 降序输出 -o 输入到原文件 (重定向只能输入到新文件) -n 按数值排序 -t 后面可以设定间隔符 -k 指定列数。 -f 忽略大小写排序 -c 检查文件是否已排好序，如果乱序输出第一个乱序的行，最后返回1 -C 同上，不输出内存，仅返回1 -M 按照月份排序 -b 会忽略掉每一行前面的所有空白字符 uniq 检查及删除文本文件中重复出现的行列，一般与sort命令结合使用 uniq可检查文本文件中重复出现的行列 uniq [-c/d/D/u/i] [-f Fields] [-s N] [-w N] [InFile] [OutFile] 参数： -c: 在每列旁边显示该行重复出现的次数。 -d: 仅显示重复出现的行列，显示一行。 -D: 显示所有重复出现的行列，有几行显示几行。 -u: 仅显示出一次的行列 -i: 忽略大小写字符的不同 -f Fields:

shell特殊符号cut、sort_wc_uniq、tee_tr_split命令

由 馋奶兔 提交于 2020-03-01 08:32:38
10月15日任务 8.10 shell特殊符号cut命令 8.11 sort_wc_uniq命令 8.12 tee_tr_split命令 8.13 shell特殊符号下 截取命令：cut -d 分隔符 -f 字段 #截取/etc/passwd的前十行的第1,2个字段 [root@centos7 ~]# cat /etc/passwd | head -n 5 | cut -d ":" -f 1,2 root:x bin:x daemon:x adm:x lp:x -c 指定第几个字符 [root@centos7 ~]# cat /etc/passwd | head -n 5 | cut -c 1 r b d a l 排序命令：sort 按 ASCII码 顺序排序内容 -n 按数字顺序排序(默认字母排在数字前) [root@centos7 ~]# cat /etc/passwd | head -n 5 | sort -n adm:x:3:4:adm:/var/adm:/sbin/nologin bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin root:x:0:0:root:/root:/bin/bash -r
订阅 uniq