uniq

Linux中查看CPU信息

由 让人想犯罪 __ 提交于 2019-12-22 09:20:37
【推荐】2019 Java 开发者跳槽指南.pdf(吐血整理) >>> cat /proc/cpuinfo中的信息 processor 逻辑处理器的id。 physical id 物理封装的处理器的id。 core id 每个核心的id。 cpu cores 位于相同物理封装的处理器中的内核数量。 siblings 位于相同物理封装的处理器中的逻辑处理器的数量。 1 查看物理CPU的个数 #cat /proc/cpuinfo |grep "physical id"|sort |uniq|wc –l 2、 查看逻辑CPU的个数 #cat /proc/cpuinfo |grep "processor"|wc –l 3、 查看CPU是几核 #cat /proc/cpuinfo |grep "cores"|uniq 4、 查看CPU的主频 #cat /proc/cpuinfo |grep MHz|uniq 5、 # uname -a 6、 Linux euis1 2.6.9-55.ELsmp #1 SMP Fri Apr 20 17:03:35 EDT 2007 i686 i686 i386 GNU/Linux (查看当前操作系统内核信息) 7、 # cat /etc/issue | grep Linux 8、 Red Hat Enterprise Linux AS release 4

Apache的Access.log分析总结

由 筅森魡賤 提交于 2019-12-21 03:54:42
#查看80端口的tcp连接 #netstat -tan | grep "ESTABLISHED" | grep ":80" | wc -l 1 #当前WEB服务器中联接次数最多的ip地址: #netstat -ntu |awk '{print $5}' |sort | uniq -c| sort -n -r 231 ::ffff:127.0.0.1:8095 23 ::ffff:192.168.50.201:5432 2 ::ffff:192.168.50.203:80 1 servers) 1 ::ffff:192.168.50.56:43314 1 ::ffff:192.168.50.21:2996 1 ::ffff:192.168.50.21:2989 1 ::ffff:192.168.50.200:8060 1 ::ffff:192.168.50.12:1300 1 ::ffff:192.168.50.12:1299 1 ::ffff:192.168.50.12:1298 1 ::ffff:127.0.0.1:57933 1 Address 1 192.168.50.41:65310 1 192.168.50.41:64949 1 192.168.50.41:49653 #查看日志中访问次数最多的前10个IP #cat access_log |cut -d ' ' -f 1

Sort & uniq in Linux shell

由 六眼飞鱼酱① 提交于 2019-12-20 08:27:37
问题 What is the difference between the following to commands? sort -u FILE sort FILE | uniq 回答1: Using sort -u does less I/O than sort | uniq , but the end result is the same. In particular, if the file is big enough that sort has to create intermediate files, there's a decent chance that sort -u will use slightly fewer or slightly smaller intermediate files as it could eliminate duplicates as it is sorting each set. If the data is highly duplicative, this could be beneficial; if there are few

Linux删除重复行

由 吃可爱长大的小学妹 提交于 2019-12-18 15:50:55
文本处理时，经常要删除重复行，下面是三种方法 第一，用sort+uniq，注意，单纯uniq是不行的。 sort -n test.txt | uniq 第二，用sort+awk命令，注意，单纯awk同样不行，原因同上。 sort -n $file | awk '{if($0!=line)print; line=$0}' 　 第三，用sort+sed命令，同样需要sort命令先排序。 sort -n $file | sed '$!N; /^\(.*\)\n\1$/!P; D' Shell脚本 # !/bin/sh file='test.txt' sort -n $file | uniq sort -n $file | awk '{if($0!=line)print; line=$0}' sort -n $file | sed '$!N; /^\(.*\)\n\1$/!P; D' 测试文件： yanggang@barry$ cat test.txt aaa bbbbb ccccc 123 aaaaa 123 bbb aaa 执行结果： yanggang@barry$ ./diffRow.sh aaa aaaaa bbb bbbbb ccccc 123 推荐参考： 删除文本中的重复行sort+uniq/awk/sed SED单行脚本快速参考Unix 流编辑器 来源： https:/

Find unique lines

由 本秂侑毒 提交于 2019-12-18 10:25:27
问题 How can I find the unique lines and remove all duplicates from a file? My input file is 1 1 2 3 5 5 7 7 I would like the result to be: 2 3 sort file | uniq will not do the job. Will show all values 1 time 回答1: uniq has the option you need: -u, --unique only print unique lines $ cat file.txt 1 1 2 3 5 5 7 7 $ uniq -u file.txt 2 3 回答2: Use as follows: sort < filea | uniq > fileb 回答3: uniq -u has been driving me crazy because it did not work. So instead of that, if you have python (most Linux

linux-网络监控命令-netstat进阶

由 ￣綄美尐妖づ 提交于 2019-12-17 08:38:04
2.网络连接状态详解 共有12中可能的状态，前面11种是按照TCP连接建立的三次握手和TCP连接断开的四次挥手过程来描述的。 1)、LISTEN:首先服务端需要打开一个socket进行监听，状态为LISTEN./* The socket is listening for incoming connections. 侦听来自远方TCP端口的连接请求 */ 2)、 SYN_SENT:客户端通过应用程序调用connect进行active open.于是客户端tcp发送一个SYN以请求建立一个连接.之后状态置为SYN_SENT./*The socket is actively attempting to establish a connection. 在发送连接请求后等待匹配的连接请求 */ 3)、 SYN_RECV:服务端应发出ACK确认客户端的 SYN,同时自己向客户端发送一个SYN. 之后状态置为SYN_RECV/* A connection request has been received from the network. 在收到和发送一个连接请求后等待对连接请求的确认 */ 4)、ESTABLISHED: 代表一个打开的连接，双方可以进行或已经在数据交互了。/* The socket has an established connection. 代表一个打开的连接

shell: select unique row of flat file

由 情到浓时终转凉″ 提交于 2019-12-14 01:41:08
问题 I have a flat file looks like this cat file ID1, VALUE1_1 ID1, VALUE1_2 ID1, VALUE1_3 ID2, VALUE2_1 ID2, VALUE2_1 ID3, VALUE3_1 ID3... As you can see from the data sample, for each ID, there are several values for that ID and they could be whatever value - same or not. For me, I don't care which value it is picking up. Any value works for me. So I only want one value from each ID. I don't really care which one, but if I have to choose, I would say the row which has the longest length. ID1,

shell编程之正则表达式（三）awk工具

由 安稳与你 提交于 2019-12-13 16:07:58
awk 工具 在 Linux/UNIX 系统中，awk 是一个功能强大的编辑工具，逐行读取输入文本，并根据指定的匹配模式进行查找，对符合条件的内容进行格式化输出或者过滤处理，可以在无交互的情况下实现相当复杂的文本操作，被广泛应用于 Shell 脚本，完成各种自动化配置任务。 awk 常见用法 通常情况下 awk 所使用的命令格式如下所示，其中，单引号加上大括号“{}”用于设置对数据进行的处理动作。awk 可以直接处理目标文件，也可以通过“-f”读取脚本对目标文件进行处理。 awk 选项 '模式或条件 {编辑指令}' 文件 1 文件 2 „ //过滤并输出文件符条件的内容awk -f 脚本文件 文件 1 文件 2 „ //从脚本中调用编辑指令,过滤并输出内容。前面提到 sed 命令常用于一整行的处理,而 awk 比较倾向于将一行分成多个“字段”然后再进行处理，且默认情况下字段的分隔符为空格或者 tab 键。awk 执行结果可以通过 print 的功能将字段数据打印显示。在使用 awk 命令的过程中,可以使用逻辑操作符“&&”，表示“与”， “||”表示“或”，“！”表示“非”；还可以进行简单的数学运算，如+、-、*、/、%、^分别 表示加、减、乘、除、取余和乘方。 在 Linux 系统中/etc/passwd 是一个非常典型的格式化文件，各字段间使用“：”作为分隔符隔开，Linux

remove dups from many csv files

由 二次信任 提交于 2019-12-13 15:48:24
问题 Given n csv files where they add up to 100 GB in size, I need to remove duplicate rows based on the following rules and conditions: The csv files are numbered 1.csv to n.csv, and each file is about 50MB in size. The first column is a string key, 2 rows are considered dup if their first columns are the same. I want to remove dups by keeping the one in a later file (2.csv is considered later than 1.csv) My algorithm is the following, I want to know if there's a better one. merge all files into

How to find max value grouped by multiple keys in array of hashes?

由 霸气de小男生 提交于 2019-12-13 13:24:53
问题 Have data that has this kind of structure. Will be in ascending order by 'c'. [ { 'a' => 1, 'b' => 1, 'c' => 1, 'd' => '?' }, { 'a' => 1, 'b' => 1, 'c' => 2, 'd' => '?' }, { 'a' => 1, 'b' => 1, 'c' => 3, 'd' => '?' }, { 'a' => 1, 'b' => 2, 'c' => 4, 'd' => '?' }, { 'a' => 1, 'b' => 2, 'c' => 5, 'd' => '?' }, { 'a' => 2, 'b' => 1, 'c' => 6, 'd' => '?' }, { 'a' => 2, 'b' => 1, 'c' => 7, 'd' => '?' }, { 'a' => 2, 'b' => 1, 'c' => 8, 'd' => '?' }, { 'a' => 2, 'b' => 2, 'c' => 9, 'd' => '?' }, {
订阅 uniq