Hi, Long time no see.
In the process of developing programs using Linux, I have encountered some shell problems, and below I will share some of the more common and more useful tips and tools. And keep updating them as I learn more.

Why some shell commands follows with cat /dev/null

Using :

Discard the normal output message.

more

Such as

ls -l >/dev/null
cat --INCORRECT_OPTION > /dev/null 2>/dev/null
ls -l

&>/dev/null
This is just a short use for >/dev/null 2>&1. It redirects file descriptor 2 (STDERR) and descriptor 1 (STDOUT) to /dev/null.
Using the command stat,we can see
~ # stat /dev/null
  File: /dev/null
  Size: 0               Blocks: 0          IO Block: 4096   character special file
Device: 6h/6d   Inode: 83          Links: 1     Device type: 1,3
Access: (0660/crw-rw----)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 1970-01-01 08:00:00.000000000
Modify: 1970-01-01 08:00:00.000000000
Change: 2023-04-27 02:04:41.000000000

It’s size is 0 and access is 0660,means all program could read and write(butu read will return EOF)
file table

But the dev/null is not an executable file,which means it cannot be redirect by | operator to redirect.The only way is use file directions > , >>, <, <<

we can discard the stderr message in two ways:

cat --INCORRECT_OPTION > /dev/null 2>/dev/null
cat --INCORRECT_OPTION > dev/null 2>&1

The second 2>&1 : &1 tell the shell 1 is a file descriptor and not a file name.

tcpdump or wireshark

Wireshark
Like the wireshark in shell, a useful tool to trace the web traffic data.
tcpdump -c 5 -X icmp means count 5 packets and diaplay the packet by dump(hex and asicc) and only filter the icmp(mostly used by ping).
Here we using 192.168.9.121 to ping the 192.168.9.141,the first 5 packets is below.

~ # tcpdump -c 5 -X icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
11:46:32.913679 IP 192.168.9.121 > 192.168.9.141: ICMP echo request, id 1, seq 212, length 40
        0x0000:  4500 003c 074a 0000 4001 df20 c0a8 0979  E..<.J..@......y
        0x0010:  c0a8 098d 0800 4c87 0001 00d4 6162 6364  ......L.....abcd
        0x0020:  6566 6768 696a 6b6c 6d6e 6f70 7172 7374  efghijklmnopqrst
        0x0030:  7576 7761 6263 6465 6667 6869            uvwabcdefghi
11:46:32.914101 IP 192.168.9.141 > 192.168.9.121: ICMP echo reply, id 1, seq 212, length 40
        0x0000:  4500 003c 220e 0000 4001 c45c c0a8 098d  E..<"...@..\....
        0x0010:  c0a8 0979 0000 5487 0001 00d4 6162 6364  ...y..T.....abcd
        0x0020:  6566 6768 696a 6b6c 6d6e 6f70 7172 7374  efghijklmnopqrst
        0x0030:  7576 7761 6263 6465 6667 6869            uvwabcdefghi
11:46:33.925833 IP 192.168.9.121 > 192.168.9.141: ICMP echo request, id 1, seq 213, length 40
        0x0000:  4500 003c 0752 0000 4001 df18 c0a8 0979  E..<.R..@......y
        0x0010:  c0a8 098d 0800 4c86 0001 00d5 6162 6364  ......L.....abcd
        0x0020:  6566 6768 696a 6b6c 6d6e 6f70 7172 7374  efghijklmnopqrst
        0x0030:  7576 7761 6263 6465 6667 6869            uvwabcdefghi
11:46:33.926234 IP 192.168.9.141 > 192.168.9.121: ICMP echo reply, id 1, seq 213, length 40
        0x0000:  4500 003c 2252 0000 4001 c418 c0a8 098d  E..<"R..@.......
        0x0010:  c0a8 0979 0000 5486 0001 00d5 6162 6364  ...y..T.....abcd
        0x0020:  6566 6768 696a 6b6c 6d6e 6f70 7172 7374  efghijklmnopqrst
        0x0030:  7576 7761 6263 6465 6667 6869            uvwabcdefghi
11:46:34.932416 IP 192.168.9.121 > 192.168.9.141: ICMP echo request, id 1, seq 214, length 40
        0x0000:  4500 003c 0754 0000 4001 df16 c0a8 0979  E..<.T..@......y
        0x0010:  c0a8 098d 0800 4c85 0001 00d6 6162 6364  ......L.....abcd
        0x0020:  6566 6768 696a 6b6c 6d6e 6f70 7172 7374  efghijklmnopqrst
        0x0030:  7576 7761 6263 6465 6667 6869            uvwabcdefghi
5 packets captured
6 packets received by filter
0 packets dropped by kernel

~ # tcpdump -tttt -c 2 -X
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
2023-04-27 11:51:29.797574 IP 192.168.9.141.23 > 192.168.9.121.51829: Flags [P.], seq 2877355743:2877355819, ack 2896325498, win 457, length 76
        0x0000:  4500 0074 e8a5 4000 4006 bd87 c0a8 098d  E..t..@.@.......
        0x0010:  c0a8 0979 0017 ca75 ab80 f6df aca2 6b7a  ...y...u......kz
        0x0020:  5018 01c9 94bd 0000 7463 7064 756d 703a  P.......tcpdump:
        0x0030:  2076 6572 626f 7365 206f 7574 7075 7420  .verbose.output.
        0x0040:  7375 7070 7265 7373 6564 2c20 7573 6520  suppressed,.use.
        0x0050:  2d76 206f 7220 2d76 7620 666f 7220 6675  -v.or.-vv.for.fu
        0x0060:  6c6c 2070 726f 746f 636f 6c20 6465 636f  ll.protocol.deco
        0x0070:  6465 0d0a                                de..
2023-04-27 11:51:29.804136 ARP, Request who-has 192.168.101.190 tell 192.168.101.184, length 46
        0x0000:  0001 0800 0604 0001 1c1b 0d3d 59e5 c0a8  ...........=Y...
        0x0010:  65b8 0000 0000 0000 c0a8 65be 0000 0000  e.........e.....
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
2 packets captured
3 packets received by filter
0 packets dropped by kernel

Yes, because we using the telnet to connet the linux, it’s using the web,so we will capcure our message in the packets.

Here are some additional ways to tweak how you call tcpdump.

  • -X : Show the packet’s _contents_ in both hex and ASCII.
  • -XX : Same as -X, but also shows the ethernet header.
  • -D : Show the list of available interfaces
  • -l : Line-readable output (for viewing as you save, or sending to other commands)
  • -q : Be less verbose (more quiet) with your output.
  • -t : Give human-readable timestamp output.
  • -tttt : Give maximally human-readable timestamp output.
  • -i eth0 : Listen on the eth0 interface.
  • -vv : Verbose output (more v’s gives more output).
  • -c : Only get _x_ number of packets and then stop.
  • -s : Define the _snaplength_ (size) of the capture in bytes. Use -s0 to get everything, unless you are intentionally capturing less.
  • -S : Print absolute sequence numbers.
  • -e : Get the ethernet header as well.
  • -q : Show less protocol information.
  • -E : Decrypt IPSEC traffic by providing an encryption key.

operator

The operator is the same with the wireshark.

  1. AND
    _and_ or &&
  2. OR
    _or_ or ||
  3. EXCEPT (not)
    not or !

time record

Using time function will be useful to measure the real time of the program.

~$ time curl https://github.com -k > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  221k    0  221k    0     0   187k      0 --:--:--  0:00:01 --:--:--  187k

real    0m1.352s
user    0m0.170s
sys     0m0.161s

curl well get the webpage to stdout, but here we just want to get the running time,so we > /device/null to discard them.
Sure, we still have mang ways to get the time,but this is the linux default.

Profilers

FlameGraph is a intereseting tool,but the function’s name is too long or meaningless,hard to analysize,maybe the later,i will get a new think about the profiler.
profiling 与性能优化总结 - 知乎

shell scripts

Write proper scripts is useful for develop who working with linux.
When I want to write a sell scripts in a embled linux shell.however I faced some problems.
Most of the shell are running by bash.so we can easily find that most shell scripts looks like this:

#!/bin/bash

It said that we will use bash as the scripts executor.
but this is the root of the reason why my shell scripts can’t run normally.
Because i only hace the /bin/sh without the bash .They have some different in the syntax.
In the sh
we can do the .sh like this:
#! /bin/sh

for i in $(seq 1 10)
do
echo $(expr $i);
done

When we want to execute sqlite3 statements directly in the shell. we can try this way :
sqlite3 some.db ".dump";

with the “ “,we can insert some query statements.

find&grep

find . -name
grep -nr "name"

kill

we usually use the kill -9 $(PID) to kill a program.
however, when we use the command ps, it’s difficult to find the pid of the process.
Sure, we can use the grep to filter the message of ps,but it’s still a little long to me. such as ps -ef | grep firefox .
so we can try pkill, it alse support the fuzzy search.it means you don’t need to type the process’s full name or pid number.

pkill -９ firefox

根据进程名杀死进程 －kill进程名 - 温迪军 - 博客园

history

find the terminl’s command’s history.
and using the ‘!!’,it will excute the last command.

Also，Using the command Ctrl + r then input some histroy cmd,it will autofill.

find the mount usage upon this disk

du -sh * | grep M | sort -nr

and df -h could tell the all disk info.

tar and tar.gz

tar

var meaning
c creat
x extract
z gzip
j gzip2

typical use:
tar -var targetFile sourcedir/file
tar -zcvf test.tar.gz ./
tar -cvf test.tar.gz ./
if we use -cvf instead of -zcvf the test.tar.gz will only be tar but not be zip.

As for zip function, we also can extract the tar.gz

tar -zxvf *.tar.gz -C existedDirPath
or
tar -zxvf *.tar.gz * extract at current path.

However, tar -xvf can extract .tar.gz and .tar file.
but tar -zxvf olny can extract .tar.gz file.

tar -jxvf *.tar.gz2
tar -jcvf *.tar.gz2