Linux essentials

Written for GNU coreutils and systemd, which is what a Linux server runs. Several of these behave differently on macOS, which ships BSD versions of the same commands. The notes say where.

Commands

Find old files and delete them in one pass:

find /var/log -name '*.log' -mtime +7 -delete

Search a tree, only in the files you care about:

grep -rn --include='*.py' 'TODO' .

Edit in place:

sed -i 's/old/new/g' file.conf

Unpack somewhere other than here:

tar -xzf archive.tar.gz -C /opt/target

Copy a tree and see what would happen first:

rsync -avh --delete --dry-run src/ dst/

Follow one service's logs:

journalctl -u nginx -f --since '10 min ago'
journalctl -p err -b        # errors, this boot only

Start a service and have it start at boot, in one command:

systemctl enable --now nginx

What is listening, with the process holding the port:

ss -ltnp
lsof -i :8080

Find what filled the disk:

du -sh * | sort -h
df -h

Run something in parallel over a list, safely:

find . -name '*.jpg' -print0 | xargs -0 -r -P4 -I{} convert {} {}.webp

Repoint a symlink that already exists:

ln -sfn /opt/app/releases/v2 /opt/app/current

The processes actually using the memory:

ps aux --sort=-rss | head

Kernel messages with real timestamps:

dmesg -T --level=err,warn

Notes

sed -i is the difference that bites most often. GNU takes no argument, so sed -i 's/a/b/' file works. BSD, and therefore macOS, reads the next word as a backup suffix, so the same command tries to edit a file named s/a/b/. On macOS write sed -i '' 's/a/b/' file.

find -delete is find's own action, not a program run once per file. It turns on -depth automatically, so contents are removed before the directory holding them. It refuses to remove a directory that is not empty, and a failed removal prints an error and makes find exit nonzero rather than passing quietly.

Two consequences of that automatic -depth. It makes -prune ineffective, so the two cannot usefully be combined. And a test run with -print behaves differently from the real run unless you add -depth to the test as well.

Order matters more here than anywhere else in find. -delete placed before the filters deletes everything under the starting point, because find reads the command line as an expression and evaluates it left to right.

xargs -r does nothing when the input is empty, which stops the command running once with no arguments. -print0 with -0 is what survives filenames containing spaces. Both are easy to leave out and only fail on the day a filename has a space in it.

ss replaced netstat. Neither is installed on a slim container image, and neither is ps: ss comes from iproute2 and ps from procps. This is the usual reason a debug command works on the host and not inside the container.

dmesg is the other half of that. It is installed, and it still fails inside a container with read kernel buffer failed: Operation not permitted, because the ring buffer belongs to the host kernel rather than to the container. Read it from the host. Its -T timestamps are computed from uptime, so they drift after a suspend, which the tool's own help admits.

ln -sfn is three flags doing one job. Without -n the command follows the existing symlink and creates the new link inside the directory it points at, which is how a release symlink ends up nested.

ps sort keys are not the column headings. rss and %mem both work, and %rss is rejected with unknown sort specifier even though RSS is the column you are looking at. rss is resident memory in kilobytes, %mem is the same figure as a percentage of total.

Links

• GNU coreutils manual
• GNU findutils manual
• journalctl