Hardening and patching

Reducing a Linux host's attack surface: fewer services, systemd sandboxing, SSH configuration, kernel sysctls, mandatory access control and benchmarks, then a patching process that actually closes the window, including reboots and live patching.

FREEBy ·Lesson·10 min read·Beginner → Intermediate
Linux hardeningsystemd sandboxingSSH hardeningsysctlSELinux AppArmorpatch management

01Start by removing things

✓What you will learn
  • Shrink a host's attack surface by removing services and packages
  • Sandbox services with systemd directives and measure the result
  • Harden SSH and apply the kernel sysctls that matter
  • Explain SELinux and AppArmor and the role of CIS benchmarks
  • Run a patching process that includes reboots, restarts and live patching

Hardening is mostly subtraction. Every listening port, installed package and running service is code an attacker can reach or reuse. A minimal image with only what the workload needs beats any amount of tuning on a general-purpose install.

Inventory before you tunebash
# What is listening, and which process owns it?
ss -tulpn

# What starts at boot?
systemctl list-unit-files --state=enabled

# Remove what the role does not need
sudo apt purge telnetd rsh-server avahi-daemon cups   # Debian/Ubuntu example
sudo systemctl disable --now rpcbind.socket rpcbind.service
  • One role per host or image. A database server does not need a compiler, a mail server or a desktop.
  • Prefer immutable images rebuilt from a pipeline over long-lived pets edited by hand: drift is where forgotten services come from.
  • Compilers and network tools on production hosts make an attacker's next step easier. Leave them out of runtime images.

02Sandboxing services with systemd

systemd can apply most of the isolation from the previous lesson to an ordinary service with a few lines of unit configuration. systemd-analyze security <unit> scores each service's exposure from 0 (locked down) to 10 (unprotected) and lists which directives are missing, which makes it a good review tool.

A sandboxed service unitini
[Service]
User=report
DynamicUser=no
NoNewPrivileges=yes
ProtectSystem=strict            # whole filesystem read-only except below
ReadWritePaths=/var/lib/report
ProtectHome=yes                 # /home, /root and /run/user invisible
PrivateTmp=yes                  # private /tmp and /var/tmp
PrivateDevices=yes
ProtectKernelTunables=yes       # /proc/sys and /sys read-only
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
RestrictNamespaces=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes      # no writable and executable mappings
SystemCallFilter=@system-service
SystemCallArchitectures=native
CapabilityBoundingSet=
iTurn directives on one at a time

ProtectSystem=strict and MemoryDenyWriteExecute=yes break some software (anything with a JIT, for example). Add directives incrementally, run the service's tests after each, and keep the ones that hold. A score of 2 to 4 is realistic for most services.

03SSH and kernel settings

SSH is the front door of most Linux fleets. Turn off password authentication so credential stuffing and brute force have nothing to attack, keep root logins off, and limit who may log in at all. Larger fleets move to SSH certificates signed by an internal CA with short lifetimes, which avoids distributing authorized_keys files.

sshd hardeningbash
# /etc/ssh/sshd_config.d/10-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AllowGroups ssh-users
MaxAuthTries 3
X11Forwarding no
# validate before reloading, or you can lock yourself out
sudo sshd -t && sudo systemctl reload ssh
sysctlValueEffect
kernel.randomize_va_space2Full ASLR for stack, heap, mmap and libraries (the usual default)
kernel.kptr_restrict2Hide kernel pointer addresses from /proc and logs, which helps defeat KASLR
kernel.dmesg_restrict1Only privileged users read the kernel log
kernel.yama.ptrace_scope1 or 21: only ancestors may ptrace; 2: only with CAP_SYS_PTRACE
kernel.unprivileged_bpf_disabled1Unprivileged users cannot load BPF programs
fs.protected_symlinks, fs.protected_hardlinks1Block classic /tmp symlink and hardlink attacks
fs.suid_dumpable0No core dumps from setuid processes, which could leak secrets
net.ipv4.conf.all.accept_redirects0Ignore ICMP redirects that could reroute traffic

Mount options add a little more: nodev,nosuid,noexec on /tmp, /var/tmp and /dev/shm stop device files, setuid binaries and direct execution from world-writable locations. noexec is a speed bump rather than a wall, since an interpreter can still read a script from there, but it breaks a lot of commodity malware.

04Mandatory access control and benchmarks

Ordinary Unix permissions are discretionary: the owner of a file decides who may read it, and root can do anything. Mandatory access control (MAC) adds a policy that even root processes must obey. A web server confined by SELinux cannot read /etc/shadow or open outbound connections to arbitrary ports even if it is compromised and running as root.

SELinuxAppArmor
Default onRHEL, Fedora, CentOS Stream, AndroidUbuntu, Debian, SUSE
ModelLabels (types) on every file and process; rules between typesPath-based profiles per program
Check statusgetenforce, sestatusaa-status
Learning modePermissive (logs, does not block)Complain mode
StrengthFine-grained, whole-systemEasier to write and read
!Do not fix denials with setenforce 0

Turning SELinux off because one application failed removes protection from everything on the host. Read the denial (ausearch -m AVC -ts recent), then fix the label (restorecon), set a boolean (setsebool -P httpd_can_network_connect on) or write a small policy module.

CIS Benchmarks and DISA STIGs are consensus checklists covering hundreds of settings like the ones above. Scan against them with OpenSCAP or Lynis, apply them through configuration management, and record justified exceptions. The value is less in any single control than in making the baseline explicit and measurable.

05Patching that actually closes the window

Installing a package does not always mean the fix is running. A patched OpenSSL library on disk does nothing for a process that loaded the old one last month; the kernel fix does nothing until reboot. Patch management has three steps, and teams often stop after the first.

  1. Install: automated security updates (unattended-upgrades on Debian and Ubuntu, dnf-automatic on RHEL-family) or, better, rebuilt images rolled out by the pipeline.
  2. Restart what uses old code: needrestart lists services still mapping deleted libraries. lsof +L1 or a scan of /proc/*/maps for (deleted) does the same by hand.
  3. Reboot or live patch the kernel: kernel fixes need a reboot unless you use live patching (kpatch on RHEL, Canonical Livepatch on Ubuntu), which covers many but not all kernel fixes and still needs periodic reboots.
▭On the whiteboard · Prioritising patches4 points
01
urgent
on CISA KEV or high EPSS, internet-facing or reachable by untrusted users: days
02
high
local privilege escalation on multi-user or container hosts (sudo, polkit, kernel): within a week or two
03
normal
everything else on the regular cadence: monthly image rebuild
04
verify
scanner checks running versions, not just installed packages; uptime and kernel version tracked per host
◎Interview radar

Interviewers like "we patched OpenSSL, are we safe?". The answer they want: only after every process using it has restarted, and for kernel bugs only after a reboot or live patch. Mentioning regreSSHion (CVE-2024-6387, a signal-handler race in OpenSSH's sshd that reintroduced a 2006 bug) as a reason to patch internet-facing daemons fast, and EPSS or KEV for prioritisation, shows practical experience.

“How to say it in the room

"I harden by removing first: a minimal image, one role per host, then systemd sandboxing scored with systemd-analyze security. SSH is key-only or certificate-based with root login off, the kernel sysctls restrict pointers, ptrace and BPF, and SELinux or AppArmor stays enforcing. For patching I rebuild images from the pipeline, prioritise by KEV and EPSS, restart services still mapping deleted libraries, and track kernel versions so reboots or live patches actually happen."

Test yourself

After `apt upgrade` installed a fixed libssl, a scanner still flags the host as vulnerable. Uptime is 90 days and `needrestart` lists nginx. What is the most likely explanation?

Q1Is `noexec` on /tmp a real control?+

A partial one. It stops direct execution of dropped binaries but not bash /tmp/x.sh or python3 /tmp/x.py. It raises the cost for commodity malware and costs little, so it is worth having, not worth relying on.

Q2Why prefer rebuilding images over patching in place?+

Rebuilt images are reproducible, tested before rollout and free of drift. Patching in place leaves every host slightly different, and you cannot be sure which fixes are actually running.

Q3What does live patching not cover?+

Changes to data structures, some architectural fixes and anything outside the kernel. Vendors publish which CVEs each live patch covers, and you still need regular reboots.

Finished this one?
Tick it and it shows as done on the roadmap and the course.