13.1 6 Enable And Disable Linux Services

You're staring at a fresh Linux install. Maybe it's a VPS you just spun up. Maybe it's a Raspberry Pi sitting on your desk. Either way, you need something to run in the background — a web server, a database, Docker, SSH — and you need it to survive a reboot Less friction, more output..

That's where service management comes in.

And if you've ever typed systemctl enable nginx and wondered what actually happened, or why service apache2 start works on Ubuntu but fails on Arch, this one's for you.

What Is a Linux Service Anyway

A service — sometimes called a daemon — is just a program that runs in the background. No user clicking buttons. Think about it: no terminal attached. It waits for requests, logs things, keeps time, serves web pages, manages containers, whatever.

The word daemon comes from Maxwell's demon, a thought experiment in physics. The name stuck. You'll still see the d suffix everywhere: sshd, httpd, crond, systemd-journald Worth keeping that in mind. Surprisingly effective..

Services start at boot. They restart when they crash (if configured right). Here's the thing — they listen on ports. They write logs. They're the plumbing of any Linux system Small thing, real impact. But it adds up..

But here's the thing: how they start has changed. A lot.

The Two Worlds You'll Meet

If you're on any modern distro — RHEL 7+, CentOS 7+, Fedora, Debian 8+, Ubuntu 16.This leads to it's the init system. 04+, Arch, openSUSE — you're using systemd. PID 1. The parent of all processes.

Older systems (and some stubborn holdouts) use SysV init. update-rc.Runlevels. Scripts in /etc/init.Which means chkconfig. Also, d/. d Simple as that..

You'll also see Upstart on Ubuntu 14.But it's dead. On the flip side, 04 and some embedded gear. Don't learn it unless you're maintaining a museum It's one of those things that adds up..

For the rest of this article, I'll focus on systemd — because that's what you're actually using — but I'll cover the legacy commands too. You'll still run into them in documentation, Stack Overflow answers from 2014, and the occasional ancient server nobody dares touch Simple, but easy to overlook. Surprisingly effective..

Why This Matters More Than You Think

Most people learn systemctl start nginx and call it a day. Then they reboot the server and wonder why the site is down It's one of those things that adds up..

Enable ≠ Start.

This is the single most common mistake. systemctl start fires it up right now. systemctl enable creates symlinks so the service starts at boot. You usually want both Small thing, real impact. Turns out it matters..

systemctl enable --now nginx

That --now flag saves you a separate command. On top of that, it enables and starts in one shot. Muscle memory this.

But there's more. Security sandboxes. Resource limits. So conflicts. Services have dependencies. Ordering. If you don't understand the unit file, you're guessing.

And guessing in production is how you get paged at 3 AM.

How systemd Actually Works

systemd doesn't just run scripts. It manages units — configuration files that describe services, sockets, timers, mount points, swap files, even paths to watch.

Unit files live in three places (in priority order):

1. /etc/systemd/system/ — your custom units, overrides, the stuff you create
2. /run/systemd/system/ — runtime units, generated at boot
3. /usr/lib/systemd/system/ — vendor units from packages

When you install nginx via apt or dnf, it drops a unit file in /usr/lib/systemd/system/nginx.service. You don't edit that file directly. Updates will overwrite it Worth knowing..

Instead, you override it.

The Unit File Anatomy

Crack open a service unit. systemctl cat nginx.service shows you the whole thing, including any drop-in overrides.

[Unit]
Description=A high performance web server
After=network.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t
ExecStart=/usr/sbin/nginx
ExecReload=/usr/sbin/nginx -s reload
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Let me break down what matters:

[Unit] — Metadata. Description shows in systemctl status. After= and Requires= define ordering. After=network.target means "start after network is up." But network.target doesn't guarantee connectivity — just that the network stack is initialized. Big difference The details matter here..

[Service] — The meat. Type= tells systemd how the daemon behaves:

• simple (default) — process stays in foreground. systemd considers it started immediately.
• forking — traditional daemon forks and exits. systemd waits for the parent to exit, then tracks the child via PID file.
• oneshot — runs once, exits. Good for scripts.
• notify — daemon sends sd_notify() when ready. Best for custom apps.
• dbus — acquires a D-Bus name.

ExecStartPre= runs before the main command. Perfect for config tests (nginx -t). If it fails, the service doesn't start.

PrivateTmp= gives the service its own /tmp namespace. Security win. There's also ProtectSystem=, ProtectHome=, NoNewPrivileges=, CapabilityBoundingSet= — a whole sandboxing toolkit most people ignore Not complicated — just consistent. No workaround needed..

[Install] — This is what enable reads. WantedBy=multi-user.target means "when the system reaches multi-user target (normal boot), start me." Other targets: graphical.target (GUI), network-online.target (network actually working), shutdown.target.

Enabling and Disabling — What Actually Happens

systemctl enable nginx creates a symlink:

/etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service

That's it. wantsdirectory. Day to day, when systemd processesmulti-user. A symlink in a .target, it sees the symlink and starts the unit.

disable removes the symlink. The unit file stays put. The service won't start at boot — but you can still start it manually Less friction, more output..

mask is the nuclear option. Even manual start fails. Because of that, it symlinks to /dev/null. Use mask when you never want a service running — like systemd-resolved on a server running Pi-hole, or bluetooth on a headless box Easy to understand, harder to ignore. Still holds up..

systemctl mask bluetooth
systemctl unmask bluetooth  # to undo

Checking Status Like You Mean It

systemctl status nginx gives you the highlight reel:

● nginx.service - A high performance web server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2024-01-16 14:23:12 UTC; 2h 14min ago
     Docs: man:nginx(8)
  Process:

nginx -g "daemon off;"
Main PID: 1234 (nginx)
   Tasks: 2 (limit: 4915)
   Memory: 15.2M
   CPU: 450ms
   CGroup: /system.slice/nginx.

Jan 16 14:23:12 server01 systemd[1]: Starting A high performance web server...
Jan 16 14:23:12 server01 nginx[1234]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Jan 16 14:23:12 server01 nginx[1234]: nginx: the configuration file /etc/nginx/nginx.conf test is successful
Jan 16 14:23:12 server01 systemd[1]: Started A high performance web server.

This output reveals critical diagnostics: the exact command executed (`Main PID`), resource consumption (`Tasks`, `Memory`), and crucially, the service's own stdout/stderr logged via the journal (since `StandardOutput=journal` is the default). When `status` shows `Active: failed` or `Active: inactive (dead)`, the journal lines immediately above pinpoint *why*—whether it's a config error from `ExecStartPre`, a missing dependency, or a crash signaled by the main process exiting.

### Beyond `status`: Deep Diving with `journalctl`
While `systemctl status` gives a snapshot, `journalctl` unlocks temporal and filtered debugging:
- **Real-time tailing**: `journalctl -u nginx -f` follows logs for *just* this unit, isolating signal from noise.
- **Boot-specific issues**: `journalctl -u nginx -b -1` shows logs from the previous boot—essential for diagnosing intermittent startup failures that vanish after a manual `start`.
- **Priority filtering**: `journalctl -u nginx -p err..alert` surfaces only critical errors, skipping informational noise.
- **Kernel ring buffer correlation**: Add `-k` to see kernel messages alongside service logs (e.g., OOM killer events: `journalctl -u nginx -k -p err`).

A common oversight: services with `PrivateTmp=true` (or `ProtectTmp=yes`) isolate `/tmp` and `/var/tmp`. If your app expects to share temporary files with other processes via `/tmp`, it will fail silently *only* when sandboxing is enabled. Check `journalctl` for `EACCES` or `ENOENT` errors on `/tmp` paths—then either adjust the app to use `XDG_RUNTIME_DIR` (via `RuntimeDirectory=`) or relax the sandbox *judiciously* with `PrivateTmp=no` (though prefer fixing the app).

### The Silent Killer: Watchdog and `ExecStop`
Even seemingly healthy services can harbor time bombs. Consider a service that occasionally hangs:

[Service] ExecStart=/usr/local/bin/my-daemon WatchdogSec=30

[Service]
ExecStart=/usr/local/bin/my‑daemon
WatchdogSec=30
ExecStop=/usr/local/bin/my‑daemon‑stop

When a unit declares WatchdogSec, systemd expects the process to call sd_notify(…WATCHDOG=1…) at least once every WatchdogSec interval. If the daemon stalls, systemd will kill it and emit a watchdog timeout message in the journal. A common pitfall is that the watchdog is enabled, but the binary never forvent the sd_notify call, so the staged ExecStop= never runs. Inspect the journal for “Watchdog timeout” lines, and either patch the daemon to signal the watchdog or disable the feature with WatchdogSec=0.

Most guides skip this. Don't Worth keeping that in mind..

ExecStop and Graceful Teardown

ExecStop= is the counterpart to ExecStart=. If the process dies abruptly, systemd will invoke ExecStop= so that the daemon can release resources, flush caches, or perform a graceful shutdown. A mis‑configured ExecStop= can leave dangling sockets or partial state; the journal will show ExecStop= errors or SIGTERM/SIGKILL transitions. Use systemctl status to see the exit status of the stop command; if it fails, you’ll often find the cause in the ExecStop= line of the unit file.

systemctl show – The Unit Property Dump

Beyond the human‑friendly status, systemctl show <unit> prints every property the unit inherits, including MainPID, ExecMainStartTimestamp, ExecMainExitTimestamp, FragmentPath, Selective=, and all the Environment= settings. This is invaluable when you need to confirm that an environment variable is actually being passed to the process or when you suspect that a ConditionPathExists= check is preventing startup That alone is useful..

systemctl show nginx.service | grep -E 'MainPID|ExecMainStartTimestamp|ExecMainExitTimestamp'

Boot‑Time Analysis

systemd-analyze informazioni and systemd-analyze blame give you a high‑level view of what’s slowing the boot:

• systemd-analyze blame lists every unit with its elapsed time, sorted descending. If your web server isobjective slow, it will appear near the top.
• systemd-analyze critical-chain shows the dependency chain that forces the boot to wait. If a service is stuck because it’s waiting for another that never starts, the chain will reveal the culprit.

Real‑Time Resource Monitoring TcG

systemd-cgls and systemd-cgtop expose the cgroup hierarchy that systemd builds under /sys/fs/cgroup. These tools let you see which services are hogging CPU or memory in real time, and Blueprinting them can help you decide whether to enable MemoryMax= or CPUQuota= for a unit.

systemd-cgtop -u nginx.service

One‑Shot and Ad‑hoc Services

When experimenting with new binaries or testing configuration changes, systemd-run is a quick way to fire off an isolated unit:

systemd-run --unit=test-foo.service --property=ExecStart=/usr/bin/foo --property=RemainAfterExit=yes

The unit lives only for the lifetime of the process; its logs are automatically collected in the journal.

Escaping Unit Names

Unit names are derived from file names, but special characters (spaces, slashes, etc.) must be escaped. systemd-escape turns a string into a valid unit name fragment:

systemd-escape /var/log/nginx
# → var-log-nginx.service

Use this when you need to refer to units that were generated from arbitrary paths.

Handling OOM Situations

systemd-oomd is a newer OOM killer that works with systemd’s cgroup hierarchy. If a service is killed by the kernel’s OOM killer, the journal will contain a line like:

``

…the journal will contain a line like:

kernel: Out of memory: Kill process 12345 (nginx) score 847 or sacrifice child
kernel: Killed process 12345 (nginx) total-vm:123456kB, anon-rss:98765kB, file-rss:0kB

When you see such entries, the first step is to verify whether the kill originated from the kernel’s OOM killer or from systemd-oomd. The latter adds a more informative message:

systemd-oomd[123]: Memory cgroup out of memory: Kill process 12345 (nginx) score 900

Diagnosing the source

1. Check the cgroup OOM killer status

   systemctl status systemd-oomd
   journalctl -u systemd-oomd -b
   

   If systemd-oomd is active and you see its messages, the kill was managed by systemd’s OOM daemon.

2. Inspect the service’s cgroup

   systemd-cgls -k nginx.service
   cat /sys/fs/cgroup/systemd/nginx.service/memory.oom_control
   

   The oom_control file shows oom_kill_disable and the current under_oom flag.

3. Look at the journal for the service itself

   journalctl -u nginx.service --since "5 minutes ago"
   

   You’ll often see a abrupt termination line (Main process exited, code=killed, status=9/KILL) right before the OOM message Turns out it matters..

Mitigation strategies

Testing your changes

After editing the unit file, validate the new limits without a full reboot:

systemctl daemon-reload
systemctl show nginx.service | grep -E 'MemoryMax|MemoryLow|OOMScoreAdjust'
# Then trigger a memory‑stress test, e.g.:
stress-ng --vm 2 --vm-bytes 1500M --timeout 30s
journalctl -u nginx.service -f   # watch for OOM messages

If the service stays alive and the journal shows no OOM kills, your adjustments are effective It's one of those things that adds up..

Putting it all together

When a service disappears because of memory pressure, the diagnostic flow is:

1. Journal → confirm OOM kill and identify the killer (kernel vs. systemd-oomd).
2. Cgroup inspection → verify current limits and oom_control state.
3. Adjust unit parameters → MemoryMax/, MemoryLow, OOMScoreAdjust, or tweak systemd-oomd settings.