Fixing Apache OOM Kill on High-Memory Servers

on

Resolving Apache Out-of-Memory (OOM) Kills on High-Memory Servers

On a high-memory server with over 125 GB of RAM, it may come as a surprise when Apache crashes due to an out-of-memory (OOM) event. Recently, we encountered such an issue, where the Apache HTTP server was killed by the Linux OOM killer — despite more than 100 GB of free memory. Here’s a breakdown of what happened, how we investigated, and what we did to fix it.

🧠 The Problem

Apache crashed unexpectedly, and the service log revealed:

apache2.service: A process of this unit has been killed by the OOM killer.
apache2.service: Failed with result 'oom-kill'.

We then checked system logs using dmesg:

Out of memory: Killed process 1500255 (apache2) total-vm:118204172kB, anon-rss:116992196kB

A single Apache process consumed over 110 GB of RAM, which triggered the OOM killer and brought down the server.

🔍 Investigation

Apache was running with the mpm_event module. The configuration looked like this:

<IfModule mpm_event_module>
    StartServers             2
    MinSpareThreads         25
    MaxSpareThreads         75
    ThreadLimit             64
    ThreadsPerChild         25
    MaxRequestWorkers      150
    MaxConnectionsPerChild   0
</IfModule>
⚠️ Issue: MaxConnectionsPerChild was set to 0, meaning Apache child processes never recycle. If one process leaks memory, it can grow indefinitely.

✅ The Fix

To prevent long-lived Apache workers from consuming all available memory, we changed:

MaxConnectionsPerChild 0

to:

MaxConnectionsPerChild 1000

This causes Apache to restart each worker process after 1000 connections, limiting memory growth over time.

After applying this, we restarted Apache:

sudo systemctl restart apache2

🛡️ Optional Best Practices

1. Monitor Apache Memory

ps -o pid,ppid,cmd,%mem,%cpu --sort=-%mem | grep apache2

2. Use mod_status

<Location "/server-status">
    SetHandler server-status
    Require local
</Location>

3. Consider PHP-FPM

If running PHP apps, switch from mod_php to php-fpm for better memory control and process isolation.

4. Add Alerts

Tools like Netdata, Monit, or custom scripts can alert you when memory or swap is dangerously high.

✅ Result

After implementing MaxConnectionsPerChild 1000, Apache became stable and memory usage was controlled. This simple tweak prevented future crashes and improved server resilience.

Have you experienced something similar? Share your fix or ask for help below!

Comments

Post a Comment