Optimizing Server Resources by Blocking Unnecessary Traffic

Why We Took This Approach

As a company managing multiple job sites across various European countries, we continuously monitor and optimize our server resources to ensure the best experience for real users. Recently, we noticed an unusual surge in traffic from countries that are not part of our target audience, such as China and India. While some of this traffic may be legitimate, the vast majority is automated—bots, scrapers, or other aggregators that consume resources without adding value.

Uncontrolled bot traffic increases server load, bandwidth consumption, and infrastructure costs, making it a major concern for operational efficiency. Instead of letting these requests drain our resources, we decided to implement a proactive firewall-based traffic filtering system. By blocking non-essential traffic at the network level, we significantly reduce unnecessary load, improve site performance, and save both energy and costs.

Our Working Method

We built a Python-based solution to systematically identify, log, and block unwanted IPs, following a structured process:

1. Extract IP addresses from logs

   • The input can be any log file, such as Apache/Nginx access logs or even a live server-status page retrieved via wget.
   • We parse these logs to extract unique IP addresses.

2. Check the geographic origin of IPs

   • Instead of blocking blindly, we use ipinfo.io to determine the country of each IP.
   • If an IP belongs to a country like China (CN), we flag it for further action.

3. Store processed IPs efficiently

   • To avoid redundant API calls, we maintain a geo_data.json file containing already-checked IPs and their respective countries.
   • If an IP has already been processed, we skip rechecking it, improving efficiency.

4. Apply firewall rules dynamically

   • Using UFW (Uncomplicated Firewall), we block flagged IPs directly from accessing the server.
   • The rules are applied only to new, unblocked IPs, ensuring we don’t waste resources.

5. Maintain a log of blocked IPs

   • Blocked IPs are appended to blocked_cn_ips.txt, so we have a historical record of all restricted addresses.
   • If an IP is already blocked, we skip adding it again, preventing redundant firewall rules.

Why This Matters

By implementing this structured approach, we ensure:

• Better server performance: Less bot traffic means more resources available for real users.
• Reduced bandwidth costs: No need to serve pages to scrapers that will never convert into customers.
• Lower energy consumption: Blocking unnecessary traffic at the firewall level is far more efficient than letting the server process and reject requests dynamically.
• Scalability: This system can be automated via a cron job and adapted to different log formats.

Our approach is simple, efficient, and highly effective in optimizing traffic and server resources. Instead of letting non-essential requests bog down performance, we take control at the firewall level—saving time, money, and energy while keeping our job sites fast and responsive.

🚀 Smart security, better efficiency, and a cleaner network—this is how we optimize our infrastructure!

Take a look at this project! It's free, simple, and open-source.

https://github.com/OnlineSolutionsGroupBV/DropIPsByCountry/  

/home/downloads/DropIPsByCountry