Shield Apache Sites From Bots

Our sites sit behind Cloudflare and currently suffer from sudden traffic spikes that end in 524 timeout errors. The common pattern is a flood of abusive requests—sometimes pure scraping, sometimes obvious spam or probing—hammering several custom-PHP domains on the same dedicated server.

The dediocated server is managed by Digital Pacific Australia. We can provide Cpanel access to sites as well as WHM access.

Regardless of current rules at Clioudflare and basic efforts on the server, the attackers get through, so I need deeper, server-level and edge-level hardening that actually throttles or blocks them before they exhaust resources.

Here’s what I expect:

• Analyse recent access/error logs plus Cloudflare analytics to pinpoint offending IP ranges, user-agents and request patterns.
• Build and deploy robust rulesets—mod_security, fail2ban, custom .htaccess directives, or Cloudflare WAF/rate-limiting—as appropriate for Apache.
• Recommend changes to our existing firewall for Digital Pacific to implement as needed. Allow legitimate crawlers continue to index the sites while bad bots are challenged, throttled or black-holed.
• Provide a concise report of changes, commands and rule locations so I can maintain them later.

Acceptance criteria:
1. Abusive traffic is cut to a negligible level without blocking real users or major search-engine bots.
2. Cloudflare 524 errors disappear under normal load-test conditions.
3. Documentation is delivered in a simple markdown or text file.

If you’ve battled mixed-type bot storms on Apache before and can work over SSH with minimal downtime, I’d like to start right away.