Just found a rant I made in a forum a year or so ago. It's good enough to warrant reposting here:
I disagree with assumptions that keeping your machine patched and up to date is the catch-all end-all in terms of security(my apologies for oversimplifying, not trying to start a flamewar). For example, what patch do I apply to make sure that a 4 year old version of phpBB isnt used to execute shell commands?
/rant mode ON
There will always be 0day, and there is more and more effort into keeping exploits private for as long as possible. This means that your OS won't be patched in time to stop it. Patching is essential, of course! But you need to make it as HARD as possible to even toy with/footprint/scan your server, so it's not an easy target.
Spammers and attackers are using new, sophisticated, and downright creative ways to get their tasks done - be it sending mail, hosting proxies, running irc bots/botnets, serving virii, cracking passwords, subverting firewalls, DDOS, uploading files, serving warez, etc.
It takes a pretty comprehensive plan to tackle all of these things at once. I'll try to split it into some of the most obvious steps that you can take:
1. Default deny packet filtering rules
- we kin argue this all day, but the fact is that it tends to work. Read up on iptables/ipf, practice on a test host, check all the normal services, try it on a production host over a weekend, and pray to god that you understand what you did?
2. Track/filter all script generated email
- depending on your MTA, this might be easy or difficult. I had to write a custom patch for qmail to invoke spamassassin whenever someone called /usr/sbin/sendmail or telnets to 127.0.0.1 on port 25. Works great.
- Logging is essential. If possible, download one of the many patches to PHP that will log the URL/CWD of any script attempting to send mail, or use phpsuexec