Searching for Spammers

There are two aspects to dealing with spam for a server administrator:

  1. Inbound spam to users
  2. Outbound spam from compromised scripts

Both need very different approaches to help detect, remove and resolve.

Inbound spam to users

Inbound spam is the scourge of the modern internet and, the inconvenience to users aside, can cause serious performance and resource issues on the server. These can affect both the server overall and the timely deliver of clean email in particular.

The best way to tackle inbound spam is at the entry point into the server – the MTA, i.e. exim the SMTP server of choice for cPanel. By blocking spam before it has even entered the server you save both on server resources used when delivering the email in addition to 3rd party tools to help detect spam further along the email relay process.

To do this you need to do work at the RCPT stage of the SMTP protocol. This occurs during the transaction between the sender and recipient SMTP servers and comes before the actual body of an email arrives on a server. The primary form of spam attack is the Dictionary Attack:

A common technique for spammers to use is what is known as a dictionary attack on a domain. A dictionary attack, in our context, is a single SMTP connection that attempts to send email from a spam source to a random set of names on our domain, e.g., in the hope that one of the many hundreds that we try will get a hit and deliver our spam.

This technique is used by spammers mainly because most people don’t advertise their email addresses (due to spam!) and they want to access this untapped market.

To prevent this type of spam getting through, it is essential that you do not use the Default Address (catchall) feature within cPanel to receive emails wherever possible. You should always setup specific Forwarders (aliases) for any email addresses you use and set the Default Address to :fail: for each domain.

By using :fail: exim will automatically reject email at the SMTP RCPT stage and make dictionary attacks redundant. Additionally, you can use exim ACLs to block such spammers who repeatedly perform dictionary attacks to further relieve the server of the load from dealing with them.

From a server performance perspective, it is essential that you use :fail: and not :blackhole: with email addresses or the Default Address to block such spam. More information about the reasoning for this is presented here.

Another preventative measure is to enable the WHM options:

WHM > Exim Configuration Editor > Verify the existance of email senders.
WHM > Exim Configuration Editor > Use callouts to verify the existance of email senders.

These two options have exim check that any server that attempts to relay email to your server can actually receive email in reply. This is part of the RFC requirements of an SMTP server and the inability of a server to do so indicates a likely spammer.

There are numerous other checks that you can also perform at the SMTP RCPT stage in exim ACLs. Examples are using RBL checks to reject email from IP addresses that originate from IP addresses that are know to harbour spammers, e.g.:

deny message = Message rejected - $sender_fullhost is in an RBL, see $dnslist_text
	!hosts = +relay_hosts
	!authenticated = *
	dnslists = :

You can also check the format of email headers to ensure that they’re RFC compliant, which many spam servers are not. A typical example is checking the SMTP HELO/EHLO protocol command to ensure it’s correctly structured, e.g.:

deny message = HELO/EHLO set to my IP address
    condition = ${if match {$sender_helo_name}{} {yes}{no}}

(where is your servers main IP address)

deny message = EHLO/HELO does not contain a dotted address
    condition = ${if match{$sender_helo_name}{\\.}{no}{yes}}

Finally, once the email has passed through these hoops, you can implement a 3rd party application to scan emails and tag them as likely spam. cPanel has an inbuilt solution that uses SpamAssassin to score email likely to be spam. You can then have such emails filtered to a special account or the client can filter such emails based on the email header record modifications made by SpamAssassin.

An alternative is to use a more thorough tool such as MailScanner which can be very effective at scoring spam emails.

A free installation tool is available for cPanel servers from us here or as a paid service here.

However, a cPanel server using such a tool is not supported by cPanel and would have to be removed/disabled before cPanel would investigate any email related issues should you need support.

Outbound spam from compromised scripts

Outgoing spam is likely to come from two sources:

  1. Indirectly from a compromised web script in a clients account
  2. Directly from a client

The starting point for both will be the exim mainlog:

/var/log/exim_mainlog # (Linux)
/var/log/exim/mainlog # (FreeBSD)

For the purpose of this document I am going to assume a Linux OS.

The most laborious way to track messages down is to trawl the exim mainlog and to look for anomalous behaviour. This is actually very difficult to do and you really need to narrow down exactly what you are looking for.

Tracking down spammers is a difficult affair, but can be made easier with some preparation of your servers environment. I would strongly advise that you add the following to the exim configuration to enable some extended logging that greatly improves the ease in tracking down on-server spammers:

Go to WHM > Exim Configuration Manager > Advanced Editor.  Search for “log_selector” and ensure that the value for this configuration option includes at least the following:

+arguments +subject +received_recipients

If you made any changes, click “Save” at the bottom.

This tells exim to log the path on disk from where the email was executed and the subject of the email. You can then interrogate the exim mainlog more easily.

The best way to do this is to obtain the original email header from the spam originating from your server. This you should receive either from the person reporting the spam, or from remnants of a spam attack in the exim mail queue.

The part required in the email is the exim message id in the Received: header line within the email header of the spam.

As an example, take the following email header:

Return-path: <>
Received: from [] (
    by with esmtps (TLSv1:AES256-SHA:256)
	(Exim 4.52)
	id 1FZ8z3-0006M4-Do
	for; Thu, 27 Apr 2006 17:04:49 +0100
Received: from forums by with local (Exim 4.43)
    id 1FZ8zt-0005lz-E7
	for; Thu, 27 Apr 2006 12:05:41 -0400
	Subject: Buy Me!

The Received: header lines are added to the email header, so the original Received: line that we’re interested in is:

Received: from forums by with local (Exim 4.43)
    id 1FZ8zt-0005lz-E7
    for; Thu, 27 Apr 2006 12:05:41 -0400

And the id we want is 1FZ8zt-0005lz-E7

This is the unique identifier for this email that has originated from the server. With this, we can follow the exim transaction on the server to see how it was processed using:

grep 1FZ8zt-0005lz-E7 /var/log/exim_mainlog

(be aware that the exim_mainlog files may have been rotated so you may have to expand compressed archives and search them instead)

This transaction may look something like this:

2006-04-27 17:43:41 1FZ8zt-0005lz-E7 <= U=nobody P=local S=4001 T="Buy Me!"
2006-04-27 17:43:50 cwd=/home/ClientX/public_html/phpBB/ 5 args: /usr/sbin/exim -Mc 1FZ8zt-0005lz-E7
2006-04-27 17:43:53 1FZ8zt-0005lz-E7 => R=lookuphost T=remote_smtp [] X=TLSv1:AES256-SHA:256
2006-04-27 17:43:53 1FZ8zt-0005lz-E7 Completed

In this example, we can see that the email originated from the nobody user locally on the server. This means that the likely spam was sent from a script on the server. The nobody user is used to run the Apache web server and is the default username and group that Apache will execute web scripts as. Two things can affect this:

  1. suexec, if enabled, will run CGI scripts as the owner of the script file, typically the cPanel account name
  2. phpsuexec, if enabled, will run PHP scripts in the same manner as CGI scripts

suexec is typically always enabled on web servers and phpsuexec may or may not be. If phpsuexec is not enabled, then in all likelihood, the script run under the nobody account will be a PHP script.

From the example above we can see that a script was run from with the /home/ClientX/public_html/phpBB/ directory on the server, which would suggest a compromised PHP script within that directory.

Here’s another example of a spam originating from a client instead of a script. This can happen either with malicious intent, or if the clients PC has been compromised by a virus or worm:

2006-04-27 17:54:51 1FZ9lT-000707-O2 <= ([]) [] P=esmtpa S=715 id=ABCDEFG T="Buy Me!"
2006-04-27 17:54:51 cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1FZ9lT-000707-O2
2006-04-27 17:54:51 1FZ9lT-000707-O2 => R=boxtraper_autowhitelist T=boxtrapper_autowhitelist
2006-04-27 17:54:52 1FZ9lT-000707-O2 => R=lookuphost T=remote_smtp [] X=TLSv1:AES256-SHA:256
2006-04-27 17:54:52 1FZ9lT-000707-O2 Completed

In this example, the key part is:

This shows that the email was authenticated for relaying using SMTP AUTH (i.e. fixed_plain) and the username from that clients PC.

As you can see, there is a great depth to the amount of work needed to track down spammers on a server, plus there’s the additional work of closing holes in insecure scripts if they are the cause. Some instances can be much more complex and require trawling through the Apache logs for domains in /usr/local/apache/domlogs/* which is not a trivial matter.

The best security from such exploitation is to keep your server secure and to be aware of who and what you allow on your server.