Email
Anti-Phishing and Spear-Phishing Version 2
15/06/09 11:31
Update 11 October 2009: This has now been moved to www.ScamNailer.com. Please check there for all future information and updates to this package.
Update 20 September 2009: The Google-hosted data file has been moved to SourceForge, so I have updated the URL it downloads it from. You need to update your script to the new version 2.05.
Update 16 June 2009: I have changed the rule structures to make them considerably faster than the old ones. Download the updated script from the link below.
I have acquired a new reliable feed of email addresses used in phishing attacks. These addresses have all been checked by real people, and they come from a very reliable and well-known source.
The new data file is provided by means of DNS and an Anycast network, which makes it pretty resilient to attack. The previous spear-phishing data is gathered from the project hosted by Google in the traditional way, that hasn’t changed.
I have updated my script so that it fetches both sets of data. It makes use of a temporary directory under /var/cache, which is configurable at the start of the script, and which needs to be writable by the user the scripts runs as (normally just ‘root’ so this doesn’t present any problem at all to most people).
You can download version 2.05 of the script.
If you are not using MailScanner with this script, you will need to comment out or delete the line that mentions “service MailScanner reload” about 1/3 of the way down the script (search and ye shall find!).
For more explanation of this whole problem and the way this script works, please refer back to my earlier article.
Update 20 September 2009: The Google-hosted data file has been moved to SourceForge, so I have updated the URL it downloads it from. You need to update your script to the new version 2.05.
Update 16 June 2009: I have changed the rule structures to make them considerably faster than the old ones. Download the updated script from the link below.
I have acquired a new reliable feed of email addresses used in phishing attacks. These addresses have all been checked by real people, and they come from a very reliable and well-known source.
The new data file is provided by means of DNS and an Anycast network, which makes it pretty resilient to attack. The previous spear-phishing data is gathered from the project hosted by Google in the traditional way, that hasn’t changed.
I have updated my script so that it fetches both sets of data. It makes use of a temporary directory under /var/cache, which is configurable at the start of the script, and which needs to be writable by the user the scripts runs as (normally just ‘root’ so this doesn’t present any problem at all to most people).
You can download version 2.05 of the script.
If you are not using MailScanner with this script, you will need to comment out or delete the line that mentions “service MailScanner reload” about 1/3 of the way down the script (search and ye shall find!).
For more explanation of this whole problem and the way this script works, please refer back to my earlier article.
Comments
Anti Spear Phishing
13/01/09 13:50
Update 2009-October-11: This package is now hosted at www.scamnailer.com. Please check there for all future information and updates.
Update 2009-June-15: There is now a brand new additional data feed of known phishing email addresses, which I have added to my script.
Spear phishing is a technique used by spammers and scammers to try to get your email username and password. They send you an email claiming to be from your email provider, in which they say that your account will be deleted unless you supply them with your username and password “for authentication” or some other similar ruse.
If they get your username and password, they then use your email account and email provider to send out millions of spam messages. Because the spam comes from a genuine email system (yours!) it will be accepted by most sites and will automatically pass many spam checks.
I have written a script which takes a file of addresses commonly used in these attacks. It also allows an additional list of addressed you can add to. From these, it generates a set of SpamAssassin rules that detect the presence of these addresses, which can be used in MailScanner to stop the spear-phishing attacks completely.
Download the script here. Note that the script is gzipped to ensure your browser doesn’t do anything silly when fetching it, so you’ll need to “gunzip” it before doing anything with it. To start with, just copy it into your “/etc/cron.hourly” directory, and run the command “chmod a+rx /etc/cron.hourly/Spear.Phishing.Rules” to make it run every hour.
It is pretty much a finished script, and is directly usable by you guys without you having to do much to it except read the settings at the top and tweak the filenames if you want to change where it puts things.
I have taken a lot of care to ensure that this won't match any false alarms, I don't just dumbly look for the strings in any surrounding text, which certain commercial AV vendors have been caught doing in the past!
I make a suggestion in the comments at the top of the script about how I use the rule within MailScanner, you probably want to do something similar, and not just delete anything that matches, just in case you do get any false alarms.
It also looks for numbers at the end of the username bit of the address, and assumes that these are numbers which the scammers may change; so if it finds them, it replaces them with a pattern that will match any number instead. There's starting to be a lot of this about, as it's the easiest way for the scammers to try to defeat simple address lists targeted against them, while still being able to remember what addresses they have to check for replies from your dumb users.
I thought I would make it a tiny bit harder for them...
You can also add addresses of your own (which can include "*" as a wildcard character to mean "any series of valid characters" in the email address), one address per line, in an optional extra file. Again, read the top of the script and you'll see it mentioned there. That file is optional, it doesn't matter if it doesn't exist. As a starter, you might want to put
m i c h a e l l o u c a s * @ g m a i l . c o m
(without the extra spaces) in that file, as it will nicely catch a lot of "Job opportunity" spams.
It looks for any of these addresses appearing **anywhere** in the message, not just in the headers. So if you start talking to people about these addresses, don't be surprised when the messages get caught by the trap.
It does a "wget", so make sure you have that binary installed, or else change the script to fetch the file by some other means.
The very end of the script does a "service MailScanner restart", so if you need some other command to restart MailScanner or your SpamAssassin setup, then edit it for your system. It needs to be a "restart" and not a "reload" as I have to force it to re-build the database of SpamAssassin rules. If you don’t use MailScanner, but do use “spamd” in some setup or other, then a simple “service spamd restart” would do at the end of the script.
My aim was that, on a RedHat system running MailScanner, you could just copy the script into /etc/cron.hourly and make it executable, and it will just get on with the job for you. I do advise you read the bit in the script about "SpamAssassin Rule Actions" though.
Please do let me know how you would like me to improve it, and tell me what you think of it in general. (be polite, now!)
Update 13th January 2009:
A colleague on the MailScanner mailing list has made this simpler to use. You don’t have the flexibility of adding your own addresses to the list, but you can get the latest list along with all your regular SpamAssassin updates with the “sa-update” command.
Here are his instructions:
wget http://www.bastionmail.co.uk/spear.txt
sa-update --import spear.txt
Add “spear.bastionmail.com” to the list of channels that you update from (either add “--channel spear.bastionmail.com” to your sa-update command, or add “spear.bastionmail.com” to the file pointed to by the sa-update “--channelfile” command-line option).
Add the key “06EF70A3” to the trusted keys (either add “--gpgkey 06EF70A3” to your sa-update command, or add “06EF70A3” to the file pointed to by the sa-update “--gpgkeyfile” command-line option).
Then these SpamAssassin rules will be automatically updated every time your system runs the “sa-update” command, which is daily on a standard MailScanner system.
Update 2009-June-15: There is now a brand new additional data feed of known phishing email addresses, which I have added to my script.
Spear phishing is a technique used by spammers and scammers to try to get your email username and password. They send you an email claiming to be from your email provider, in which they say that your account will be deleted unless you supply them with your username and password “for authentication” or some other similar ruse.
If they get your username and password, they then use your email account and email provider to send out millions of spam messages. Because the spam comes from a genuine email system (yours!) it will be accepted by most sites and will automatically pass many spam checks.
I have written a script which takes a file of addresses commonly used in these attacks. It also allows an additional list of addressed you can add to. From these, it generates a set of SpamAssassin rules that detect the presence of these addresses, which can be used in MailScanner to stop the spear-phishing attacks completely.
Download the script here. Note that the script is gzipped to ensure your browser doesn’t do anything silly when fetching it, so you’ll need to “gunzip” it before doing anything with it. To start with, just copy it into your “/etc/cron.hourly” directory, and run the command “chmod a+rx /etc/cron.hourly/Spear.Phishing.Rules” to make it run every hour.
It is pretty much a finished script, and is directly usable by you guys without you having to do much to it except read the settings at the top and tweak the filenames if you want to change where it puts things.
I have taken a lot of care to ensure that this won't match any false alarms, I don't just dumbly look for the strings in any surrounding text, which certain commercial AV vendors have been caught doing in the past!
I make a suggestion in the comments at the top of the script about how I use the rule within MailScanner, you probably want to do something similar, and not just delete anything that matches, just in case you do get any false alarms.
It also looks for numbers at the end of the username bit of the address, and assumes that these are numbers which the scammers may change; so if it finds them, it replaces them with a pattern that will match any number instead. There's starting to be a lot of this about, as it's the easiest way for the scammers to try to defeat simple address lists targeted against them, while still being able to remember what addresses they have to check for replies from your dumb users.
I thought I would make it a tiny bit harder for them... You can also add addresses of your own (which can include "*" as a wildcard character to mean "any series of valid characters" in the email address), one address per line, in an optional extra file. Again, read the top of the script and you'll see it mentioned there. That file is optional, it doesn't matter if it doesn't exist. As a starter, you might want to put
m i c h a e l l o u c a s * @ g m a i l . c o m
(without the extra spaces) in that file, as it will nicely catch a lot of "Job opportunity" spams.
It looks for any of these addresses appearing **anywhere** in the message, not just in the headers. So if you start talking to people about these addresses, don't be surprised when the messages get caught by the trap.
It does a "wget", so make sure you have that binary installed, or else change the script to fetch the file by some other means.
The very end of the script does a "service MailScanner restart", so if you need some other command to restart MailScanner or your SpamAssassin setup, then edit it for your system. It needs to be a "restart" and not a "reload" as I have to force it to re-build the database of SpamAssassin rules. If you don’t use MailScanner, but do use “spamd” in some setup or other, then a simple “service spamd restart” would do at the end of the script.
My aim was that, on a RedHat system running MailScanner, you could just copy the script into /etc/cron.hourly and make it executable, and it will just get on with the job for you. I do advise you read the bit in the script about "SpamAssassin Rule Actions" though.
Please do let me know how you would like me to improve it, and tell me what you think of it in general. (be polite, now!)
Update 13th January 2009:
A colleague on the MailScanner mailing list has made this simpler to use. You don’t have the flexibility of adding your own addresses to the list, but you can get the latest list along with all your regular SpamAssassin updates with the “sa-update” command.
Here are his instructions:
wget http://www.bastionmail.co.uk/spear.txt
sa-update --import spear.txt
Add “spear.bastionmail.com” to the list of channels that you update from (either add “--channel spear.bastionmail.com” to your sa-update command, or add “spear.bastionmail.com” to the file pointed to by the sa-update “--channelfile” command-line option).
Add the key “06EF70A3” to the trusted keys (either add “--gpgkey 06EF70A3” to your sa-update command, or add “06EF70A3” to the file pointed to by the sa-update “--gpgkeyfile” command-line option).
Then these SpamAssassin rules will be automatically updated every time your system runs the “sa-update” command, which is daily on a standard MailScanner system.
Installing Mailman
20/10/08 09:26
Fortunately, to make life very easy, there is a copy of the latest Mailman included with RedHat 5.2 or CentOS, so just
yum install mailman
yum update
and you’re on your way.
However, Mailman 2 currently does not support virtual email domains, so I have applied a small patch by hand to add this functionality.
The Mailman Installation Manual is very good and will walk you through all the configuration steps required, which shouldn’t take you more than an hour or two at most. You will find most of it has been done for you by the RPM packagers at RedHat. About the only bits you need to bother with are
If you are moving from Majordomo to Mailman, you may be interested in my majordomo2mailman script which will do all the hard work for you.
yum install mailman
yum update
and you’re on your way.
However, Mailman 2 currently does not support virtual email domains, so I have applied a small patch by hand to add this functionality.
The Mailman Installation Manual is very good and will walk you through all the configuration steps required, which shouldn’t take you more than an hour or two at most. You will find most of it has been done for you by the RPM packagers at RedHat. About the only bits you need to bother with are
- 7 Review your site defaults
- 8 Create a site-wide mailing list
- 11 Check the hostname settings
- 12 Create the site password
- 13 Create your first mailing list
If you are moving from Majordomo to Mailman, you may be interested in my majordomo2mailman script which will do all the hard work for you.
Converting Majordomo Mailing Lists to Mailman
20/10/08 09:08
This appears to be a fairly common problem, with very few decent solutions to the problem. It’s all very well converting lists over by hand if you only have a few, but if you have hundreds of them then that is not practical.
So I have taken a script originally written by Brad Marshall (b.marshall@cqu.edu.au) and fixed some bugs, extended it and improved it.
You can download the resulting majordomo2mailman script here.
If you run it as “majordomo2mailman --help” then it will show you how to use it.
You will need to edit the settings at the top of the script to match the layout of your server, as it needs a copy of the Majordomo lists directory and the Majordomo aliases file to work from. I wrote this to work with sendmail, but converting it to work with any other MTA should be trivial, you just need to bash your aliases database into a file that looks like a sendmail one, i.e. one alias per line, with the format
alias: value
on each line.
To start with, you might want to enable debugging, which you can do at the top of the script.
If you use it, I would greatly appreciate a small donation. I have an Amazon.co.uk wishlist. Thank you.
Don’t forget to make the majordomo2mailman script executable after you have downloaded it!
So I have taken a script originally written by Brad Marshall (b.marshall@cqu.edu.au) and fixed some bugs, extended it and improved it.
You can download the resulting majordomo2mailman script here.
If you run it as “majordomo2mailman --help” then it will show you how to use it.
You will need to edit the settings at the top of the script to match the layout of your server, as it needs a copy of the Majordomo lists directory and the Majordomo aliases file to work from. I wrote this to work with sendmail, but converting it to work with any other MTA should be trivial, you just need to bash your aliases database into a file that looks like a sendmail one, i.e. one alias per line, with the format
alias: value
on each line.
To start with, you might want to enable debugging, which you can do at the top of the script.
If you use it, I would greatly appreciate a small donation. I have an Amazon.co.uk wishlist. Thank you.
Don’t forget to make the majordomo2mailman script executable after you have downloaded it!
Talking from sendmail to Exchange over SMTP auth
05/06/08 09:42
There are various things you can do in Exchange, such as control who can address distribution lists, that can be restricted to authenticated senders only. So how do you make your sendmail box an authenticated sender?
Start at http://www.sendmail.org/~ca/email/auth.html -- about half way down it starts talking about “Using sendmail as a client with AUTH”. That tells you how to setup your sendmail box (which is the client) so that it talks SMTP auth to Exchange (which is the server).
Start at http://www.sendmail.org/~ca/email/auth.html -- about half way down it starts talking about “Using sendmail as a client with AUTH”. That tells you how to setup your sendmail box (which is the client) so that it talks SMTP auth to Exchange (which is the server).
Obfuscating Email Addresses for Web Pages
03/06/08 14:04
http://www.fingerlakesbmw.org/main/flobfuscate.php
That will generate a very obfuscated version of the HTML of an email address, suitable for putting on a web page, so there is far less chance that the spammers will be able to harvest it for their address lists.
That will generate a very obfuscated version of the HTML of an email address, suitable for putting on a web page, so there is far less chance that the spammers will be able to harvest it for their address lists.
Building an MX the Easy Way
06/02/08 12:15
Boot off RHEL5 disk 1.
linux rescue
Activate network interface eth0 (first interface).
Give IP and so on.
Use fdisk to create
/dev/sda1 /boot Linux 100Mb
/dev/sda2 Linux swap 2048Mb
/dev/sda3 / Linux all the rest
mkfs.ext3 /dev/sda1
mkfs.ext3 /dev/sda3
mkswap /dev/sda2
mkdir /mnt2
mount /dev/sda1 /mnt2
ssh crow ‘dump 0f - /dev/sda3’ | ( cd /mnt2 && restore -rf - )
umount /mnt2
mount /dev/sda3 /mnt2
ssh crow.ecs.soton.ac.uk ‘dump 0f - /dev/mapper/VolGroup00-LogVol00’ | ( cd /mnt2 && restore -rf - )
or else
ssh crow.ecs.soton.ac.uk ‘cd / && tar clf - .’ | ( cd /mnt2 && tar xvBpf - )
Fix /mnt2/etc/fstab so it points to all the right partitions.
If you really want to use partition labels, use the “e2label” command to set the label of each partition so that your shiny new /etc/fstab can find them. Syntax is obvious: /sbin/e2label device [ new-label ]
umount /mnt2
Installing Grub
Cloning an RHEL4 system using the RHEL5 rescue disc? You must use Grub from RHEL4 to setup a boot record for an RHEL4 system. So mount the (newly copied) root filesystem in /mnt2 and copy /mnt2/sbin/grub to /sbin/grub. Then follow the instructions below.
mkdir /boot
mount /dev/sda1 /boot
rm -rf /boot/boot
grub
grub> root (hd0,0)
grub> setup (hd0)
Edit /boot/grub/grub.conf and change the root command to (hd0,0) and the kernel root-filesystem argument to /dev/sda3.
Repeat that edit for all the other kernels available.
Unplug network interface
Reboot and it should boot from hard disk
cd /var/spool/mqueue.in
rm -f *
cd /var/spool/mqueue
rm -f *
cd /var/spool/MailScanner/quarantine
rm -rf *
cd ../incoming
rm -rf *
cd ../archive
rm -rf *
cd /var/log
Remove all old logs
service syslog restart
Fix ethernet and IP address in /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network
Fix /etc/hosts
Repair ownership and permissions of /home/* and /usr/local/share/clamav (and subdirectories).
Fix extra ClamAV databases so that “MailScanner --lint” runs correctly.
Reboot with network interface connected.
Re-register with RedHat network for yum updates, get the info from the Systems KB.
yum update
linux rescue
Activate network interface eth0 (first interface).
Give IP and so on.
Use fdisk to create
/dev/sda1 /boot Linux 100Mb
/dev/sda2 Linux swap 2048Mb
/dev/sda3 / Linux all the rest
mkfs.ext3 /dev/sda1
mkfs.ext3 /dev/sda3
mkswap /dev/sda2
mkdir /mnt2
mount /dev/sda1 /mnt2
ssh crow ‘dump 0f - /dev/sda3’ | ( cd /mnt2 && restore -rf - )
umount /mnt2
mount /dev/sda3 /mnt2
ssh crow.ecs.soton.ac.uk ‘dump 0f - /dev/mapper/VolGroup00-LogVol00’ | ( cd /mnt2 && restore -rf - )
or else
ssh crow.ecs.soton.ac.uk ‘cd / && tar clf - .’ | ( cd /mnt2 && tar xvBpf - )
Fix /mnt2/etc/fstab so it points to all the right partitions.
If you really want to use partition labels, use the “e2label” command to set the label of each partition so that your shiny new /etc/fstab can find them. Syntax is obvious: /sbin/e2label device [ new-label ]
umount /mnt2
Installing Grub
Cloning an RHEL4 system using the RHEL5 rescue disc? You must use Grub from RHEL4 to setup a boot record for an RHEL4 system. So mount the (newly copied) root filesystem in /mnt2 and copy /mnt2/sbin/grub to /sbin/grub. Then follow the instructions below.
mkdir /boot
mount /dev/sda1 /boot
rm -rf /boot/boot
grub
grub> root (hd0,0)
grub> setup (hd0)
Edit /boot/grub/grub.conf and change the root command to (hd0,0) and the kernel root-filesystem argument to /dev/sda3.
Repeat that edit for all the other kernels available.
Unplug network interface
Reboot and it should boot from hard disk
cd /var/spool/mqueue.in
rm -f *
cd /var/spool/mqueue
rm -f *
cd /var/spool/MailScanner/quarantine
rm -rf *
cd ../incoming
rm -rf *
cd ../archive
rm -rf *
cd /var/log
Remove all old logs
service syslog restart
Fix ethernet and IP address in /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network
Fix /etc/hosts
Repair ownership and permissions of /home/* and /usr/local/share/clamav (and subdirectories).
Fix extra ClamAV databases so that “MailScanner --lint” runs correctly.
Reboot with network interface connected.
Re-register with RedHat network for yum updates, get the info from the Systems KB.
yum update