Ray's Mail Filter Version 1.04/1.14
On this page:- Description Platform Copyright and Licence Requirements Rationale Theory Method Rejected Message Files SMTP reply Log files Known Bugs Interaction with Other Software Release Notes	Other pages:- Installation Configuration Operation Utilities

Description

Ray's Mail Filter is a mail message filter for use on mail servers that run Sendmail versions 8.10.1 onwards.

The filter examines messages being processed by Sendmail, and accepts or rejects them on the basis of their header contents. In addition to the main message headers, the filter examines the MIME part headers within a multipart message. It can therefore be used to reject messages containing attachments with particular filenames or filename extensions.

Rejection criteria are controlled by configuration files, and can be changed without having to re-start the filter. Using the configuration files as supplied, the filter will reject any message that has an attachment of a type that is listed as "unsafe" in the Microsoft Outlook E-mail Security Update (Article ID: Q262617).

As a partial defence against malicious exploitation of the buffer overrun problem in certain versions of Microsoft Outlook, the filter will also reject messages whose Date header is more than 60 characters in length.

Rejected messages are saved in message files, annotated to show the reason for rejection. A log is kept of all messages processed by the filter.

Platform

Rays Mail Filter has been run on the following platforms:

H/W O/S Sendmail
version Remarks

Dec Alpha Digital Unix V4.0F 8.10.1
8.11.0.Beta1
8.11.0 Development machine.

i686 Linux (Red Hat 6.1) 8.10.1

i686 Linux (Red Hat 6.2) 8.11.0

Copyright and Licence

The software is distributed under the terms of the GNU General Public Licence.

Requirements

An ANSI-compatible C compiler is required to compile the program.
The filter program has to be linked with the Sendmail Mail Filter API ("Milter"), which is distributed with Sendmail version 8.10.1 onwards.
The Unix Syslog utility is used to keep a log of all messages processed.
The shell scripts provided to start, stop and reset the filter require awk.
The optional utility provided to analyze rejected messages requires perl.

Rationale

Executable entities (Visual Basic scripts, Windows Shell Scrap files, ".EXE" files, etc.) may be included as "attachments" in multipart, MIME format e-mail messages. Some of these attachments may be harmful viruses, such as the recently prevalent "ILOVEYOU" and "Life Stages" viruses. It was considered desirable to filter out and reject all messages which have such attachments, on the grounds that they may be viruses.

Theory

Within a multipart MIME message, the "parts" (main message and "attachments") are delimited by a string of characters known as a boundary. The boundary is defined in the Content-Type header of the main message. The following is an example:

Content-Type: multipart/mixed;
        boundary="----_=_NextPart_000_01BFB5AF.795C3FBA"

Within the body of the message, each occurrence of the boundary is followed by a number of headers relating specifically to that part. An "attachment" part containing a Visual Basic script, for example, can be identified by the presence of a file name with an extension such as ".vbs" in its Content-Type and Content-Disposition headers. For instance, the "ILOVEYOU" virus was contained in an attachment which began with the following boundary and headers:

------_=_NextPart_000_01BFB5AF.795C3FBA
Content-Type: application/octet-stream;
	name="LOVE-LETTER-FOR-YOU.TXT.vbs"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="LOVE-LETTER-FOR-YOU.TXT.vbs"

The Sendmail Mail Filter API allows third-party programs access to mail messages as they are being processed, in order to filter meta-information and content. This allowed a filter to be developed which can scan a message for the presence of characteristic text strings (e.g. 'name="xxxxx.vbs"') in certain specified headers (e.g. "Content-Type"), and reject the message if such a string is found.

As the filter can scan all the message headers, it can also be configured to reject messages where, for example, the Subject header of the main message contains text which suggests the possible presence of a virus.

Method

On startup (or on receipt of a USR1 signal), the program reads a set of header names and suspect text strings from configuration files. Regular expressions, like those used with the Unix 'grep'command, are used to specify both the headers and the suspect strings.

Then, for each message received by Sendmail, the filter performs the following tasks:

Reads each of the message headers. If the header name matches one of those in the configuration file, the header text (after the ':') is examined to see whether it contains any of the suspect text strings. The message is flagged for rejection if a match is found. If a Date header is found, it is checked for length. The message is flagged for rejection if the Date header is more than 60 characters in length. If a Content-Type header is found, the program looks in it for the definition of a boundary.
If a boundary definition is found, the filter program scans the body of the message looking for occurrences of that boundary (this is done even if the message has already been flagged for rejection, so that all the possible causes of rejection are found). Whenever it finds a boundary, the program reads the subsequent MIME part headers, and subjects each one to the same test as the main message headers. Again, the message is flagged for rejection if a match is found.
(If the body of the message is longer than 65,535 characters, Sendmail passes it to the filter in "chunks" of no more than that size. When scanning the message body, the filter "overlaps" adjacent chunks in order to detect boundaries and suspect text strings which would otherwise be split between chunks.)
While a message is being processed it is stored in a "temporary" file. If the message is rejected, this file is renamed and made permanent. The name of this file then incorporates the username part of the sender's e-mail address. The file contains the complete text of the message, annotated to identify each string which caused the message to be rejected and the header in which it occurred.
If a message is rejected, the filter provides Sendmail with an SMTP reply to return to the sender, informing them that the message could not be delivered and giving the reason.
Acceptance or rejection of the message is recorded in the system log.

Rejected Message Files

A rejected message will be saved in a file.

The name of the file will consist of the "username" part of the sender's e-mail address and a "random" string of characters. For example, a rejected message from A.N.Other@some.co.uk may be saved as "A.N.Other.aawfiA".

The file contains the text of the message, annotated as follows:-

A diagnostic note will appear at the end of the message or, in a long message, at the end of each "chunk" of 65,535 characters, e.g.

=====================================================
Note inserted by Ray's Mail Filter
-----------------------------------------------------
This chunk (bodylen):    65535
Overlap from previous:       0
Buffer contains:         65535
Overlap to next chunk:    1024
=====================================================

If a suspect string occurs in one of the main message headers, (or the Date header is found to be suspiciously long), a note is inserted into the file immediately after the offending header, e.g.

Message-ID: <20000626101717.2799.qmail@ab.cdefg.com>
From: lists@wxyz.com
To: A.Student@sbu.ac.uk
Subject: Mail from Jokes Online
-----------------------------------------------------
Ray's Mail Filter found the following pattern:
:_ Jokes
in this header:
Subject: Mail from Jokes Online
-----------------------------------------------------
Reply-to: lists@wxyz.com
X-Mailer: WXYZ-Mailing List Service

If a suspect string occurs in a MIME header, it will be reported in the diagnostic note at the end of the 64kB chunk in which the MIME header occurs, e.g.

=====================================================
Note inserted by Ray's Mail Filter
-----------------------------------------------------
This chunk (bodylen):    65535
Overlap from previous:       0
Buffer contains:         65535
-----------------------------------------------------
Ray's Mail Filter found the following pattern:
:_ name=\".*\.exe\"
in this header:
Content-Type: application/octet-stream;
        name="Beckham.exe"

-----------------------------------------------------
Overlap to next chunk:    1024
=====================================================

Note: The token ":_" is not part of the suspect string. It is included in these notes for the benefit of the Analysis Utility

SMTP reply

The filter program specifies an SMTP reply for Sendmail to return in response to a rejected message. For example:

554 5.7.1 Message was rejected because it contains signs of a possible virus (name=\".*\.vbs\")

This consists of the following components:-

The RFC 821 three-digit reply code 554 (Transaction failed)
The RFC 1893 / RFC 2034 extended reply code 5.7.1:-
- 5 = Permanent failure;
- 7 = Security or policy status;
- 1 = Delivery not authorised, message refused.
A text message which can be specified at compile time (default message shown above).
The suspect string(s) which caused the message to be rejected are appended to the text message as a comma-separated list in brackets.

Log files

The filter program keeps its own log file, mail-filter.log in its working directory, and also makes entries in a system log file.

mail-filter.log records when the filter is started, stopped or signalled to re-read its configuration files, e.g.

21-Jun-2000 11:19:46 : Starting rays-filter; FAIL_COUNT = 0
28-Jun-2000 14:35:17 : Signalling rays-filter to read configuration files
04-Jul-2000 16:31:49 : Stopping rays-filter; WAIT = 6

The filter makes entries in the system log file using the following function calls:

openlog("rays-filter", LOG_PID, LOG_USER);

syslog(LOG_INFO, "text");

The following are typical examples of system log entries:-

Program starting Read 3 header names from /usr/local/src/mail-filters/header_list.conf Read 48 strings from /usr/local/src/mail-filters/string_list.conf Accepted message: From: <a.n.other@sbu.ac.uk> To: <some.body@talk21.com> Subject: medical stories
*Rejected message: From: <some.one@somewhere.ac.uk> To: <butlerra@sbu.ac.uk> Subject: Test

Known Bugs

H/W	O/S	Sendmail version	Observations
Dec Alpha	Digital Unix V4.0F	*	The program will save up to 32 rejected messages from any one user. After this, messages from that user which meet the rejection criteria will continue to be rejected, but will not be saved. Rejected messages from other users will continue to be saved.
Dec Alpha	Digital Unix V4.0F	8.11.0.Beta1	If an "internet" type socket is used, the filter will accept a telnet connection on the specified port from a user on the same host.
i686	Linux (Red Hat 6.1)	8.10.1	If an "file" type socket is used, the filter must be run by root. A non-root user may run the filter if an "internet" type socket is used.

Interaction with Other Software

S/W	Version	Observations
Netscape	4.61 (Linux) 4.73 (Win32)	Displays the RFC 1893/2034 reply code and text part of the SMTP reply returned by the filter.
Pegasus Mail	2.55 (Win32)	Displays the whole of the SMTP reply returned by the filter.
Eudora		Displays the text part of the SMTP reply returned by the filter.
Outlook 97		Displays: "Mail could not be delivered to the Mail Server. Be sure you entered the correct server name, or specify a new server."
Outlook 97	8.04.5619	Returns a message with the subject "Undeliverable: subject", with contents as follows: The following recipients could not be reached: 'user@address' on date No transport provider was available for delivery to this recipient.
Internet Mail Service (Microsoft Exchange)	5.5.2650.21	Does not respond as anticipated to the SMTP reply returned by the filter. Continues to re-send the rejected message at 20-minute intervals until it times out (several days later).

Release Notes

Version	Date	Notes
1.04/1.14	08-Mar-2001	Fixed a bug which would have prevented disk write failures from being recorded by `syslog.` (Thanks to Rich Jones, University of Western Ontario, for reporting this).
1.12	25-Sep-2000	Conditional compilation of 'bool' type definition and TRUE/FALSE values in `rays-filter.c.` Fixes a problem encountered when compiling with gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)
1.01/1.11	08-Aug-2000	Improved scripts for more reliable recovery after crash or re-boot. Any socket file left behind by a previous run, which would prevent the binary from running, will be removed automatically.
1.1	04-Aug-2000	Version for Sendmail version 8.11.0
1.0	02-Aug-2000	Version for Sendmail versions 8.10.* to 8.11.0.Beta*

Installing the Filter
Filter Configuration
Running the Filter
Utilities

butlerra@sbu.ac.uk

08 March 2001