Ray's Mail Filter - Utilities

Version 1.04/1.14
On this page:-
Archiving Utility
Analysis Utility
Memory Monitoring Utility 		Other pages:-
Home Page
Installation
Configuration
Operation

The utilities described here are not essential for running the filter, but may be found useful.


Archiving Utility

Command /usr/local/bin/mf-archive
Description A simple utility for organising the storage and disposal of rejected messages.
Requirements Rejected messages to be saved in WORK_DIR/rejects
Input /usr/local/etc/mail-filter/mail-filter.conf
WORK_DIR/rejects/*
Output WORK_DIR/mf-archive.log
WORK_DIR/rejects/Arc*
Method On its first invocation, the utility creates a set of subdirectories within WORK_DIR/rejects as follows: 
                         WORK_DIR
                            |
                         rejects
                            |   
      +---------+---------+-------- - - - --------+
      |         |         |                       |
Arc-yesterday   |         |                       |
          Arc-2-days-ago  |                       |
                    Arc-3-days-ago                |
                                   .......        |
                                            Arc-7-days-ago
Then, and on every subsequent invocation, it -
Deletes any files in Arc-7-days-ago;
Moves any files in Arc-6-days-ago into Arc-7-days-ago;
Moves any files in Arc-5-days-ago into Arc-6-days-ago;
etc...
Moves any files in Arc-yesterday into Arc-2-days-ago;
Moves any non-zero-size files1 in rejects into Arc-yesterday.
Recommended Use Run once every 24 hours (e.g. as a "cron" job), at a time when the volume of mail traffic is likely to be low.
Notes
  1. Files representing messages in progress are likely to have zero size, unless the message length exceeds 64kB. In the latter case, it is possible that a file could be moved while it is being written to. If this results in a disk write failure, it will be noted in the system log file, unless the message is rejected, in which case the usual "Rejected message" record will appear.

  2. The maximum size of the log file mf-archive.log is regulated by the MAX_LOG variable in the script (500 lines as supplied).


Analysis Utility

Command /usr/local/bin/mf-analyze
Description Analyzes the previous day's rejected messages and produces a report showing the number of times each suspect text string1 caused a message to be rejected.
Requirements The utility requires:
  • The previous day's rejected messages to be saved in the directory rejects/Arc-yesterday (as done by the Archiving Utility).

  • An analysis directory in which to save its reports.
Input /usr/local/etc/mail-filter/mail-filter.conf
WORK_DIR/rejects/Arc-yesterday/*
Output WORK_DIR/analysis/analysis.date
Method The filter program inserts annotations into the rejected messages, showing which suspect string pattern(s) have caused them to be rejected. This utility -
  • Reads the list of suspect text patterns from string-list.conf.

  • Scans the saved messages, counting the number of times each pattern occurs, preceded by the token ":_", at the beginning of a line (i.e. in a note inserted by the filter program2).

  • Saves a report in a file such as analysis.20-Jul-2000
Recommended Use Run once every 24 hours (e.g. as a "cron" job), after running the Archiving Utility.
Notes
  1. This utility will not count messages that have been rejected because of a suspiciously long Date header.

    A suspect string matching a pattern such as name=\".*\.vbs\" will often occur twice in a message, once in a Content-Type header and once in a Content- Disposition header. The utility will count both of these. A note warning of this is included in the report.

  2. It is of course possible, though unlikely, that the ":_" token, followed by text matching a "literal" pattern like "Jokes", could occur by chance at the beginning of a line in a message which has been rejected for some other reason. The utility would count this and give a slightly misleading report.



Memory Monitoring Utility

Command /usr/local/bin/mf-memory
Description Records the filter's memory usage.
Requirements None.
Input /usr/local/etc/mail-filter/mail-filter.conf
Output WORK_DIR/mf-memory.log
Method Runs the Unix "ps" command and uses awk to extract the memory usage field. Appends the date and time and writes the data into a log file.
Recommended Use Run as a background process as required.
Notes The maximum size of the log file mf-memory.log is regulated by the MAX_LOG variable in the script (500 lines as supplied).


Ray's Mail Filter Home Page
Installing the Filter
Configuring the Filter
Running the Filter

butlerra@sbu.ac.uk
08 March 2001