The Big Picture

If you maintain a mail server you want to keep it out of any blacklists and save the users the efforts to check their spam folders. Having a blacklisted server means that the mails sent from it will have high spam probability and will end in the spam folders of the recipients. Checking the spam folder for incorrectly evaluated as spam mails is time consuming, is done seldom and can lead to delayed reading of an important email, or even overseeing it.

Even today an old and proven technology like email continues to evolve. Emails are bounced, instead of SMTP rejected, the bounces terminate in spamtraps/honeypots, and the bouncing server is blacklisted/marked as spammer. In turn, all consequent emails coming from that server have increased spam probability and terminate in the spam folders of the recipients.

To save the users' efforts to clean the spam folder, some providers discard emails above certain spam probability, and neither the recipient nor the sender is notified. However spam filters are not perfect, and a ham/non-spam email can be evaluated as spam and then send to Nirvana.

To approach both problems -- avoiding flooded spam folders and unwanted bounces, we in AEGEE stopped accepting emails, that will end in the spam folders of the members, or would be later bounced. Instead, we reject the mails during the SMTP dialog and tell the sender by this way what is wrong with her message.

As intermediate tool we have chosen Sieve. It is an easy to learn language for mail filtering. Our users can express in Sieve which emails they do not want to receive, based on spam probability, the sender or the size of the message. When a mail matches the users' preferences for unwanted message, the sender is informed that her email will not be delivered -- a much better approach than discarding emails. In the case when a ham is evaluated as spam, this approach is also better -- most users check irregularly their spam folders and reading ham in this folders is not done immediately. But with the system in AEGEE, there is immediate feedback to the sender that she shall consider contacting the recipient by alternative means. Going further, a user can specify that the sender of all mails above certain (high) spam probability shall be notified, and emails, where it cannot be determined precise if they are spam or ham, can be stored in the spam folder. The users proceed this folder then with a different attention, compared to the usual Inbox.

Sieve has some good "side-effects" -- with it our users can set up out-of-office/vacation messages, and depending e.g. on the domain of the sender, send different response.

Sieve is good also for mailing lists. From the settings of each mailing list we project a Sieve script, that SMTP rejects the emails, if and only if the mailing list software would bounce that mail. In this way we do not bounce the emails sent to mailing lists, we reject them, so we are not blacklisted.

That's all. It was a lot of work to implement these ideas. Here we share our code with you.

• aegee-milter (not published yet): a modular system for proceeding the emails during the SMTP dialog. Its most useful module is to evaluate Sieve scripts. aegee-milter is intended for MTAs, implements the milter inteface and can be used with sendmail or postfix. Extending it to use pmilter for Meta1 is not difficult.
• liblistserv: library for TCP communucation with listserv. Provides C-interface for most Listserv commands and generates Sieve scripts for the mailing lists.
• mailing-lists-sieve: Communicates with mailing lists managers and ManageSieve servers. For each Listserv list, this programme uploads a Sieve script and keeps it up-to-date, when the list state changes.
• listserv-utilities (not published yet): different utilities for Listserv (make sense if you use listserv for quite some time):
  • listserv-monitor: checks the lists for different criteria (unused in the last year, only one listowner, never actively used) and asks the listowners to take actions (can the list be deleted, add a second listowner, do they need help to start using the list)
  • generate-listowners-l: subscribes all listowners to the mailing lists LISTOWNERS-L and removes from there all email addresses, that are not listowners anymore
  • post-listserv-milter: modifies the emails leaving Listserv. It replaces X-To, X-Cc headers with To: and CC; (so that option SUBJECT preserves the origianl recipients) and when Reply-To = Both removes the single email address, if it is already on the list and would anyway get the reply.

Keep in mind...

While we do our best to deliver high-quality software, we realize that there are places in the inital release, that need further polishing. Do not hesitate to ask, if something is unclear, we will be glad to update the package, so that things get better explanation for everybody. Of course, external reviews are also necessary to prove or disprove what we consider as right.

Download