Mail Services

anchored to [[143.00_anchor]]

denotes the fourth lab-day with information / practical examples for working / managing mail-servers.


Overview

establishing communication over several networks can be accomplished by using different systems / protocols like Matrix, XMPP and further yet one of the most fundamental and important communication structures involves e-mails.

Communication via e-mails requires the use of DNS systems because we ought to be able to denote/point toward a server to handle the mails for the given domain. This is done by MX-Records that point toward the server being responsible for handling mails in said domain.

( denote 143.09_DNS for further information)


To understand their structure / process of exchanging / communicating we ought to grasp the hierarchy / actors involved in emails.

Considering a simple communication between two clients we traverse several stages / servers which actors do we have with emails? #card As seen above we have two actors per side:

  • MUA - Mail user agents, which are the interface used by the client to write,edit,read mails
    • they query changes / new messages from the mail-server via IMAP / POP3 -> stateful/stateless
    • also pushing messages written to the Mail-server which will process / enqueue and send them
  • MTA - Mail transfer agent, that acts as instance to store/send/forward messages both incoming / outgoing
    • they store messages per user -> which can then Query with a MUA
    • they queue messages going outwards -> in case delivery failures occur
    • with DNS they are querying on how / where to send messages to

Further we introduces some terminology like:

MDA - Mail Delivery Agents _whats their purpose? #card

  • those are used as mail filter
  • examples included procmail,sieve,maildrop

MX - Mail Exchanger whats their purpose? #card

  • they accept messages for local recipients
  • hence we have Mx-RR in DNS!

For some additional information on mail transfer agents see this entry.

Open Relay Servers: whats their objective #card

  • they act as relays that accept messages for non-local recipients from some unauthenticated senders --> further forwarding these messages to the a given destination
  • because they accept and forward from unauthenticated senders they acted as great transport mechanism of spam

As an evolution of Open relays may server Smart Hosts whats their purpose? #card

  • those also denote relays that accept and forward messages from certain clients --> however it requires authentication or filters based on specific address-ranges --> preventing spam from random actors.

Interworking of MX ( DNS ) and Mail

As observed in 143.09_DNS we have a RR denoting the address of a given Mail-server for a domain. With this knowledge we can combine and describe the exchange of mails across networks with the usage of domain-identifiers instead of ip-addresses: how would we send a given mail to some service ? #card The following figure may show the possible path to take and send a message:

Structure of Mails

Its obvious yet its important to denote and mention the structure of an email-address:

local.part@domain-space.part

we split into two parts:

  • a local part ( everything before "@")
  • a domain part ( everything after the "@")

[!Definition] parts of internet e-mails we have three, which? #card there are three parts that are contained with every e-mail address. Envelope ( defined by SMTP and solely used by it to process mails). -> contain the recipients and senders address. -> only used during mail transfer

Header ( actual data belonging to the mail itself) -> contains several information for the mail ( metadata, recipient and such)

Body ( contains the actual information that are sent) -> with MIME we can further add data of all types, as well as encryption as denoted here :Securing e-mails

Further information may be found here

The header is subdivided into fields containing key-value pairs.

There are many different ones listed here however below are the most important ones:

  • The date representing the point of time when the e-mail was sent, in the “Date:"-field
  • The sender who claims to be the author of the e-mail, in the “From:"-field
  • The destination to which the e-mail is send to, in the “To:"-field
  • The subject addressed in the message, in the “Subject:"-field

A possible e-mail header could be:

Date: Wed, 16 Jul 2014 10:43:42 +0200 (CEST)
From: Bob Foo
To: Alice Bar
Subject: Measures against Spam

We ought to define and observe the structure of the mail-format. As per RFC 5322 we have the following requirements: what are requirements for the mail header? #card

  • Header fields are key:value pairs
  • a key starts at the beginning of a line and ends with a colon and space
  • the rest of the line - after the key - is defined as the linked value
  • line starting with white space are continuations of the previous header field --> in case we would like to transfer multi-line information
  • headers may included the following:
    • FROM:
    • TO:
    • SUBJECT:
    • DATE:
    • Cc:

And further the body of an email requires a structure too: what is it encoded in, extending for images and more? #card

  • the messages are encoded in ASCII --> american system was used as standard thus using ASCII and nothing better like UTF-8 or such
  • MIME - Multipurpose Internet Mail Extensions allows for different encoding, images or other formats to be appended
    • also supports encryption
  • Mails are not encrypted by default!

SMTP - Simple Mail Transfer Protocol

To send messages across MTAs we required some sort of standard denoting packet structures. This is defined by SMTP in RFC 5321 what protocol does it utilize? phases of sending messages? commands? #card

  • Its based on TCP -> requires error correction and connections between host<->recipient (port 25)
  • requires 3 phases of dialogue
    • handshake - opening connection
    • transfer of message
    • closing connection
  • During connection any command is sent as ASCII
  • with every request comes a response with statuscodes and additional information

Mail envelopes

there may exists a difference in the addressed destination and the actual destination of a given mail. We ought to accomplish this idea with SMTP as well how? what are examples denoting these two different values? #card in the context of SMTP we are also calling addresses "envelopes" usually the envelope-sender matches the From: header and envelope-receiver matches the "To:"-header attached in a mail. however those can also differ:

  • mail redirections or simple mailing lists have a "To:" header that will not change while the envelope-receiver will with every message
  • blind carbon copies - Bcc - also don't include the envelope-recipient in the "To:"-header.

Dialog of SMTP

Consider that we would like to exchange messages between two mail server: We ought to establish a certain standard to make communication universal for all potential servers. For that we can observe specific dialogue-options: we describe 6 different ones, what's their purpose / what are they named? #card

  • HELO / ( EHLO for ESMTP)
    • greets client, takes FQDN of the client as argument
  • MAIL FROM: -> denotes envelope sender
  • RCPT TO: -> envelope recipient
  • DATA -> denotes the message as payload
  • "." -> denotes end of message payload
  • QUIT -> terminates SMTP session

Besides these message we always gather a response - as mentioned before - which denotes a status code define the 3 categories of them #card They are pretty similar to HTTP-Statuscodes! with 2xx: we denote success 4xx: notifies about temporary errors -> Quote exceeded, mail in queue or similar 5xx: notifies about permanent errors --> denial by policy, unkown users or similar.

example interaction might look like the following:

S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM: <alice@envelope.crepes.fr>
S: 250 alice@crepes.fr... Sender ok
C: RCPT TO: <bob@hamburger.edu>
S: 250 bob@hamburger.edu ... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: From: Alice <alice@header.crepes.fr>
C: Subject: Fast Food
C:
C: Do you like ketchup?
C: . (a single dot indicates the end of the message)
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection

Extensions of SMTP

Security Extensions

Authentication with SASL: as defined in RFC4954 we can add means to authenticate users by utilizing / adding SASL - Simple Authentication and Security Layer - the whole process of mails.

source for explanation of sasl ( iana )

[!Quote] Description of SASL by iana

The Simple Authentication and Security Layer (SASL) [RFC4422] is a method for adding authentication support to connection-based protocols.

To use this specification, a protocol includes a command for identifying and authenticating a user to a server and for optionally negotiating a security layer for subsequent protocol interactions.
The command has a required argument identifying a SASL mechanism.

MTA-STS | Strict Transport Security

With MTA-STS we can easily signal whether a given provider is able to use TLS for SMTP or not by adding a separate entry into the DNS-Records. They can then be queried upon connection and thus enable / disble TLS and what to do if its not possible with host.

source for this extensions found here: RFC8461

DSN - Delivery Status Notifications

We ought to signal the status of a request / process somehow to enable us to act accordingly to resolve potential problems or being notified about them at al.. For that we use DSN and introduce the idea of Bounce whats meant with them? What are they signaling? #card Bounces may also be known as mailer daemon messages. They provide information like: non-delivery notifications, delayed delivery notifications. -> their defined destination is denoted by the envelope-sender for example if a message by an MTA cannot be delivered - after being queued

[!Tip] Interaction with DSN service Because a DSN is a single-sided status update they don't accept / expect a return --> .leaving the return-path empty so "MAIL FROM: " is left empty!

Now due to this construction we may encounter late bounces as potential problem: whats meant / defined by late bounce? #card Consider that a MTA notices that a message cannot be delivered after it accepted it - enqueued to send. -> this might be the case if a redirection or user defined filters are used Now there is no guarantee that the return-path is correct -> here the delivery status notification may be sent to the wrong sender --> as the information in the envelope sender is not sepciic enough / wrong. This is called BACKSCATTER

[!Tip] Filtering before submitting to resolve late bounces we ought to run the filters during the SMTP-dialogue to prevent the message from being submitted - and having the error occur afterwards!

Spam Prevention:

SFP - Sender Policy Framework

taken from here follows a short introduction / concept of SFP: Briefly, the design intent of the SPF resource record (RR) is to allow a receiving MTA (Message Transfer Agent) to interrogate the Name Server (DNS) of the domain which appears in the email (the sender) and determine if the originating IP of the mail (the source) is authorized to send mail for the sender's domain. The mail sender is required to publish an SPF TXT RR (documented here) in the DNS zone file for their domain but this is transparent to the sending MTA. That is, the sending MTA does not use the sending domain's SPF RR(s) but the receiving domain's MTA will interrogate and use the sending domain's SPF RR(s).

The SPF information must be defined using a standard TXT resource record (RR).

SPF - Sender Policy Framework what is it build upon? #card The idea is to set-up / use a reverse mx-record.

  • within those we are storing the sender-policies to either reject / accept certain ranges or similar.
  • those policies are stored in a TXT RR Possible examples may include:
example.org IN TXT "v=spf1 10.0.2.0/24 ~all”
example.com IN TXT “v=spf1 a mx –all”

What can we gain from this approach? #card With these policies being stored as DNS-RR we can prevent unauthorized servers from abusing a domain as sender address

  • reducing spam load for all services that accept the policy
  • reducing the amount of backscatter load for those who are publishing the policy --> DSN that are directed to the domain and do not qualify to pass the policy are dropped!
  • However this could also prevent legitimate mail forwarding/redirections where the servers are unaware of SFP - they don't read it and are rejected or such

Greylisting

Another approach to reduce the amount of spam for mail-servers is introduced by utilizing "Greylisting" idea of this approach, required information to store? #card Usually spam-sending services are utilizing the idea of "fire-and-forget" ( shotgun) approaches instead of some MTA with queues and all --> too expensive / much effort Hence one could easily reject messages after their first delivery attempt --> i.e signaling 4xx error -> temporary!

  • a combination of the envelope sender, recipient address, ip-address are then stored in a database together with a timestamp (triplet)
  • if the message is sent again --> matching the entries in the database, we may allow it and add them to a whitelist
    • only after the timer expired ( that was set internally)
  • --> this works because authentic mail servers resend a temporarily failed message delivery after a given time ( ~ 5 - 10 minutes) which the receiving server tracks and then grants to be authentic in the end

Issues with greylisting? #card

  • because we disallow once and allow afterwards we essential slow down mail delivery
  • if the sender is aware of this idea they can send twice too - lol.
  • In case an open-relay / mail server or freemail provider are abused / hacked they will work with real MTAs and thus bypass this prevention.