smtp-proxy, test-smtp — Simple Mail Transfer Protocol (SMTP) proxy
smtp-proxy
[-hv
] [-d
] dbglev
-f
cfgfile
test-smtp
[-hv
] [-d
] dbglev
-f
[cfgfile
-r
] [-t
]test-expr
Program smtp-proxy is the proxy daemon for the Simple Mail Transfer Protocol (RFCs 2821, 2822, 2045 etc.). The proxy is not a final delivery server, so it indeed supports the SMTP commands like VRFY and EXPN but responds by the SMTP defined negative response.
The proxy reads its configuration and starts listening on TCP
sockets (address/port couples) specified by listen-on
configuration section, see listen-on(5).
If support of transparent connections (see transparency(7)) is requested by item transparent
in
section listen-on
, the corresponding NAT redirections
are established during proxy startup and removed upon exit.
However, transparent connections are in fact not supported by this
proxy, decisions about servers are made according to
the proxy configuration, not by original destination.
Format of the configuration file is described in smtp-proxy.cfg(5). General syntax of Kernun configuration files is explained in configuration(7).
Program test-smtp tests syntax and partially semantics of configuration; for test expression syntax, see test-expr(5).
The smtp-proxy uses standard Kernun access control (see access-control(7)) with four types of ACLs on three levels:
session-acl
(level 1) is checked
once for each client connection and defines general protocol
behavior, or rejecting the connection. In addition to the
generic ACL conditions and actions,
some smtp-proxy-specific
conditions and parameters can be set (see smtp-proxy(5)).
delivery-acl
(level 2) is checked once
for each mail recipient and defines the response to the
RCPT TO command, i.e. way of delivery,
or rejecting particular addressee.
During this phase, the correctness of all the commands
(HELO, MAIL FROM and
RCPT TO) and its arguments is checked
and proper decision is made.
When resolving domain names, the current
resolver
section setting is applied.
It means that the search
list (if present)
is tried when some resolution fails.
If this search-list contains some domain having
the *.domain
MX record set,
then every resolution will
succeed and errors like unknown-perm
will
never occur. This is probably not what you want.
To solve this problem you can define a new
resolver
section
without the search
item
and then use this resolver
in the particular smtp-proxy only
by the use-resolver
item
in the proxy configuration section.
mail-acl
(level 3M) is checked once for
each mail recipient and it defines rules whether to
reject/accept forwarding the mail to him/her.
doc-acl
(level 3D) is checked once for
each recipient and document (MIME part) and it defines
document processing mode (e.g. filtering, replacing
etc.).
The proxy uses the same approach to the complexity
of ACL set as any other Kernun proxy. However, there is a special
feature of the SMTP protocol that within a single session, data
for many users can be transported. An incomplex ACL set leading to
the ACL search failure will cause (comparably to other proxies)
an abort of the whole session. Thus, it is highly recommended
to add a single denial ACL to the end of every layer ACL set
(i.e. delivery-acl
, mail-acl
and doc-acl
). These “sentinel”
ACLs should cover “the rest of the world”,
i.e. they will typically have no entry conditions and
a proper denial operation included.
For detail description of mail processing, see below the section called “Mail Processing”.
The Kernun smtp-proxy provides some configurable client identity checking to protect against unsolicited mails.
The client address can be deresolved to check whether
it is a known station (see unknown-client
item in session-acl
).
If the client deresolution fails, the session is rejected.
The client address can be cross-resolved to check whether
its name is not faked (see unmatching-client
item in session-acl
).
Every name got by deresolution is
resolved back and the resulting addresses are checked against
the client IP address.
If no one address matches, client session is rejected.
The client address can be queried at any
black-list database that uses the DNS
based querying model (like
http://www.spamhaus.org
or http://www.ordb.org). The list of databases
used can be specified in
session-acl.blacklisted-client
configuration
directives. If the client address is found in any of the
configured databases, client session is rejected.
After receiving the MAIL FROM
command,
the client can be checked according to the Sender
Policy Framework (http://www.openspf.org) of
domain presented
by the client as the mail return path
(see white-listing
item in session-acl
).
The result of this check can be used for matching during
delivery-acl
selection
(see spf
item).
This feature has some weaknesses.
SPF is unusable in case of forwarding mails
from a foreign mailserver outside our domain
to a target mailserver in our domain.
Such cases should be handled specially in your configuration.
In case of server-to-server forwarding
(e.g. from your secondary MX)
you will probably deny white-listing
in particular session-acl
,
in case of individual forwarding
(between users' mailboxes) you can handle it
in per-user delivery-acl
.
The smtp-proxy provides possibility
of the grey-listing method
(http://projects.puremagic.com/greylisting) if
configured by grey-listing
section in the
delivery-acl
. For more details, see the
triplicator(1) tool description.
The Kernun smtp-proxy is a security proxy,
not a regular MTA (Mail Transfer Agent).
Its task is to apply security policy, check incoming mails for
correctness and then use some mail-forwarder
to queue and distribute the mails.
For this purpose, several smtp-forwarder
global
sections can be defined in the configuration.
Every section specifies internet domain
which the forwarder is to be used for and connection parameters
of forwarding channels (e.g. addresses, timeouts etc.).
You can omit the domain
settings
in smtp-forwarder
sections and postpone
forwarders selection to the delivery-acl
sections
using the via
element of the deliver
item. For the original recipients of a mail, this solution is
sufficient. However, if you will use sending a copy of a mail
to another address by the copy-to
item, you have to
use the domain setting because for such recipients there are no other
way how to select proper SMTP forwarder channel.
Forwarders can be instances of common UNIX MTA daemons (e.g.
postfix distributed together with Kernun) running
on the firewall machine, or some external MTAs (e.g. the central site
mailserver for incoming mails and the ISP mailserver for outgoing ones).
In the former case, the configuration of the postfix forwarder can be
entered directly in smtp-proxy configuration
(in the agent
section within
the smtp-forwarder
one)
and the proper configuration files for postfix
are generated automatically.
Kernun smtp-proxy does not provide any queuing service. If the mail is not deliverable to forwarders for all recipients, mail is rejected and resending is in clients' responsibility. If at least one delivery succeeds, mail is accepted and the DSN (Delivery Status Notification) message is constructed and sent to the original sender (if not null).
The proxy supports secure communication via SSL/TLS protocols, see ssl(5).
Item session-acl.client-ssl
switches on SSL/TLS
on the client connection and sets various SSL/TLS parameters.
If the connection from the client uses SSL/TLS, item
client-cert-match
defines
the acceptable client certificates.
If the client certificate does not pass the test, SSL/TLS connection
establishment fails and the connection is closed.
There are three possible modes of SSL/TLS usage:
The SSL/TLS handshake is started immediately after accepting the client connection. This mode is configured by
ssl
or simply
name
connection;ssl
name
;
Such a mode of SSL is typically configured on a special port different from the SMTP port 25 (usually 465).
The session is started in normal way. However, the STARTTLS command is expected to be used by the client. Without using it, only a subset of SMTP commands is accepted (e.g. the MAIL command is not allowed). This mode is configured by
ssl
name
command required;
The session is started in normal way. The STARTTLS command is offered to the client and the decision whether to use SSL/TLS or not is left to it. This mode is configured by
ssl
or simply
name
command allowed;ssl
name
command;
If the client does not issue the
STARTTLS
command, session continues.
If the client uses the command, proxy responds
by the 220
response code
and starts the SSL/TLS handshake.
If it succeeds, the session is reset according to RFC 3207.
If it fails, the connection is closed.
Item smtp-forwarder.server-ssl
switches on SSL/TLS
on the server connection and sets various SSL/TLS parameters.
If the connection to the server uses SSL/TLS, item
server-cert-match
defines the acceptable server
certificates.
If the server certificate does not pass the test, SSL/TLS connection
establishment fails and the request terminates with an error.
There are again the three possible modes of SSL/TLS usage listed above:
The SSL/TLS handshake is started immediately after connecting to the server. This mode is configured by
ssl
or simply
name
connection;ssl
name
;
The session is started in normal way. However, if the STARTTLS command is not offered by the server, or its issuing fails, session is closed. This mode is configured by
ssl
name
command required;
The session is started in normal way. If the STARTTLS command is not offered by the server, session continues without trying to issue it. If the server supports SSL/TLS, the STARTTLS command is issued immediately. This mode is configured by
ssl
or simply
name
command allowed;ssl
name
command;
The proxy uses common Kernun mechanisms for listening on its sockets, accepting client connections, and managing its processes. It can also run in a chrooted environment and change its user identity upon startup. See also application(5), tcpserver(5) and tcpserver(7).
The proxy uses a common Kernun mechanism for network input/output.
The configuration allows to specify several parameters like
buffer sizes and timeouts, both for client and server connections.
The parameters are set in configuration sections
client-conn
and server-conn
.
See netio(7) for details.
The proxy uses common Kernun mechanism for name resolving (see resolving(7)).
The proxy uses common Kernun mechanism for logging (see logging(7)).
For every session,
the SESSION-START (SMTP-801-I), the session-acl
decision (SMTR-801-I) and the SESSION-END
(SMTP-809-I) messages are logged.
For every mail, a couple of MAIL-START (SMTR-811-I) and MAIL-END (SMTR-819-I) messages is logged.
For every mail recipient,
the delivery-acl
decision message
(SMTR-802-I), the mail-acl
decision message
(SMTR-803-I), the doc-acl
decision messages
(one SMTD-803-I for each MIME document) and the RCPT-RESULT
(SMTS-815-I) messages are logged.
The proxy uses common Kernun mechanism for runtime monitoring. For more detailed information, see monitoring(7).
The proxy uses common Kernun mechanism for traffic shaping. For more detailed information, see traffic-shaping(7).
-h
Display usage information and exit.
-v
Print version information and exit.
-d
dbglev
Set debuging level to a specific number. Permitted values are 3 through to 9, 3 being the least and 9 the most verbose. See logging(7) for details. This setting is relevant only till configuration reading is finished.
-f
cfgfile
Read configuration from
cfgfile
.
-r
Resolve names in configuration prior to testing.
-t
test-expr
Test configuration according to given expression.
Format of the test-expr
is
described in test-expr(5).
Following steps are gone through when processing incoming mails:
When a connection from an SMTP client arrives,
the configuration
is searched for a matching session-acl
.
If the ACL says that the connection should be denied or there is
no matching ACL, the proxy closes regularly the session.
According to the session-acl
settings,
the client identity is verified (see the Client verification
section above). If the verification fails,
the proxy closes regularly the session.
Proxy reads the HELO/EHLO command,
checks it according to the RFC 2821,
tries to resolve its argument (A DNS RR) in case of domain name
and stores all results for further processing.
The command is always responded
by 250 OK
reply.
Proxy reads the MAIL command,
checks it according to the RFC 2821,
tries to resolve domain part of sender's email address,
checks client identity according to the SPF (if configured,
see above the section called “Client verification”)
and stores all results for further processing.
The command is always responded
by 250 OK
reply.
The domain name resolution proceeds as follows:
The proxy tries to find MX DNS RRs for the domain name.
If it succeedes, each mail exchanger is resolved again
until at least one A DNS RR is found. If no MX DNS RR exists,
the proxy tries to get an A DNS RR for the domain name.
If no A DNS RR is found by either way, the sender name
is treated as unknown
for further processing
(in delivery-acl.sender
items).
The whole resolution process has a single total timeout
equal to the conn-timeout
value in
the sense of resolver(5) settings.
For every mail, a unique internal message identification is generated. This MSGID is then used for identifying log messages, mail documents etc.
Proxy repeatedly reads RCPT commands,
checks them according to the RFC 2821,
tries to resolve domain part of recipient's email address
(in the same manner as the sender's one)
and then finds the appropriate delivery-acl
taking paricular recipient and also HELO/MAIL check results
into account.
According to the delivery-acl
settings,
proxy responds to the command.
If no delivery-acl
is found,
the session is aborted and the mail is not delivered to any
addressee. The ACL set should be closed by an ACL with
no entry conditions and a proper denial operation.
There are several ways how to deny
service for particular recipient:
abort
The RCPT command is responded
by given reply code and text and then the session
is ended with 421
response code.
reject
The RCPT command is responded
by given reply,
so as all following ones till the DATA
command (if issued by client).
refuse
The RCPT command is responded by given reply and processing of the mail regularly continues.
discard
The RCPT command is responded
by 250 OK
,
but mail will not be sent to the recipient.
This action has no impact to other recipients.
If the recipient is accept
ed,
the default behavior is to deliver
the mail to him/her via proper smtp-forwarder
.
The list of forwarders is searched for the first one
who serves domain of particular recipient.
The item deliver
can be used to choose another
forwarder (via
elem) or even another recipient
(to
elem).
If the response code was 250
(accept
or
deny
+discard
used),
a copy of this mail can be sent to another person.
This person's email address
is given as a parameter of the copy-to
item.
This address is added to the set of recipients
for further processing (searching for proper ACLs of phase 3)
with the same delivery-acl
as the originator
has (no new search for it is done).
The smtp-forwarder
for the delivery
will be determined by searching through forwarders'
domain
s regardless of the deliver
item of the original delivery-acl
.
Under the same condition,
the mail can be stored into the quarantine
by the quarantine
item.
For easy controlling the mails in the quarantine,
use the quarc.sh(1) tool.
When the DATA command comes,
proxy checkes the number of accepted recipients and, if not zero,
responds by 354
response code
and starts receiving the mail.
Otherwise, the proxy responds by
554
response code
and waits for the next MAIL command.
The mail is read into the mail-pool
directory under a name derived from the MSGID.
It is stored as-is, without any changes, checking only
the RFC 2821 rules (line length, leading dot dubbing etc.).
By default, the proxy checks every line, whether its
normalized length
(line length not counting the leading dot duplicated
for transparency and the CR+LF line separators)
fits the 998 chars limitation.
This check can be changed or switched off by
the session-acl.mail-line-len
configuration item.
Even in this case, mail line length (unnormalized) cannot be
greater than the client-conn.recv-bufsize
and
the server-conn.send-bufsize
value.
The mail size-limit
is also checked
during mail reading.
Every line is counted in its normalized length plus two bytes
(for the standard CR+LF line separator).
This normalized size is then used in “statistical”
log messages (SMTS-815-I and SMTR-819-I), too.
The mail MIME structure is decoded according to the RFC 2822, 2045 and others. All MIME documents are stored into the mail-pool and several checks are done. For detail configuration description, see mod-mail-doc(5).
The smtp-proxy can check mail documents by
an antivirus.
Antivirus checking is defined by item use-antivirus
which selects a global antivirus
section.
See antivirus(5) for details about configuration
of virus checking.
The smtp-proxy can check the whole mail by
an antispam.
In the current version of Kernun firewall,
SpamAssassin is supported.
Antispam checking is defined by item use-antispam
which selects a global antispam
section.
See mod-antispam(5) for details about configuration
of spam checking.
The smtp-proxy can check particular MIME types of mail body documents according to the rules defined in configuration, see doctype-identification(7) for details.
The smtp-proxy checks correctness of the mail headers and the MIME structure. Mails not conforming the RFCs are rejected. However, many clients do not respect RFC and if the security policy allows sending such mails you can enforce proxy to correct or even to pass them.
Read carefully the mail-filter
section description in the mod-mail-doc(5) and set only really necessary
exceptions to the RFC and preferably only in direction inside
your organisation where you can check, in some extent,
ability of mailservers to accept incorrect mails.
If a document header folded line is longer than the internal
buffer size, the header is not processed and the mail is rejected.
The buffer size is 16kB, by default, and it can be increased by
the client-conn.recv-bufsize
and
server-conn.send-bufsize
configuration directives.
If smtp-proxy forwards delivery error
report, it checks (and corrects, if required) also the reported
mail (if included). If the correction fails, it rejects whole
mail. You can switch off this behavior by setting the
treat-rfc822-as-text
configuration directive that
tells proxy to read included mails as regular texts.
After receiving the whole mail data, proxy finds
the appropriate mail-acl
taking all checks
from previous steps into account.
After that, for each MIME document, the appropriate
doc-acl
is searched for.
These steps are repeated for every recipient
(including the ones added by copy-to
items),
because mail processing can vary for different addressee
(level 3 ACLs can use also sender's and recipient's
email addresses as entry condition).
In the case of changing the final recipient
(by deliver
+to
item),
the original recipient's address is used for this search.
If no mail-acl
or
no doc-acl
is found,
the session is aborted and the mail is not delivered to any
addressee. The ACL sets should be closed by two ACLs with
no entry conditions and a proper denial operation.
In phase 3 ACL, following actions are available to
deny
the service for particular recipient:
abort
The mail final CRLF.CRLF
is
responded by given reply
and then the session is ended
with the 421
response
code.
reject
The mail final CRLF.CRLF
is
responded by given reply, the session continues.
cancel
The mail will not be sent to this recipient, given reply is stored as the mail forwarding result for this recipient and processing of the mail regularly continues.
discard
The mail will not be sent to this recipient, but no failure is registered for this recipient so the client will assume successfull delivery.
Among all denial actions for one recipient ordered
by all mail-acl
and delivery-acl
sections, the one with the highest severity
is chosen to use.
If the mail/document is accept
ed,
there are several modifications that can be done:
HTML filtering, document replacement, header modifications etc.,
see mod-mail-doc(5) for more details.
If the mail is not aborted, it can be stored into
the quarantine by the quarantine
item.
If you configure mail denial and some further
processing (storing to quarantine, sending a copy etc.),
be careful because without special handling, a repeated mail
processing (when resent from quarantine or when sending to
a copy-to
address) will fall to the same
level 3 ACL set. That's why you have to handle these cases
by individual mail-acl
s
and/or doc-acl
s prepended
before the regular ACLs to avoid the denial.
If the mail is accepted to delivery, all forwarder channels are opened and the mail is forwarded to all recipients. While sending copies, the recipients are grouped together according to the set of phase 3 ACL used.
For every such a group, the mail is reassembled from parts (excluding the case of signed documents) using the filtering operations specified in level 3 ACLs (see mod-mail-doc(5)).
Moreover, some header modifications are done in following steps:
The Return-Path
header (if any)
is removed.
If the stamp-filter
directive
is used, all Received
headers are removed and replaced by
the X-Kernun-Loop-Info
header to preserve
the function of "Too many hops" checking (controlled by
the stamp-limit
directive).
If the antivirus check was completed, the result
of it is reported by adding/replacing
the X-Kernun-Virus-Status
header.
Notice that replacement of
the X-Kernun-Spam-*
headers
(antispam check result)
is done already in the phase of mail reading.
The Content-Type
header is
changed according to the value recognized by
the doctype-identification tool
(see doctype-identification(7)), if requested by the configuration
directive force-doctype-ident
.
The header modifications ordered by the configuration
directives modify-header
are realized.
No check for correctness of changes requested is done. The resulting state is in administrator's responsibility.
The header modifications required by the RFCs for given types, character sets etc. are realized.
The Subject
header is modified
according to the configuration
directive prefix-subject
.
During mail reassembly, the RFC 2821 and 2045 rules
are respected.
Every line longer than 998 bytes (in the sense of normalized length
— see above) is splitted into parts (converting the subdocument
into MIME quoted-printable transfer encoding, if necessary).
Subdocuments being encoded by some MIME encoding
(quoted-printable
or
base64
)
are re-encoded using 76 chars long lines.
If smtp-proxy forwards signed document, it must not alter its content. That's why the standard way of processing mails (decomposition and reconstruction) cannot be used. Thus, the proxy uses original mail image saved on the beginning of processing, except two cases:
Some changes were forced by the configuration (e.g. filtering or replacing some documents).
The mail violates RFC in such
extent that cannot be ignored by itself
(e.g. wrong line length, incorrect header format etc.) and
passing of such mails is not explicitly allowed
by the administrator (see the mail-filter
description in mod-mail-doc(5)).
In these cases the proxy does reconstruct mail (because site security has priority) and signature becomes not valid.
In the case of transient failure during forwarding, all servers available for particular forwarder channel are tried. In the case of permanent failure, or when all servers have failed, sending the copy is given up. If any forwarding server does not support 8bit transfer, the mail is converted to 7bit stream. If this is not possible, mail forwarding is rejected.
The result of forwarding is stored for every recipient
and in the case of failure of mail coming from a non-null sender,
the DSN (Delivery Status Notification) message is constructed
and sent to the original sender.
This behavior and the content of the message can be partially
controlled by the dsn-mail-copy
and
omit-dsn
configuration items.
In the case of forwarding mail to a forwarder without
ESMTP DSN extension, the proxy sends also DSN RELAYED messages
if requested by the clients.
If the mail was ordered to be stored into the quarantine, the image of it, stored at the beginning of processing, is moved or copied to the quarantine directory. Together with this copy, the quarantine control file is stored. It contains all available information about sender, recipient, MIME structure, viruses etc. and also reasons for storing the mail into the quarantine.
The final response code is sent to the client and the proxy is waiting for a new mail, or the QUIT command.
The Kernun smtp-proxy is a security proxy, not a regular MTA (Mail Transfer Agent), it does not provide for queuing facility, so it requires some regular MTA to be used as mail-forwarder.
The proxy does not support some obsoleted features (e.g. source routes).
The proxy does not support caching of SSL/TLS sessions.
If an excerpt of original mail is sent as part of the DSN
(Delivery Status Notification) message, the excerpt is sent as-is
(e.g. with only LF
as end-of-line separator).
The statistical messages then contain “real”
sizes, not the normalized ones.
quarc.sh(1), triplicator(1), antivirus(5), application(5), listen-on(5), mod-antispam(5), mod-html-filter(5), mod-mail-doc(5), smtp-proxy(5), smtp-proxy.cfg(5), ssl(5), tcpserver(5), test-expr(5), SMTR-801(6), SMTR-811(6), SMTR-802(6), SMTR-803(6), SMTD-803(6), SMTS-815(6), SMTR-819(6), SMTP-809(6), SMTS-815(6), SMTR-819(6), SMTP-801(6), access-control(7), configuration(7), doctype-identification(7), host-matching(7), logging(7), monitoring(7), netio(7), resolving(7), tcpserver(7), time-matching(7), traffic-shaping(7), transparency(7)