2
0
mirror of https://github.com/vdukhovni/postfix synced 2025-08-22 18:07:41 +00:00

postfix-2.8-20101217

This commit is contained in:
Wietse Venema 2010-12-17 00:00:00 -05:00 committed by Viktor Dukhovni
parent 3828e1ddd9
commit 5d604c14b4
13 changed files with 157 additions and 224 deletions

View File

@ -16250,3 +16250,16 @@ Apologies for any names omitted.
handling of out-of-range numbers. Files: global/cfg_parser.c,
global/conv_time.c, global/mail_conf_int.c,
global/mail_conf_long.c, global/mail_conf_nint.c.
20101217
Cleanup: eliminated the code that copied TLS protocol
messages between the OpenSSL TLS engine and the network.
This change hopefully simplifies the TLS library enough
that it can be used in an event-driven TLS proxy in front
of postscreen. Files: tls/tls_bio.c, tls/tls_server.c,
tls/tls_client.c.
This change eliminates an obscure bug where the SMTP server
would wait for another $smtpd_timeout seconds after sending
the "421 Error: timeout exceeded" message to the client.

View File

@ -9,10 +9,28 @@ connections in parallel. While a single postscreen(8) process keeps zombies
away from Postfix SMTP server processes, more Postfix SMTP server processes
remain available for legitimate clients.
By doing these checks in a single postscreen(8) process, Postfix can avoid
wasting one SMTP server process per zombie. A side benefit of postscreen(8)'s
DNSBL lookups is that DNS records will already be cached before the Postfix
SMTP server looks them up later.
postscreen(8) is the first layer in a multi-layer defense.
* The postscreen(8) layer blocks connections from zombies and other spambots
that are responsible for about 90% of all spam. It is implemented as a
single process to make this defense as cheap as possible.
* The second layer implements more complex SMTP-level access checks that are
available with Postfix SMTP servers, policy daemons, and Milter
applications.
* The third layer performs light-weight content inspection with the Postfix
built-in header_checks and body_checks. This can block unacceptable
attachments such as executable programs, and worms or viruses with easy-to-
recognize signatures.
* The fourth layer provides heavy-weight content inspection with external
content filters. Typical examples are Amavisd-new, SpamAssassin, and Milter
applications.
Each layer reduces the spam volume. The general strategy is to eliminate spam
early with the less expensive defenses and to use the more expensive defenses
for the spam that remains.
Topics in this document:
@ -39,9 +57,7 @@ The main challenge for postscreen(8) is to make an is-it-a-zombie decision
based on a single measurement. This is necessary because many zombies avoid
spamming the same site repeatedly, in an attempt to fly under the radar. Once
postscreen(8) decides that a client is not-a-zombie, it whitelists the client
temporarily to avoid further delays for legitimate mail. Clients that pass
postscreen(8) are still subject to the checks that are built into Postfix smtpd
(8), Postfix built-in content filters, and external content filters.
temporarily to avoid further delays for legitimate mail.
Zombies have challenges too: they have only a limited amount of time to deliver
spam before their IP address becomes blacklisted. To speed up spam deliveries,

View File

@ -23,10 +23,33 @@ process keeps zombies away from Postfix SMTP server processes,
more Postfix SMTP server processes remain available for legitimate
clients. </p>
<p> By doing these checks in a single <a href="postscreen.8.html">postscreen(8)</a> process, Postfix
can avoid wasting one SMTP server process per zombie. A side benefit
of <a href="postscreen.8.html">postscreen(8)</a>'s DNSBL lookups is that DNS records will already be
cached before the Postfix SMTP server looks them up later. </p>
<p> <a href="postscreen.8.html">postscreen(8)</a> is the first layer in a multi-layer defense. <p>
<ul>
<li> <p> The <a href="postscreen.8.html">postscreen(8)</a> layer blocks connections from zombies
and other spambots that are responsible for about 90% of all spam.
It is implemented as a single process to make this defense as cheap
as possible. </p>
<li> <p> The second layer implements more complex SMTP-level access
checks that are available with Postfix SMTP servers, policy daemons,
and Milter applications. </p>
<li> <p> The third layer performs light-weight content inspection
with the Postfix built-in <a href="postconf.5.html#header_checks">header_checks</a> and <a href="postconf.5.html#body_checks">body_checks</a>. This can
block unacceptable attachments such as executable programs, and
worms or viruses with easy-to-recognize signatures. </p>
<li> <p> The fourth layer provides heavy-weight content inspection
with external content filters. Typical examples are Amavisd-new,
SpamAssassin, and Milter applications. </p>
</ul>
<p> Each layer reduces the spam volume. The general strategy is to
eliminate spam early with the less expensive defenses and to use
the more expensive defenses for the spam that remains. </p>
<p> Topics in this document: </p>
@ -67,10 +90,7 @@ decision based on a single measurement. This is necessary because
many zombies avoid spamming the same site repeatedly, in an attempt
to fly under the radar. Once <a href="postscreen.8.html">postscreen(8)</a> decides that a client
is not-a-zombie, it whitelists the client temporarily to avoid
further delays for legitimate mail. Clients that pass <a href="postscreen.8.html">postscreen(8)</a>
are still subject to the checks that are built into Postfix <a href="smtpd.8.html">smtpd(8)</a>,
Postfix built-in content filters, and external content filters.
</p>
further delays for legitimate mail. </p>
<p> Zombies have challenges too: they have only a limited amount
of time to deliver spam before their IP address becomes blacklisted.

View File

@ -23,10 +23,33 @@ process keeps zombies away from Postfix SMTP server processes,
more Postfix SMTP server processes remain available for legitimate
clients. </p>
<p> By doing these checks in a single postscreen(8) process, Postfix
can avoid wasting one SMTP server process per zombie. A side benefit
of postscreen(8)'s DNSBL lookups is that DNS records will already be
cached before the Postfix SMTP server looks them up later. </p>
<p> postscreen(8) is the first layer in a multi-layer defense. <p>
<ul>
<li> <p> The postscreen(8) layer blocks connections from zombies
and other spambots that are responsible for about 90% of all spam.
It is implemented as a single process to make this defense as cheap
as possible. </p>
<li> <p> The second layer implements more complex SMTP-level access
checks that are available with Postfix SMTP servers, policy daemons,
and Milter applications. </p>
<li> <p> The third layer performs light-weight content inspection
with the Postfix built-in header_checks and body_checks. This can
block unacceptable attachments such as executable programs, and
worms or viruses with easy-to-recognize signatures. </p>
<li> <p> The fourth layer provides heavy-weight content inspection
with external content filters. Typical examples are Amavisd-new,
SpamAssassin, and Milter applications. </p>
</ul>
<p> Each layer reduces the spam volume. The general strategy is to
eliminate spam early with the less expensive defenses and to use
the more expensive defenses for the spam that remains. </p>
<p> Topics in this document: </p>
@ -67,10 +90,7 @@ decision based on a single measurement. This is necessary because
many zombies avoid spamming the same site repeatedly, in an attempt
to fly under the radar. Once postscreen(8) decides that a client
is not-a-zombie, it whitelists the client temporarily to avoid
further delays for legitimate mail. Clients that pass postscreen(8)
are still subject to the checks that are built into Postfix smtpd(8),
Postfix built-in content filters, and external content filters.
</p>
further delays for legitimate mail. </p>
<p> Zombies have challenges too: they have only a limited amount
of time to deliver spam before their IP address becomes blacklisted.

View File

@ -20,7 +20,7 @@
* Patches change both the patchlevel and the release date. Snapshots have no
* patchlevel; they change the release date only.
*/
#define MAIL_RELEASE_DATE "20101210"
#define MAIL_RELEASE_DATE "20101217"
#define MAIL_VERSION_NUMBER "2.8"
#ifdef SNAPSHOT

View File

@ -464,6 +464,9 @@ static void ps_service(VSTREAM *smtp_client_stream,
/*
* This program handles all incoming connections, so it must not block.
* We use event-driven code for all operations that introduce latency.
*
* Note: instead of using VSTREAM-level timeouts, we enforce limits on the
* total amount of time to receive a complete SMTP command line.
*/
non_blocking(vstream_fileno(smtp_client_stream), NON_BLOCKING);

View File

@ -554,9 +554,9 @@ static void ps_smtpd_read_event(int event, char *context)
/*
* Try to match the current character desired by the state
* machine. If that fails, try to restart the machine with a
* match for its first state. smtpd(8) incompatibility: we
* require that lines end in <CR><LF>, while smtpd(8) allows
* lines ending in <CR><LF> and bare <LF>.
* match for its first state. Like smtpd(8), we understand lines
* ending in <CR><LF> and bare <LF>. Unlike smtpd(8), we may
* treat lines ending in bare <LF> as an offense.
*/
for (transp = cmd_trans; transp->state != state->read_state; transp++)
if (transp->want == 0)

View File

@ -112,6 +112,7 @@ tls_certkey.o: ../../include/vstring.h
tls_certkey.o: tls.h
tls_certkey.o: tls_certkey.c
tls_client.o: ../../include/argv.h
tls_client.o: ../../include/iostuff.h
tls_client.o: ../../include/mail_params.h
tls_client.o: ../../include/msg.h
tls_client.o: ../../include/mymalloc.h
@ -235,6 +236,7 @@ tls_seed.o: tls_seed.c
tls_server.o: ../../include/argv.h
tls_server.o: ../../include/dict.h
tls_server.o: ../../include/hex_code.h
tls_server.o: ../../include/iostuff.h
tls_server.o: ../../include/mail_params.h
tls_server.o: ../../include/msg.h
tls_server.o: ../../include/mymalloc.h

View File

@ -92,8 +92,6 @@ typedef struct {
int cipher_algbits;
/* Private. */
SSL *con;
BIO *internal_bio; /* postfix/TLS side of pair */
BIO *network_bio; /* network side of pair */
char *cache_type; /* tlsmgr(8) cache type if enabled */
char *serverid; /* unique server identifier */
char *namaddr; /* nam[addr] for logging */

View File

@ -2,7 +2,7 @@
/* NAME
/* tls_bio_ops 3
/* SUMMARY
/* TLS network BIO management
/* TLS network basic I/O management
/* SYNOPSIS
/* #define TLS_INTERNAL
/* #include <tls.h>
@ -36,76 +36,31 @@
/* int timeout;
/* TLS_SESS_STATE *context;
/* DESCRIPTION
/* This layer synchronizes the TLS network buffers with the network
/* while performing TLS handshake or input/output operations.
/* This module enforces timeouts on non-blocking I/O while
/* performing TLS handshake or input/output operations.
/*
/* When the TLS layer is active, it converts plain-text
/* data from Postfix into encrypted network data and vice versa.
/* However, to handle network timeout conditions, Postfix
/* needs to maintain control over network input/output. This
/* rules out the usual approach of placing the TLS layer
/* between the application and the network socket.
/* The Postfix VSTREAM read/write routines invoke the
/* tls_bio_read/write routines to send and receive plain-text
/* data. In addition, this module provides tls_bio_connect/accept
/* routines that trigger the initial TLS handshake. The
/* tls_bio_xxx routines invoke the corresponding SSL routines
/* that translate the requests into TLS protocol messages.
/*
/* As shown below, Postfix reads/writes plain-text data from/to
/* the TLS layer. The TLS layer informs Postfix when it needs
/* to read/write encrypted data from/to the network; Postfix
/* then reads/writes encrypted data from/to the TLS layer and
/* takes care of the network socket I/O.
/* Whenever an SSL operation indicates that network input (or
/* output) needs to happen, the tls_bio_xxx routines wait for
/* the network to become readable (or writable) within the
/* timeout limit, then retry the SSL operation. This works
/* because the network socket is in non-blocking mode.
/*
/* The TLS layer to network interface is realized with a BIO pair:
/* tls_bio_connect() performs the SSL_connect() operation.
/*
/* Postfix SMTP layer | TLS layer
/* |
/* smtp/smtpd |
/* /\ || |
/* || \/ |
/* vstream read/write <===> TLS read/write/etc
/* | /\ ||
/* | || \/
/* | BIO pair (internal_bio)
/* | BIO pair (network_bio)
/* Postfix socket layer | /\ ||
/* | || \/
/* socket read/write <===> BIO read/write
/* /\ || |
/* || \/ |
/* network |
/* tls_bio_accept() performs the SSL_accept() operation.
/*
/* The Postfix VSTREAM read/write operations invoke the SSL
/* read/write operations to send and retrieve plain-text data. Inside
/* the TLS layer the data are converted to/from TLS protocol.
/* tls_bio_shutdown() performs the SSL_shutdown() operation.
/*
/* Whenever an SSL operation reports success, or whenever it
/* indicates that network input/output needs to happen, Postfix
/* uses the BIO read/write routines to synchronize the
/* network_bio buffer with the network. Writing data to the
/* network has precedence over reading from the network. This
/* is necessary to avoid deadlock.
/* tls_bio_read() performs the SSL_read() operation.
/*
/* The BIO pair buffer size is set to 8192 bytes. This is much
/* larger than the typical Path MTU, and avoids sending tiny TCP
/* segments. It is also larger than the default VSTREAM_BUFSIZE
/* (4096, see vstream.h), so that large write operations can
/* be handled within one request. The internal buffer in the
/* network/network_bio handling layer is set to the same
/* value, since this seems to be reasonable. The code is
/* however able to handle arbitrary values smaller or larger
/* than the buffer size in the BIO pair.
/*
/* tls_bio_connect() performs the SSL_connect() operation while
/* synchronizing the network_bio buffer with the network.
/*
/* tls_bio_accept() performs the SSL_accept() operation while
/* synchronizing the network_bio buffer with the network.
/*
/* tls_bio_shutdown() performs the SSL_shutdown() operation while
/* synchronizing the network_bio buffer with the network.
/*
/* tls_bio_read() performs the SSL_read() operation while
/* synchronizing the network_bio buffer with the network.
/*
/* tls_bio_write() performs the SSL_write() operation while
/* synchronizing the network_bio buffer with the network.
/* tls_bio_write() performs the SSL_write() operation.
/*
/* Arguments:
/* .IP fd
@ -161,87 +116,6 @@
#define TLS_INTERNAL
#include <tls.h>
/* Application-specific. */
#define NETLAYER_BUFFERSIZE 8192
/* network_biopair_interop - synchronize network with BIO pair */
static int network_biopair_interop(int fd, int timeout, BIO *network_bio)
{
const char *myname = "network_biopair_interop";
int want_write;
int num_write;
int write_pos;
int from_bio;
int want_read;
int num_read;
int to_bio;
char buffer[NETLAYER_BUFFERSIZE];
/*
* To avoid deadlock, write all pending data to the network before
* attempting to read from the network.
*/
while ((want_write = BIO_ctrl_pending(network_bio)) > 0) {
if (want_write > sizeof(buffer))
want_write = sizeof(buffer);
from_bio = BIO_read(network_bio, buffer, want_write);
/*
* Write the complete buffer contents to the network.
*/
for (write_pos = 0; write_pos < from_bio; /* see below */ ) {
if (timeout > 0 && write_wait(fd, timeout) < 0)
return (-1);
num_write = write(fd, buffer + write_pos, from_bio - write_pos);
if (num_write <= 0) {
if ((num_write < 0) && (timeout > 0) && (errno == EAGAIN)) {
msg_warn("write() returns EAGAIN on a writable file descriptor!");
msg_warn("pausing to avoid going into a tight select/write loop!");
sleep(1);
} else {
msg_warn("%s: error writing %d bytes to the network: %m",
myname, from_bio - write_pos);
return (-1);
}
} else {
write_pos += num_write;
}
}
}
/*
* Read data from the network into the BIO pair.
*/
while ((want_read = BIO_ctrl_get_read_request(network_bio)) > 0) {
if (want_read > sizeof(buffer))
want_read = sizeof(buffer);
if (timeout > 0 && read_wait(fd, timeout) < 0)
return (-1);
num_read = read(fd, buffer, want_read);
if (num_read == 0)
/* FIX 200412 Cannot return a zero read count. */
return (-1);
if (num_read < 0) {
if ((num_read < 0) && (timeout > 0) && (errno == EAGAIN)) {
msg_warn("read() returns EAGAIN on a readable file descriptor!");
msg_warn("pausing to avoid going into a tight select/write loop!");
sleep(1);
} else {
msg_warn("%s: error reading %d bytes from the network: %m",
myname, want_read);
return (-1);
}
} else {
to_bio = BIO_write(network_bio, buffer, num_read);
if (to_bio != num_read)
msg_panic("%s: BIO_write error: to_bio != num_read", myname);
}
}
return (0);
}
/* tls_bio - perform SSL input/output operation with extreme prejudice */
int tls_bio(int fd, int timeout, TLS_SESS_STATE *TLScontext,
@ -254,7 +128,6 @@ int tls_bio(int fd, int timeout, TLS_SESS_STATE *TLScontext,
int status;
int err;
int retval = 0;
int biop_retval;
int done;
/*
@ -319,13 +192,14 @@ int tls_bio(int fd, int timeout, TLS_SESS_STATE *TLScontext,
case SSL_ERROR_NONE: /* success */
retval = status;
done = 1;
/* FALLTHROUGH */
case SSL_ERROR_WANT_WRITE: /* flush/update buffers */
break;
case SSL_ERROR_WANT_WRITE:
if (write_wait(fd, timeout) < 0)
return (-1); /* timeout error */
break;
case SSL_ERROR_WANT_READ:
biop_retval =
network_biopair_interop(fd, timeout, TLScontext->network_bio);
if (biop_retval < 0)
return (-1); /* network read/write error */
if (read_wait(fd, timeout) < 0)
return (-1); /* timeout error */
break;
/*

View File

@ -139,6 +139,7 @@
#include <vstream.h>
#include <stringops.h>
#include <msg.h>
#include <iostuff.h> /* non-blocking */
/* Global library. */
@ -817,21 +818,6 @@ TLS_SESS_STATE *tls_client_start(const TLS_CLIENT_START_PROPS *props)
| ((protomask & TLS_PROTOCOL_SSLv3) ? SSL_OP_NO_SSLv3 : 0L)
| ((protomask & TLS_PROTOCOL_SSLv2) ? SSL_OP_NO_SSLv2 : 0L));
/*
* The TLS connection is realized by a BIO_pair, so obtain the pair.
*
* XXX There is no need to make internal_bio a member of the TLScontext
* structure. It will be attached to TLScontext->con, and destroyed along
* with it. The network_bio, however, needs to be freed explicitly.
*/
if (!BIO_new_bio_pair(&TLScontext->internal_bio, TLS_BIO_BUFSIZE,
&TLScontext->network_bio, TLS_BIO_BUFSIZE)) {
msg_warn("Could not obtain BIO_pair");
tls_print_errors();
tls_free_context(TLScontext);
return (0);
}
/*
* XXX To avoid memory leaks we must always call SSL_SESSION_free() after
* calling SSL_set_session(), regardless of whether or not the session
@ -876,11 +862,21 @@ TLS_SESS_STATE *tls_client_start(const TLS_CLIENT_START_PROPS *props)
SSL_set_connect_state(TLScontext->con);
/*
* Connect the SSL connection with the Postfix side of the BIO-pair for
* reading and writing.
* Connect the SSL connection with the network socket.
*/
SSL_set_bio(TLScontext->con, TLScontext->internal_bio,
TLScontext->internal_bio);
if (SSL_set_fd(TLScontext->con, vstream_fileno(props->stream)) != 1) {
msg_info("SSL_set_fd error to %s: %d", props->namaddr, sts);
tls_print_errors();
uncache_session(app_ctx->ssl_ctx, TLScontext);
tls_free_context(TLScontext);
return (0);
}
/*
* Turn on non-blocking I/O so that we can enforce timeouts on network
* I/O.
*/
non_blocking(vstream_fileno(props->stream), NON_BLOCKING);
/*
* If the debug level selected is high enough, all of the data is dumped:

View File

@ -582,8 +582,6 @@ TLS_SESS_STATE *tls_alloc_sess_context(int log_level, const char *namaddr)
TLScontext = (TLS_SESS_STATE *) mymalloc(sizeof(TLS_SESS_STATE));
memset((char *) TLScontext, 0, sizeof(*TLScontext));
TLScontext->con = 0;
TLScontext->internal_bio = 0;
TLScontext->network_bio = 0;
TLScontext->cache_type = 0;
TLScontext->serverid = 0;
TLScontext->peer_CN = 0;
@ -609,8 +607,6 @@ void tls_free_context(TLS_SESS_STATE *TLScontext)
*/
if (TLScontext->con != 0)
SSL_free(TLScontext->con);
if (TLScontext->network_bio)
BIO_free(TLScontext->network_bio);
if (TLScontext->namaddr)
myfree(TLScontext->namaddr);

View File

@ -110,6 +110,7 @@
#include <stringops.h>
#include <msg.h>
#include <hex_code.h>
#include <iostuff.h> /* non-blocking */
/* Global library. */
@ -598,22 +599,6 @@ TLS_SESS_STATE *tls_server_start(const TLS_SERVER_START_PROPS *props)
return (0);
}
/*
* The TLS connection is realized by a BIO_pair, so obtain the pair.
*
* XXX There is no need to store the internal_bio handle in the TLScontext
* structure. It will be attached to and destroyed with TLScontext->con.
* The network_bio, however, needs to be freed explicitly, so we need to
* store its handle in TLScontext.
*/
if (!BIO_new_bio_pair(&TLScontext->internal_bio, TLS_BIO_BUFSIZE,
&TLScontext->network_bio, TLS_BIO_BUFSIZE)) {
msg_warn("Could not obtain BIO_pair");
tls_print_errors();
tls_free_context(TLScontext);
return (0);
}
/*
* Before really starting anything, try to seed the PRNG a little bit
* more.
@ -629,11 +614,21 @@ TLS_SESS_STATE *tls_server_start(const TLS_SERVER_START_PROPS *props)
SSL_set_accept_state(TLScontext->con);
/*
* Connect the SSL connection with the Postfix side of the BIO-pair for
* reading and writing.
* Connect the SSL connection with the network socket.
*/
SSL_set_bio(TLScontext->con, TLScontext->internal_bio,
TLScontext->internal_bio);
if (SSL_set_fd(TLScontext->con, vstream_fileno(props->stream)) != 1) {
msg_info("SSL_set_fd error to %s: %d", props->namaddr, sts);
tls_print_errors();
uncache_session(app_ctx->ssl_ctx, TLScontext);
tls_free_context(TLScontext);
return (0);
}
/*
* Turn on non-blocking I/O so that we can enforce timeouts on network
* I/O.
*/
non_blocking(vstream_fileno(props->stream), NON_BLOCKING);
/*
* If the debug level selected is high enough, all of the data is dumped: