postfix-1.1.11-20020527

postfix/HISTORY

@@ -6473,12 +6473,14 @@ Apologies for any names omitted.
 	Feature: new MIME parser, written from scratch, that
 	recognizes the structure of MIME encapsulated mail. Influenced
 	by comments from Victor Duchovny. This code can detect but
-	will not fix illegal MIME encapsulations that Liviu Daia
+	will not decode obscure MIME formats or obscure character
-	expresses concern about.  MIME header scanning now happens
+	string encoding that Liviu Daia expresses concern about.
-	in header_checks, and is faster than body_checks could ever
+
-	be. Thus also eliminates the problem with multi-line MIME
+	MIME header scanning now happens in header_checks, and is
-	headers being matched one line at a time.  Files:
+	faster than body_checks could ever be. This also eliminates
-	global/mime_state.[hc], cleanup/cleanup_message.c.
+	the problem with multi-line MIME headers being matched one
+	line at a time.  Files:  global/mime_state.[hc],
+	cleanup/cleanup_message.c.
 
 20020521-22
 
@@ -6486,12 +6488,12 @@ Apologies for any names omitted.
 	the Postfix SMTP client. File: smtp/smtp_proto.c.
 
 	Logging: the Postfix SMTP and LMTP clients now report the
-	stage of the protocol when they report a server reply.
+	the protocol stage when they report a server reply.  File:
-	File:  smtp/smtp_proto.c, lmtp/lmtp_proto.c.
+	smtp/smtp_proto.c, lmtp/lmtp_proto.c.
 
-	Bugfix: the SMTP server warned about ignored client
+	Bugfix: the SMTP server warned about ignored client attributes
-	attributes (introduced in 20020510) in mail submitted with
+	(these were introduced 20020510) in mail that was submitted
-	"sendmail -bs".  File: smtpd/smtpd.c.
+	with "sendmail -bs".  File: smtpd/smtpd.c.
 
 20020525
 
@@ -6521,20 +6523,30 @@ Apologies for any names omitted.
 
 	Feature: disable_mime_output_conversion=yes/no controls
 	whether Postfix will convert 8BITMIME to 7BIT mail when
-	delivering to an SMTP server that does not announce 8BITMIME.
+	delivering mail to an SMTP server that does not announce
-	Default is NO.
+	8BITMIME support.  Default is NO.
 
 	Feature: strict_8bitmime=yes/no controls whether Postfix
 	rejects 8-bit characters in headers and 7-bit body parts.
-	This blocks mail from poorly written software, as well as
+	This blocks mail from poorly written software, including
-	mail that is piped into ancient /bin/mail implementations
+	majordomo approval requests that contain a valid 8BITMIME
-	that do not MIME format 8-bit content. Default is NO.
+	email message, as well as mail that is piped into ancient
+	/bin/mail implementations that do not MIME format 8-bit
+	content. Default is NO.
 
 	Feature: strict_mime_encoding_domain=yes/no controls whether
 	Postfix rejects illegal content transfer encodings for
 	multipart/* and message/*. This blocks mail from poorly
 	written software. Default is NO.
 
+20020527
+
+	Feature: "FILTER transport:nexthop" in header/body checks.
+	After the message is queued, the message is sent through
+	a content filter. This requires different cleanup servers
+	before and after the filter, with header/body checks turned
+	off in the second cleanup server.
+
 Open problems:
 
 	Medium: old maildrop files are no longer readable by the

postfix/Makefile.in

@@ -62,6 +62,7 @@ tidy:	clean
 	    *.orig */*.orig */*/*.orig \
 	    *.bak */*.bak */*/*.bak \
 	    make.err */make.err */*/make.err \
+	    *.gmon */*.gmon */*/*.gmon \
 	    conf/main.cf.default
 	find . -type s -print | xargs rm -f
 	find . -type d -print | xargs chmod 755

postfix/RELEASE_NOTES

@@ -12,36 +12,59 @@ snapshot release).  Patches change the patchlevel and the release
 date. Snapshots change only the release date, unless they include
 the same bugfixes as a patch release.
 
-Incompatible changes with Postfix snapshot 1.1.10-20020526
+Incompatible changes with Postfix snapshot 1.1.11-20020527
 ==========================================================
 
 Message headers in MIME attachments etc. are no longer matched by
 body_checks, one input line at a time. They are now by default
 matched by header_checks, one multi-line header at a time.  To get
-the old behavior, specify "disable_mime_input_processing = yes".
+the old behavior, specify "disable_mime_input_processing = yes",
+or specify separate patterns for header_checks, mime_header_checks
+and nested_header_checks. See conf/sample-mime.cf for details.
 
-Postfix rejects mail if the MIME multipart structure is nested more
+Postfix now rejects mail if the MIME multipart structure is nested
-than mime_nesting_limit levels (default: 20) while receiving mail,
+more than mime_nesting_limit levels (default: 20) when MIME input
-or when Postfix is performing 8BITMIME to 7BIT conversion while
+processing is enabled while receiving mail, or when Postfix is
-delivering mail.
+performing 8BITMIME to 7BIT conversion while delivering mail.
 
 Postfix now recognizes "name :" as a valid message header, but
 normalizes it to "name:" for consistency (actually, there is so
-much code in Postfix that would break with "name :" that there
+much code in Postfix that would break with "name :" that there is
-is little choice, except to not recognize "name :" headers).
+little choice, except to not recognize "name :" headers).
 
-Major changes with Postfix snapshot 1.1.10-20020526
+Queue files created with the header/body_checks "FILTER" feature
+are not compatible with "postqueue -r" (move queue files back to
+the maildrop directory) of previous Postfix releases.
+
+Major changes with Postfix snapshot 1.1.11-20020527
 ===================================================
 
-Postfix now properly recognizes MIME headers in attachments, which
+Postfix now has real MIME support. This improves content filtering
-is much more efficient than recognizing them via body_checks.  In
+efficiency and accuracy, and improves inter-operability with mail
-fact, Postfix now has three classes of header patterns:  header_checks
+systems that cannot receive 8-bit mail. See conf/sample-mime.cf
-(for primary message headers except MIME headers), mime_header_checks
+for details.
-(for MIME headers), and nested_header_checks (for headers of attached
+
-email messages except MIME headers).  By default, all headers are
+Postfix header_checks now properly recognize MIME headers in
-matched with header_checks.  To revert to the the old behavior,
+attachments. This is much more efficient than previous versions
-specify "disable_mime_input_processing = yes".  More details in
+that recognized MIME headers via body_checks.  MIME headers are
-conf/sample-filter.cf.
+now processed one multi-line header at a time, instead of one body
+line at a time.  
+
+In fact, Postfix now has three classes of header patterns:
+header_checks (for primary message headers except MIME headers),
+mime_header_checks (for MIME headers), and nested_header_checks
+(for headers of attached email messages except MIME headers).  By
+default, all headers are matched with header_checks.  To get the
+the old behavior, specify "disable_mime_input_processing = yes".
+More details in conf/sample-filter.cf.
+
+Selective content filtering. In header/body_check patterns, specify
+"FILTER transport:nexthop" for mail that needs filtering. This
+requires different cleanup servers before and after the filter,
+with header/body checks turned off in the second cleanup server.
+More info about content filtering is in the Postfix FILTER_README
+file. Examples for this new feature still need to be developed.
+This feature overrides the main.cf content_filter setting.
 
 The Postfix SMTP client will now convert 8BITMIME mail to 7BIT when
 delivering to an SMTP server that does not announce 8BITMIME support.
@@ -52,7 +75,8 @@ Postfix can enforce some aspects of the MIME standards while
 receiving mail. Specify "strict_8bitmime = yes" to disallow 8-bit
 characters except where allowed by the MIME standard, and specify
 "strict_mime_encoding_domain = yes" to block mail from poorly
-written mail software. More details in conf/sample-mime.cf.
+written mail software, including majordomo approval requests that
+contain valid 8BITMIME mail. More details in conf/sample-mime.cf.
 
 Incompatible changes with Postfix snapshot 1.1.10-20020514
 ==========================================================

postfix/conf/main.cf

@@ -1,5 +1,5 @@
 # Global Postfix configuration file. This file lists only a subset
-# of all 100+ parameters. See the sample-xxx.cf files for a full list.
+# of all 250+ parameters. See the sample-xxx.cf files for a full list.
 # 
 # The general format is lines with parameter = value pairs. Lines
 # that begin with whitespace continue the previous line. A value can
@@ -418,6 +418,11 @@ mail_owner = postfix
 # IGNORE the header line is silently discarded.
 # WARN   the header is logged (not rejected) with a warning message.
 # WARN text... as above, and the text is logged, too.
+# FILTER transport:nexthop
+#	after the message is queued, send the entire message through
+#	a content filter. This requires different cleanup servers
+#	before and after the filter, with header/body checks turned
+#	off in the second cleanup server.
 #
 # By default, these patterns also apply to MIME headers and to the
 # headers of attached messages. With older Postfix versions, MIME and

postfix/conf/sample-filter.cf

@@ -16,6 +16,13 @@
 # IGNORE the header line is silently discarded.
 # WARN   the header is logged (not rejected) with a warning message.
 # WARN text... as above, and the text is logged, too.
+# FILTER transport:nexthop
+#       after the message is queued, the message is sent through
+#       a content filter. This requires different cleanup servers
+#       before and after the filter, with header/body checks turned
+#       off in the second cleanup server. More info about content
+#	filtering is in the Postfix FILTER_README file. This feature
+#	overrides the main.cf content_filter setting.
 #
 # By default, these patterns apply the primary message headers, to
 # MIME headers, and to the headers of attached messages. With older
@@ -38,6 +45,13 @@ header_checks = regexp:/etc/postfix/header_checks
 # IGNORE the header line is silently discarded.
 # WARN   the header is logged (not rejected) with a warning message.
 # WARN text... as above, and the text is logged, too.
+# FILTER transport:nexthop
+#       after the message is queued, the message is sent through
+#       a content filter. This requires different cleanup servers
+#       before and after the filter, with header/body checks turned
+#       off in the second cleanup server. More info about content
+#	filtering is in the Postfix FILTER_README file. This feature
+#	overrides the main.cf content_filter setting.
 #
 # By default, the same patterns are applied as for header_checks.
 #
@@ -56,6 +70,13 @@ mime_header_checks = $header_checks
 # IGNORE the header line is silently discarded.
 # WARN   the header is logged (not rejected) with a warning message.
 # WARN text... as above, and the text is logged, too.
+# FILTER transport:nexthop
+#       after the message is queued, the message is sent through
+#       a content filter. This requires different cleanup servers
+#       before and after the filter, with header/body checks turned
+#       off in the second cleanup server. More info about content
+#	filtering is in the Postfix FILTER_README file. This feature
+#	overrides the main.cf content_filter setting.
 #
 # By default, the same patterns are applied as for header_checks.
 #
@@ -76,5 +97,12 @@ nested_header_checks = $header_checks
 # IGNORE the body line is silently discarded.
 # WARN   the body line is logged (not rejected) with a warning message.
 # WARN text... as above, and the text is logged, too.
+# FILTER transport:nexthop
+#       after the message is queued, the message is sent through
+#       a content filter. This requires different cleanup servers
+#       before and after the filter, with header/body checks turned
+#       off in the second cleanup server. More info about content
+#	filtering is in the Postfix FILTER_README file. This feature
+#	overrides the main.cf content_filter setting.
 #
 body_checks = regexp:/etc/postfix/body_checks

postfix/conf/sample-mime.cf

@@ -5,14 +5,14 @@
 # parameters that control MIME processing.
 
 # Specify "yes" to disable special processing of Content-Type: headers
-# while receiving mail. This treats all text after the primary mail
+# while RECEIVING mail. This treats all text after the primary mail
 # headers as message body text, and disables the optional features
 # below that enforce some aspects of MIME standards.
 #
 disable_mime_input_processing = no
 
 # Specify "yes" to disable special processing of Content-Type: headers
-# while delivering mail. This treats all text after the primary mail
+# while DELIVERING mail. This treats all text after the primary mail
 # headers as message body text, and disables 8BITMIME to 7BIT conversion
 # when delivering mail to an SMTP server that does not announce
 # 8BITMIME support.
@@ -23,7 +23,7 @@ disable_mime_output_conversion = no
 # boundary strings. The MIME processor is unable to distinguish
 # between boundary strings that are identical in the first
 # $mime_boundary_length_limit characters. The default limit is the
-# input line length limit.
+# Postfix input line length limit.
 #
 mime_boundary_length_limit = 2048
 

postfix/conf/sample-pcre-body.cf

@@ -26,6 +26,14 @@
 #	IGNORE		The line is silently discarded.
 #	WARN		The line is logged (not rejected) with a warning.
 #	WARN text....	As above, and the text is logged, too.
+#	FILTER transport:nexthop
+#			After the message is queued, send the entire
+#			message through a content filter. This
+#			requires different cleanup servers before
+#			and after the filter, with header/body
+#			checks turned off in the second cleanup
+#			server. More information about content filters
+#			is in the Postfix FILTER_README file.
 #
 #	Substitution of sub-strings from the matched expression is
 #	possible using the conventional perl syntax. The macros in the

postfix/conf/sample-pcre-header.cf

@@ -26,6 +26,14 @@
 #	IGNORE		The header line is silently discarded.
 #	WARN		The header is logged (not rejected) with a warning.
 #	WARN text....	As above, and the text is logged, too.
+#	FILTER transport:nexthop
+#			After the message is queued, send the entire
+#			message through a content filter. This
+#			requires different cleanup servers before
+#			and after the filter, with header/body
+#			checks turned off in the second cleanup
+#			server. More information about content filters
+#			is in the Postfix FILTER_README file.
 #
 #	Substitution of sub-strings from the matched expression is
 #	possible using the conventional perl syntax. The macros in the

postfix/conf/sample-regexp-body.cf

@@ -22,6 +22,13 @@
 #   IGNORE          The header line is silently discarded.
 #   WARN            The header is logged (not rejected) with a warning.
 #   WARN text....   As above, and the text is logged, too.
+#   FILTER transport:nexthop
+#		    After the message is queued, send the entire
+#		    message through a content filter. This requires
+#		    different cleanup servers before and after the
+#		    filter, with header/body checks turned off in
+#		    the second cleanup server. More information about 
+#		    content filters is in the Postfix FILTER_README file.
 
 # Skip over base 64 encoded blocks. This saves lots of CPU cycles.
 # Expressions by Liviu Daia. Amended by Victor Duchovni.

postfix/conf/sample-regexp-header.cf

@@ -23,6 +23,13 @@
 #   IGNORE          the header line is silently discarded.
 #   WARN            the header is logged (not rejected) with a warning.
 #   WARN text...    As above, and the text is logged, too.
+#   FILTER transport:nexthop
+#		    After the message is queued, send the entire
+#		    message through a content filter. This requires
+#		    different cleanup servers before and after the
+#		    filter, with header/body checks turned off in
+#		    the second cleanup server. More information about 
+#		    content filters is in the Postfix FILTER_README file.
 
 /^Subject: Make Money Fast/	REJECT
 /^To: friend@public.com/	REJECT

postfix/html/uce.html

@@ -138,6 +138,14 @@ the originator.
 
 <dt>WARN text... <dd> As above, and also log the text.
 
+<dt>FILTER <i>transport</i>:<i>nexthop</i> <dd>
+After the message is queued, send the entire message through
+a content filter. This requires different cleanup servers
+before and after the filter, with header/body checks turned
+off in the second cleanup server. More details about content
+filtering are in the Postfix FILTER_README file. This feature
+overrides the main.cf <b>content_filter</b> setting.
+
 </dl>
 
 <p>
@@ -207,6 +215,14 @@ the originator.
 
 <dt>WARN text... <dd> As above, and also log the text.
 
+<dt>FILTER <i>transport</i>:<i>nexthop</i> <dd>
+After the message is queued, send the entire message through
+a content filter. This requires different cleanup servers
+before and after the filter, with header/body checks turned
+off in the second cleanup server. More details about content
+filtering are in the Postfix FILTER_README file. This feature
+overrides the main.cf <b>content_filter</b> setting.
+
 </dl>
 
 <p>

postfix/src/cleanup/cleanup.h

@@ -64,6 +64,7 @@ typedef struct CLEANUP_STATE {
     NVTABLE *attr;			/* queue file attribute list */
     MIME_STATE *mime_state;		/* MIME state engine */
     int     mime_errs;			/* MIME error flags */
+    char   *filter;			/* from header/body patterns */
 } CLEANUP_STATE;
 
  /*

postfix/src/cleanup/cleanup_extracted.c

@@ -81,6 +81,14 @@ void    cleanup_extracted(CLEANUP_STATE *state, int type, char *buf, int len)
      */
     cleanup_out_string(state, REC_TYPE_XTRA, "");
 
+    /*
+     * Put the optional content filter before the mandatory Return-Receipt-To
+     * and Errors-To so that the queue manager will pick up the filter name
+     * before starting deliveries.
+     */
+    if (state->filter != 0)
+	cleanup_out_string(state, REC_TYPE_FILT, state->filter);
+
     /*
      * Older Postfix versions didn't emit encoding information, so this
      * record can only be optional. Putting this before the mandatory

postfix/src/cleanup/cleanup_message.c

@@ -294,6 +294,16 @@ static int cleanup_act(CLEANUP_STATE *state, char *context, const char *buf,
 		 cleanup_strerror(CLEANUP_STAT_CONT));
 	return (CLEANUP_ACT_KEEP);
     }
+    if (STREQUAL(value, "FILTER", command_len)) {
+	if (*optional_text == 0) {
+	    msg_warn("missing FILTER command argument in %s map", map_class);
+	} else {
+	    if (state->filter)
+		myfree(state->filter);
+	    state->filter = mystrdup(optional_text);
+	}
+	return (CLEANUP_ACT_KEEP);
+    }
     if (*optional_text)
 	msg_warn("unexpected text after command in %s map: %s",
 		 map_class, value);

postfix/src/cleanup/cleanup_state.c

@@ -89,6 +89,7 @@ CLEANUP_STATE *cleanup_state_alloc(void)
     state->attr = nvtable_create(10);
     state->mime_state = 0;
     state->mime_errs = 0;
+    state->filter = 0;
     return (state);
 }
 
@@ -122,5 +123,7 @@ void    cleanup_state_free(CLEANUP_STATE *state)
     nvtable_free(state->attr);
     if (state->mime_state)
 	mime_state_free(state->mime_state);
+    if (state->filter)
+	myfree(state->filter);
     myfree((char *) state);
 }

postfix/src/global/mail_version.h

@@ -20,10 +20,10 @@
   * Patches change the patchlevel and the release date. Snapshots change the
   * release date only, unless they include the same bugfix as a patch release.
   */
-#define MAIL_RELEASE_DATE	"20020526"
+#define MAIL_RELEASE_DATE	"20020527"
 
 #define VAR_MAIL_VERSION	"mail_version"
-#define DEF_MAIL_VERSION	"1.1.10-" MAIL_RELEASE_DATE
+#define DEF_MAIL_VERSION	"1.1.11-" MAIL_RELEASE_DATE
 extern char *var_mail_version;
 
  /*

postfix/src/global/rec_type.h

@@ -64,8 +64,7 @@
   */
 #define REC_TYPE_ENVELOPE	"MCTFILSDROWVA"
 #define REC_TYPE_CONTENT	"XLN"
-#define REC_TYPE_EXTRACT	"EDROPreA"
+#define REC_TYPE_EXTRACT	"EDROPreAFI"
-#define REC_TYPE_NOEXTRACT	"E"
 
  /*
   * The record at the beginning of the envelope segment specifies the message

postfix/src/nqmgr/qmgr_message.c

@@ -378,11 +378,13 @@ static int qmgr_message_read(QMGR_MESSAGE *message)
 	    if (message->arrival_time == 0)
 		message->arrival_time = atol(start);
 	} else if (rec_type == REC_TYPE_FILT) {
-	    if (message->filter_xport == 0)
+	    if (message->filter_xport != 0)
-		message->filter_xport = mystrdup(start);
+		myfree(message->filter_xport);
+	    message->filter_xport = mystrdup(start);
 	} else if (rec_type == REC_TYPE_INSP) {
-	    if (message->inspect_xport == 0)
+	    if (message->inspect_xport != 0)
-		message->inspect_xport = mystrdup(start);
+		myfree(message->inspect_xport);
+	    message->inspect_xport = mystrdup(start);
 	} else if (rec_type == REC_TYPE_FROM) {
 	    if (message->sender == 0) {
 		message->sender = mystrdup(start);

postfix/src/pickup/pickup.c

@@ -212,7 +212,7 @@ static int copy_segment(VSTREAM *qfile, VSTREAM *cleanup, PICKUP_INFO *info,
 		     (long) info->st.st_uid, attr_name, attr_value);
 	    continue;
 	}
-	if (type == REC_TYPE_FILT && *expected == REC_TYPE_ENVELOPE[0])
+	if (type == REC_TYPE_FILT && info->st.st_uid != var_owner_uid)
 	    continue;
 	else {
 

postfix/src/qmgr/qmgr_message.c

@@ -259,11 +259,13 @@ static int qmgr_message_read(QMGR_MESSAGE *message)
 	    if (message->arrival_time == 0)
 		message->arrival_time = atol(start);
 	} else if (rec_type == REC_TYPE_FILT) {
-	    if (message->filter_xport == 0)
+	    if (message->filter_xport != 0)
-		message->filter_xport = mystrdup(start);
+		myfree(message->filter_xport);
+	    message->filter_xport = mystrdup(start);
 	} else if (rec_type == REC_TYPE_INSP) {
-	    if (message->inspect_xport == 0)
+	    if (message->inspect_xport != 0)
-		message->inspect_xport = mystrdup(start);
+		myfree(message->inspect_xport);
+	    message->inspect_xport = mystrdup(start);
 	} else if (rec_type == REC_TYPE_FROM) {
 	    if (message->sender == 0) {
 		message->sender = mystrdup(start);