postfix-3.2-20161105

postfix/HISTORY

@@ -22545,3 +22545,17 @@ Apologies for any names omitted.
 	Files: mantools/postlink, proto/MILTER_README.html,
 	proto/postconf.proto, global/mail_params.h, smtpd/smtpd.c,
 	smtpd/smtpd.h, smtpd/smtpd_sasl_proto.c, smtpd/smtpd_state.c.
+
+20161103
+
+	Cleanup: error reporting for IDNA (non-ASCII domain name)
+	conversion errors. File: util/midna_domain.c.
+
+	Cleanup: non-transitional conversion of UTF8 to/from ASCII
+	domain name labels used in DNS queries. This disables
+	'transitional' compatibility between IDNA2003 and IDNA2008,
+	and affects some corner cases such as German sz and Greek
+	zeta. Specify "enable_idna2003_compatibility = yes" to
+	restore historical behavior. Files: util/midna_domain.[hc],
+	mantools/postlink, global/mail_params.[hc], proto/postconf.proto,
+	proto/SMTPUTF8_README.html.

postfix/README_FILES/SMTPUTF8_README

@@ -18,6 +18,7 @@ Topics covered in this document:
   * SMTPUTF8 autodetection
   * Limitations of the current implementation
   * Compatibility with pre-SMTPUTF8 environments
+  * Compatibility with IDNA2003
   * Credits
 
 BBuuiillddiinngg PPoossttffiixx wwiitthh//wwiitthhoouutt SSMMTTPPUUTTFF88 ssuuppppoorrtt
@@ -259,6 +260,18 @@ at the same time. There is no problem with RFC 2047 encodings such as "=?ISO-
 8859-1?Q?text?=", because those use only characters from the ASCII
 characterset.
 
+CCoommppaattiibbiilliittyy wwiitthh IIDDNNAA22000033
+
+Postfix >= 3.2 by default disables the 'transitional' compatibility between
+IDNA2003 and IDNA2008, when converting UTF-8 domain names to/from the ASCII
+form that is used in DNS lookups. This makes Postfix behavior consistent with
+current versions of the Firefox and Chrome web browsers. Specify
+"enable_idna2003_compatibility = yes" to get the historical behavior.
+
+This affects the conversion of domain names that contain for example the German
+sz (ß) and the Greek zeta (ς). See http://unicode.org/cldr/utility/idna.jsp
+for more examples.
+
 CCrreeddiittss
 
   * May 15, 2014: Arnt Gulbrandsen posted his patch for Unicode email support.

postfix/RELEASE_NOTES

@@ -16,6 +16,20 @@ specifies the release date of a stable release or snapshot release.
 If you upgrade from Postfix 3.0 or earlier, read RELEASE_NOTES-3.1
 before proceeding.
 
+Incompatible changes with snapshot 20161103
+===========================================
+
+Postfix 3.2 by default disables the 'transitional' compatibility
+between IDNA2003 and IDNA2008, when converting UTF-8 domain names
+to/from the ASCII form that is used in DNS lookups.  This makes
+Postfix behavior consistent with current versions of the Firefox
+and Chrome web browsers. Specify "enable_idna2003_compatibility =
+yes" for historical behavior.
+
+This affects the conversion of, for example, the German sz and the
+Greek zeta. See http://unicode.org/cldr/utility/idna.jsp for more
+examples.
+
 Major changes with snapshot 20161031
 ====================================
 

postfix/html/SMTPUTF8_README.html

@@ -7,7 +7,7 @@
 
 <title>Postfix SMTPUTF8 support</title>
 
-<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
+<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 
 </head>
 
@@ -44,6 +44,8 @@ header values. </p>
 
 <li><a href="#compatibility">Compatibility with pre-SMTPUTF8 environments</a>
 
+<li><a href="#idna2003">Compatibility with IDNA2003</a>
+
 <li><a href="#credits">Credits</a>
 
 </ul>
@@ -351,6 +353,19 @@ for Postfix to support multiple encodings at the same time.  There
 is no problem with <a href="http://tools.ietf.org/html/rfc2047">RFC 2047</a> encodings such as "=?ISO-8859-1?Q?text?=",
 because those use only characters from the ASCII characterset.  </p>
 
+<h2> <a name="idna2003">Compatibility with IDNA2003</a> </h2>
+
+<p> Postfix &ge; 3.2 by default disables the 'transitional'
+compatibility between IDNA2003 and IDNA2008, when converting UTF-8
+domain names to/from the ASCII form that is used in DNS lookups.
+This makes Postfix behavior consistent with current versions of the
+Firefox and Chrome web browsers. Specify "<a href="postconf.5.html#enable_idna2003_compatibility">enable_idna2003_compatibility</a>
+= yes" to get the historical behavior. </p>
+
+This affects the conversion of domain names that contain for example
+the German sz (ß) and the Greek zeta (ς). See
+<a href="http://unicode.org/cldr/utility/idna.jsp">http://unicode.org/cldr/utility/idna.jsp</a> for more examples. </p>
+
 <h2> <a name="credits">Credits</a> </h2>
 
 <ul>

postfix/html/cleanup.8.html

@@ -414,6 +414,13 @@ CLEANUP(8)                                                          CLEANUP(8)
               Detect that a message requires SMTPUTF8 support for  the  speci-
               fied mail origin classes.
 
+       Available in Postfix version 3.2 and later:
+
+       <b><a href="postconf.5.html#enable_idna2003_compatibility">enable_idna2003_compatibility</a> (no)</b>
+              Enable   'transitional'   compatibility   between  IDNA2003  and
+              IDNA2008, when converting UTF-8 domain names to/from  the  ASCII
+              form that is used for DNS lookups.
+
 <b>MISCELLANEOUS CONTROLS</b>
        <b><a href="postconf.5.html#config_directory">config_directory</a> (see 'postconf -d' output)</b>
               The  default  location of the Postfix <a href="postconf.5.html">main.cf</a> and <a href="master.5.html">master.cf</a> con-

postfix/html/lmtp.8.html

@@ -720,6 +720,13 @@ SMTP(8)                                                                SMTP(8)
               Detect that a message requires SMTPUTF8 support for  the  speci-
               fied mail origin classes.
 
+       Available in Postfix version 3.2 and later:
+
+       <b><a href="postconf.5.html#enable_idna2003_compatibility">enable_idna2003_compatibility</a> (no)</b>
+              Enable   'transitional'   compatibility   between  IDNA2003  and
+              IDNA2008, when converting UTF-8 domain names to/from  the  ASCII
+              form that is used for DNS lookups.
+
 <b>TROUBLE SHOOTING CONTROLS</b>
        <b><a href="postconf.5.html#debug_peer_level">debug_peer_level</a> (2)</b>
               The  increment  in verbose logging level when a remote client or

postfix/html/postconf.5.html

@@ -3079,6 +3079,22 @@ turned off by default with Postfix version 2.1, and is always turned on
 with older Postfix versions).  </p>
 
 
+</DD>
+
+<DT><b><a name="enable_idna2003_compatibility">enable_idna2003_compatibility</a>
+(default: no)</b></DT><DD>
+
+<p> Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+when converting UTF-8 domain names to/from the ASCII form that is
+used for DNS lookups. Specify "yes" for compatibility with Postfix
+&le; 3.1 (not recommended). This affects the conversion of domain
+names that contain for example the German sz and the Greek zeta.
+See <a href="http://unicode.org/cldr/utility/idna.jsp">http://unicode.org/cldr/utility/idna.jsp</a> for more examples.
+</p>
+
+<p> This feature is available in Postfix 3.2 and later.  </p>
+
+
 </DD>
 
 <DT><b><a name="enable_long_queue_ids">enable_long_queue_ids</a>

postfix/html/qmqpd.8.html

@@ -66,6 +66,13 @@ QMQPD(8)                                                              QMQPD(8)
               Detect  that  a message requires SMTPUTF8 support for the speci-
               fied mail origin classes.
 
+       Available in Postfix version 3.2 and later:
+
+       <b><a href="postconf.5.html#enable_idna2003_compatibility">enable_idna2003_compatibility</a> (no)</b>
+              Enable  'transitional'  compatibility   between   IDNA2003   and
+              IDNA2008,  when  converting UTF-8 domain names to/from the ASCII
+              form that is used for DNS lookups.
+
 <b>RESOURCE AND RATE CONTROLS</b>
        <b><a href="postconf.5.html#line_length_limit">line_length_limit</a> (2048)</b>
               Upon input, long lines are chopped up into  pieces  of  at  most

postfix/html/smtp.8.html

@@ -720,6 +720,13 @@ SMTP(8)                                                                SMTP(8)
               Detect that a message requires SMTPUTF8 support for  the  speci-
               fied mail origin classes.
 
+       Available in Postfix version 3.2 and later:
+
+       <b><a href="postconf.5.html#enable_idna2003_compatibility">enable_idna2003_compatibility</a> (no)</b>
+              Enable   'transitional'   compatibility   between  IDNA2003  and
+              IDNA2008, when converting UTF-8 domain names to/from  the  ASCII
+              form that is used for DNS lookups.
+
 <b>TROUBLE SHOOTING CONTROLS</b>
        <b><a href="postconf.5.html#debug_peer_level">debug_peer_level</a> (2)</b>
               The  increment  in verbose logging level when a remote client or

postfix/html/smtpd.8.html

@@ -596,6 +596,13 @@ SMTPD(8)                                                              SMTPD(8)
               Detect that a message requires SMTPUTF8 support for  the  speci-
               fied mail origin classes.
 
+       Available in Postfix version 3.2 and later:
+
+       <b><a href="postconf.5.html#enable_idna2003_compatibility">enable_idna2003_compatibility</a> (no)</b>
+              Enable   'transitional'   compatibility   between  IDNA2003  and
+              IDNA2008, when converting UTF-8 domain names to/from  the  ASCII
+              form that is used for DNS lookups.
+
 <b>VERP SUPPORT CONTROLS</b>
        With  VERP  style delivery, each recipient of a message receives a cus-
        tomized copy of the message with his/her own recipient address  encoded

postfix/html/verify.8.html

@@ -169,6 +169,13 @@ VERIFY(8)                                                            VERIFY(8)
               Detect  that  a message requires SMTPUTF8 support for the speci-
               fied mail origin classes.
 
+       Available in Postfix version 3.2 and later:
+
+       <b><a href="postconf.5.html#enable_idna2003_compatibility">enable_idna2003_compatibility</a> (no)</b>
+              Enable  'transitional'  compatibility   between   IDNA2003   and
+              IDNA2008,  when  converting UTF-8 domain names to/from the ASCII
+              form that is used for DNS lookups.
+
 <b>MISCELLANEOUS CONTROLS</b>
        <b><a href="postconf.5.html#config_directory">config_directory</a> (see 'postconf -d' output)</b>
               The default location of the Postfix <a href="postconf.5.html">main.cf</a> and  <a href="master.5.html">master.cf</a>  con-

postfix/man/man5/postconf.5

@@ -1907,6 +1907,15 @@ non\-standard Errors\-To: message header, instead of the envelope
 sender address (this feature is removed with Postfix version 2.2, is
 turned off by default with Postfix version 2.1, and is always turned on
 with older Postfix versions).
+.SH enable_idna2003_compatibility (default: no)
+Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+when converting UTF\-8 domain names to/from the ASCII form that is
+used for DNS lookups. Specify "yes" for compatibility with Postfix
+<= 3.1 (not recommended). This affects the conversion of domain
+names that contain for example the German sz and the Greek zeta.
+See http://unicode.org/cldr/utility/idna.jsp for more examples.
+.PP
+This feature is available in Postfix 3.2 and later.
 .SH enable_long_queue_ids (default: no)
 Enable long, non\-repeating, queue IDs (queue file names).  The
 benefit of non\-repeating names is simpler logfile analysis and

postfix/man/man8/cleanup.8

@@ -370,6 +370,12 @@ in RFC 6531..6533.
 .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 Detect that a message requires SMTPUTF8 support for the specified
 mail origin classes.
+.PP
+Available in Postfix version 3.2 and later:
+.IP "\fBenable_idna2003_compatibility (no)\fR"
+Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+when converting UTF\-8 domain names to/from the ASCII form that is
+used for DNS lookups.
 .SH "MISCELLANEOUS CONTROLS"
 .na
 .nf

postfix/man/man8/qmqpd.8

@@ -78,6 +78,12 @@ in RFC 6531..6533.
 .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 Detect that a message requires SMTPUTF8 support for the specified
 mail origin classes.
+.PP
+Available in Postfix version 3.2 and later:
+.IP "\fBenable_idna2003_compatibility (no)\fR"
+Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+when converting UTF\-8 domain names to/from the ASCII form that is
+used for DNS lookups.
 .SH "RESOURCE AND RATE CONTROLS"
 .na
 .nf

postfix/man/man8/smtp.8

@@ -631,6 +631,12 @@ in RFC 6531..6533.
 .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 Detect that a message requires SMTPUTF8 support for the specified
 mail origin classes.
+.PP
+Available in Postfix version 3.2 and later:
+.IP "\fBenable_idna2003_compatibility (no)\fR"
+Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+when converting UTF\-8 domain names to/from the ASCII form that is
+used for DNS lookups.
 .SH "TROUBLE SHOOTING CONTROLS"
 .na
 .nf

postfix/man/man8/smtpd.8

@@ -538,6 +538,12 @@ Enable stricter enforcement of the SMTPUTF8 protocol.
 .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 Detect that a message requires SMTPUTF8 support for the specified
 mail origin classes.
+.PP
+Available in Postfix version 3.2 and later:
+.IP "\fBenable_idna2003_compatibility (no)\fR"
+Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+when converting UTF\-8 domain names to/from the ASCII form that is
+used for DNS lookups.
 .SH "VERP SUPPORT CONTROLS"
 .na
 .nf

postfix/man/man8/verify.8

@@ -177,6 +177,12 @@ Preliminary SMTPUTF8 support is introduced with Postfix 3.0.
 .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 Detect that a message requires SMTPUTF8 support for the specified
 mail origin classes.
+.PP
+Available in Postfix version 3.2 and later:
+.IP "\fBenable_idna2003_compatibility (no)\fR"
+Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+when converting UTF\-8 domain names to/from the ASCII form that is
+used for DNS lookups.
 .SH "MISCELLANEOUS CONTROLS"
 .na
 .nf

postfix/mantools/postlink

@@ -1079,6 +1079,7 @@ while (<>) {
     s;\bsmtputf8_enable\b;<a href="postconf.5.html#smtputf8_enable">$&</a>;g;
     s;\bstrict_smtputf8\b;<a href="postconf.5.html#strict_smtputf8">$&</a>;g;
     s;\bsmtputf8_autodetect_classes\b;<a href="postconf.5.html#smtputf8_autodetect_classes">$&</a>;g;
+    s;\benable_idna2003_compatibility\b;<a href="postconf.5.html#enable_idna2003_compatibility">$&</a>;g;
 
     # Service-defined parameters...
 

postfix/proto/SMTPUTF8_README.html

@@ -7,7 +7,7 @@
 
 <title>Postfix SMTPUTF8 support</title>
 
-<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
+<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 
 </head>
 
@@ -44,6 +44,8 @@ header values. </p>
 
 <li><a href="#compatibility">Compatibility with pre-SMTPUTF8 environments</a>
 
+<li><a href="#idna2003">Compatibility with IDNA2003</a>
+
 <li><a href="#credits">Credits</a>
 
 </ul>
@@ -351,6 +353,19 @@ for Postfix to support multiple encodings at the same time.  There
 is no problem with RFC 2047 encodings such as "=?ISO-8859-1?Q?text?=",
 because those use only characters from the ASCII characterset.  </p>
 
+<h2> <a name="idna2003">Compatibility with IDNA2003</a> </h2>
+
+<p> Postfix &ge; 3.2 by default disables the 'transitional'
+compatibility between IDNA2003 and IDNA2008, when converting UTF-8
+domain names to/from the ASCII form that is used in DNS lookups.
+This makes Postfix behavior consistent with current versions of the
+Firefox and Chrome web browsers. Specify "enable_idna2003_compatibility
+= yes" to get the historical behavior. </p>
+
+This affects the conversion of domain names that contain for example
+the German sz (ß) and the Greek zeta (ς). See
+http://unicode.org/cldr/utility/idna.jsp for more examples. </p>
+
 <h2> <a name="credits">Credits</a> </h2>
 
 <ul>

postfix/proto/postconf.proto

@@ -16704,3 +16704,15 @@ Milter support should be disabled. </p>
 </pre>
 
 <p> This feature is available in Postfix 3.2 and later.  </p>
+
+%PARAM enable_idna2003_compatibility no
+
+<p> Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+when converting UTF-8 domain names to/from the ASCII form that is
+used for DNS lookups. Specify "yes" for compatibility with Postfix
+&le; 3.1 (not recommended). This affects the conversion of domain
+names that contain for example the German sz and the Greek zeta.
+See http://unicode.org/cldr/utility/idna.jsp for more examples.
+</p>
+
+<p> This feature is available in Postfix 3.2 and later.  </p>

postfix/src/cleanup/cleanup.c

@@ -340,6 +340,12 @@
 /* .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 /*	Detect that a message requires SMTPUTF8 support for the specified
 /*	mail origin classes.
+/* .PP
+/*	Available in Postfix version 3.2 and later:
+/* .IP "\fBenable_idna2003_compatibility (no)\fR"
+/*	Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+/*	when converting UTF-8 domain names to/from the ASCII form that is
+/*	used for DNS lookups.
 /* MISCELLANEOUS CONTROLS
 /* .ad
 /* .fi

postfix/src/global/Makefile.in

@@ -1087,8 +1087,27 @@ dict_memcache.o: dict_memcache.c
 dict_memcache.o: dict_memcache.h
 dict_memcache.o: memcache_proto.h
 dict_memcache.o: string_list.h
+dict_mysql.o: ../../include/argv.h
+dict_mysql.o: ../../include/check_arg.h
+dict_mysql.o: ../../include/dict.h
+dict_mysql.o: ../../include/events.h
+dict_mysql.o: ../../include/find_inet.h
+dict_mysql.o: ../../include/match_list.h
+dict_mysql.o: ../../include/msg.h
+dict_mysql.o: ../../include/myflock.h
+dict_mysql.o: ../../include/mymalloc.h
+dict_mysql.o: ../../include/myrand.h
+dict_mysql.o: ../../include/split_at.h
+dict_mysql.o: ../../include/stringops.h
 dict_mysql.o: ../../include/sys_defs.h
+dict_mysql.o: ../../include/vbuf.h
+dict_mysql.o: ../../include/vstream.h
+dict_mysql.o: ../../include/vstring.h
+dict_mysql.o: cfg_parser.h
+dict_mysql.o: db_common.h
 dict_mysql.o: dict_mysql.c
+dict_mysql.o: dict_mysql.h
+dict_mysql.o: string_list.h
 dict_pgsql.o: ../../include/argv.h
 dict_pgsql.o: ../../include/check_arg.h
 dict_pgsql.o: ../../include/dict.h
@@ -1715,6 +1734,7 @@ mail_params.o: ../../include/htable.h
 mail_params.o: ../../include/inet_addr_list.h
 mail_params.o: ../../include/inet_proto.h
 mail_params.o: ../../include/iostuff.h
+mail_params.o: ../../include/midna_domain.h
 mail_params.o: ../../include/msg.h
 mail_params.o: ../../include/msg_syslog.h
 mail_params.o: ../../include/myaddrinfo.h
@@ -2494,6 +2514,7 @@ smtp_reply_footer.o: ../../include/check_arg.h
 smtp_reply_footer.o: ../../include/mac_expand.h
 smtp_reply_footer.o: ../../include/mac_parse.h
 smtp_reply_footer.o: ../../include/msg.h
+smtp_reply_footer.o: ../../include/mymalloc.h
 smtp_reply_footer.o: ../../include/sys_defs.h
 smtp_reply_footer.o: ../../include/vbuf.h
 smtp_reply_footer.o: ../../include/vstring.h

postfix/src/global/mail_params.c

@@ -167,6 +167,11 @@
 /*	IBM T.J. Watson Research
 /*	P.O. Box 704
 /*	Yorktown Heights, NY 10598, USA
+/*
+/*	Wietse Venema
+/*	Google, Inc.
+/*	111 8th Avenue
+/*	New York, NY 10011, USA
 /*--*/
 
 /* System library. */
@@ -196,6 +201,7 @@
 #include <inet_proto.h>
 #include <vstring_vstream.h>
 #include <iostuff.h>
+#include <midna_domain.h>
 
 /* Global library. */
 
@@ -652,6 +658,7 @@ void    mail_params_init()
     static const CONFIG_NBOOL_TABLE first_nbool_defaults[] = {
 	/* read and process the following before opening tables. */
 	VAR_SMTPUTF8_ENABLE, DEF_SMTPUTF8_ENABLE, &var_smtputf8_enable,
+	VAR_IDNA2003_COMPAT, DEF_IDNA2003_COMPAT, &midna_domain_transitional,
 	0,
     };
     static const CONFIG_STR_FN_TABLE function_str_defaults[] = {

postfix/src/global/mail_params.h

@@ -3955,6 +3955,9 @@ extern int var_strict_smtputf8;
 					MAIL_SRC_NAME_VERIFY
 extern char *var_smtputf8_autoclass;
 
+#define VAR_IDNA2003_COMPAT		"enable_idna2003_compatibility"
+#define DEF_IDNA2003_COMPAT		"no"
+
  /*
   * Workaround for future incompatibility. Our implementation of RFC 2308
   * negative reply caching relies on the promise that res_query() and
@@ -3976,6 +3979,11 @@ extern bool var_dns_ncache_ttl_fix;
 /*	IBM T.J. Watson Research
 /*	P.O. Box 704
 /*	Yorktown Heights, NY 10598, USA
+/*
+/*	Wietse Venema
+/*	Google, Inc.
+/*	111 8th Avenue
+/*	New York, NY 10011, USA
 /*--*/
 
 #endif

postfix/src/global/mail_version.h

@@ -20,7 +20,7 @@
   * Patches change both the patchlevel and the release date. Snapshots have no
   * patchlevel; they change the release date only.
   */
-#define MAIL_RELEASE_DATE	"20161101"
+#define MAIL_RELEASE_DATE	"20161105"
 #define MAIL_VERSION_NUMBER	"3.2"
 
 #ifdef SNAPSHOT

postfix/src/qmqpd/qmqpd.c

@@ -60,6 +60,12 @@
 /* .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 /*	Detect that a message requires SMTPUTF8 support for the specified
 /*	mail origin classes.
+/* .PP
+/*	Available in Postfix version 3.2 and later:
+/* .IP "\fBenable_idna2003_compatibility (no)\fR"
+/*	Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+/*	when converting UTF-8 domain names to/from the ASCII form that is
+/*	used for DNS lookups.
 /* RESOURCE AND RATE CONTROLS
 /* .ad
 /* .fi

postfix/src/smtp/smtp.c

@@ -595,6 +595,12 @@
 /* .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 /*	Detect that a message requires SMTPUTF8 support for the specified
 /*	mail origin classes.
+/* .PP
+/*	Available in Postfix version 3.2 and later:
+/* .IP "\fBenable_idna2003_compatibility (no)\fR"
+/*	Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+/*	when converting UTF-8 domain names to/from the ASCII form that is
+/*	used for DNS lookups.
 /* TROUBLE SHOOTING CONTROLS
 /* .ad
 /* .fi

postfix/src/smtpd/smtpd.c

@@ -500,6 +500,12 @@
 /* .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 /*	Detect that a message requires SMTPUTF8 support for the specified
 /*	mail origin classes.
+/* .PP
+/*	Available in Postfix version 3.2 and later:
+/* .IP "\fBenable_idna2003_compatibility (no)\fR"
+/*	Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+/*	when converting UTF-8 domain names to/from the ASCII form that is
+/*	used for DNS lookups.
 /* VERP SUPPORT CONTROLS
 /* .ad
 /* .fi

postfix/src/util/Makefile.in

@@ -1110,6 +1110,7 @@ dict_cidr.o: dict.h
 dict_cidr.o: dict_cidr.c
 dict_cidr.o: dict_cidr.h
 dict_cidr.o: msg.h
+dict_cidr.o: mvect.h
 dict_cidr.o: myaddrinfo.h
 dict_cidr.o: myflock.h
 dict_cidr.o: mymalloc.h
@@ -1750,6 +1751,8 @@ load_file.o: vbuf.h
 load_file.o: vstream.h
 load_file.o: warn_stat.h
 load_lib.o: load_lib.c
+load_lib.o: load_lib.h
+load_lib.o: msg.h
 load_lib.o: sys_defs.h
 lowercase.o: check_arg.h
 lowercase.o: lowercase.c
@@ -1831,6 +1834,7 @@ midna_domain.o: midna_domain.c
 midna_domain.o: midna_domain.h
 midna_domain.o: msg.h
 midna_domain.o: mymalloc.h
+midna_domain.o: name_mask.h
 midna_domain.o: stringops.h
 midna_domain.o: sys_defs.h
 midna_domain.o: valid_hostname.h

postfix/src/util/midna_domain.c

@@ -7,6 +7,7 @@
 /*	#include <midna_domain.h>
 /*
 /*	int midna_domain_cache_size;
+/*	int midna_domain_transitional;
 /*
 /*	const char *midna_domain_to_ascii(
 /*	const char *name)
@@ -47,8 +48,10 @@
 /*
 /*	midna_domain_cache_size specifies the size of the conversion
 /*	result cache.  This value is used only once, upon the first
-/*	lookup
+/*	lookup request.
-/*	request.
+/*
+/*	midna_domain_transitional enables transitional conversion
+/*	between UTF8 and ASCII labels.
 /* SEE ALSO
 /*	http://unicode.org/reports/tr46/ Unicode IDNA Compatibility processing
 /*	msg(3) diagnostics interface
@@ -66,6 +69,11 @@
 /*	IBM T.J. Watson Research
 /*	P.O. Box 704
 /*	Yorktown Heights, NY 10598, USA
+/*
+/*	Wietse Venema
+/*	Google, Inc.
+/*	111 8th Avenue
+/*	New York, NY 10011, USA
 /*--*/
 
  /*
@@ -86,6 +94,7 @@
 #include <ctable.h>
 #include <stringops.h>
 #include <valid_hostname.h>
+#include <name_mask.h>
 #include <midna_domain.h>
 
  /*
@@ -94,10 +103,73 @@
 #define DEF_MIDNA_CACHE_SIZE	256
 
 int     midna_domain_cache_size = DEF_MIDNA_CACHE_SIZE;
+int     midna_domain_transitional = 0;
 static VSTRING *midna_domain_buf;	/* x.suffix */
 
 #define STR(x)	vstring_str(x)
 
+/* midna_domain_strerror - pick one for error reporting */
+
+static const char *midna_domain_strerror(UErrorCode error, int info_errors)
+{
+    static LONG_NAME_MASK uidna_errors[] = {
+#ifdef UIDNA_ERROR_EMPTY_LABEL
+	"UIDNA_ERROR_EMPTY_LABEL", UIDNA_ERROR_EMPTY_LABEL,
+#endif
+#ifdef UIDNA_ERROR_LABEL_TOO_LONG
+	"UIDNA_ERROR_LABEL_TOO_LONG", UIDNA_ERROR_LABEL_TOO_LONG,
+#endif
+#ifdef UIDNA_ERROR_DOMAIN_NAME_TOO_LONG
+	"UIDNA_ERROR_DOMAIN_NAME_TOO_LONG", UIDNA_ERROR_DOMAIN_NAME_TOO_LONG,
+#endif
+#ifdef UIDNA_ERROR_LEADING_HYPHEN
+	"UIDNA_ERROR_LEADING_HYPHEN", UIDNA_ERROR_LEADING_HYPHEN,
+#endif
+#ifdef UIDNA_ERROR_TRAILING_HYPHEN
+	"UIDNA_ERROR_TRAILING_HYPHEN", UIDNA_ERROR_TRAILING_HYPHEN,
+#endif
+#ifdef UIDNA_ERROR_HYPHEN_3_4
+	"UIDNA_ERROR_HYPHEN_3_4", UIDNA_ERROR_HYPHEN_3_4,
+#endif
+#ifdef UIDNA_ERROR_LEADING_COMBINING_MARK
+	"UIDNA_ERROR_LEADING_COMBINING_MARK", UIDNA_ERROR_LEADING_COMBINING_MARK,
+#endif
+#ifdef UIDNA_ERROR_DISALLOWED
+	"UIDNA_ERROR_DISALLOWED", UIDNA_ERROR_DISALLOWED,
+#endif
+#ifdef UIDNA_ERROR_PUNYCODE
+	"UIDNA_ERROR_PUNYCODE", UIDNA_ERROR_PUNYCODE,
+#endif
+#ifdef UIDNA_ERROR_LABEL_HAS_DOT
+	"UIDNA_ERROR_LABEL_HAS_DOT", UIDNA_ERROR_LABEL_HAS_DOT,
+#endif
+#ifdef UIDNA_ERROR_INVALID_ACE_LABEL
+	"UIDNA_ERROR_INVALID_ACE_LABEL", UIDNA_ERROR_INVALID_ACE_LABEL,
+#endif
+#ifdef UIDNA_ERROR_BIDI
+	"UIDNA_ERROR_BIDI", UIDNA_ERROR_BIDI,
+#endif
+#ifdef UIDNA_ERROR_CONTEXTJ
+	"UIDNA_ERROR_CONTEXTJ", UIDNA_ERROR_CONTEXTJ,
+#endif
+#ifdef UIDNA_ERROR_CONTEXTO_PUNCTUATION
+	"UIDNA_ERROR_CONTEXTO_PUNCTUATION", UIDNA_ERROR_CONTEXTO_PUNCTUATION,
+#endif
+#ifdef UIDNA_ERROR_CONTEXTO_DIGITS
+	"UIDNA_ERROR_CONTEXTO_DIGITS", UIDNA_ERROR_CONTEXTO_DIGITS,
+#endif
+	0,
+    };
+
+    if (info_errors) {
+	return (str_long_name_mask_opt((VSTRING *) 0, "idna error",
+				       uidna_errors, info_errors,
+				       NAME_MASK_NUMBER | NAME_MASK_COMMA));
+    } else {
+	return u_errorName(error);
+    }
+}
+
 /* midna_domain_to_ascii_create - convert domain to ASCII */
 
 static void *midna_domain_to_ascii_create(const char *name, void *unused_context)
@@ -121,7 +193,8 @@ static void *midna_domain_to_ascii_create(const char *name, void *unused_context
     /*
      * Perform the requested conversion.
      */
-    idna = uidna_openUTS46(UIDNA_DEFAULT, &error);/* XXX check error */
+    idna = uidna_openUTS46(midna_domain_transitional ? UIDNA_DEFAULT
+			   : UIDNA_NONTRANSITIONAL_TO_ASCII, &error);
     anl = uidna_nameToASCII_UTF8(idna,
 				 name, strlen(name),
 				 buf, sizeof(buf) - 1,
@@ -146,7 +219,7 @@ static void *midna_domain_to_ascii_create(const char *name, void *unused_context
 	return (mystrndup(buf, anl));
     } else {
 	msg_warn("%s: Problem translating domain \"%.100s\" to ASCII form: %s",
-		 myname, name, u_errorName(info.errors));
+		 myname, name, midna_domain_strerror(error, info.errors));
 	return (0);
     }
 }
@@ -174,7 +247,8 @@ static void *midna_domain_to_utf8_create(const char *name, void *unused_context)
     /*
      * Perform the requested conversion.
      */
-    idna = uidna_openUTS46(UIDNA_DEFAULT, &error);/* XXX check error */
+    idna = uidna_openUTS46(midna_domain_transitional ? UIDNA_DEFAULT
+			   : UIDNA_NONTRANSITIONAL_TO_UNICODE, &error);
     anl = uidna_nameToUnicodeUTF8(idna,
 				  name, strlen(name),
 				  buf, sizeof(buf) - 1,
@@ -195,7 +269,7 @@ static void *midna_domain_to_utf8_create(const char *name, void *unused_context)
 	return (mystrndup(buf, anl));
     } else {
 	msg_warn("%s: Problem translating domain \"%.100s\" to UTF8 form: %s",
-		 myname, name, u_errorName(info.errors));
+		 myname, name, midna_domain_strerror(error, info.errors));
 	return (0);
     }
 }

postfix/src/util/midna_domain.h

@@ -19,6 +19,8 @@ extern const char *midna_domain_to_utf8(const char *);
 extern const char *midna_domain_suffix_to_ascii(const char *);
 extern const char *midna_domain_suffix_to_utf8(const char *);
 
+extern int midna_domain_cache_size;
+extern int midna_domain_transitional;
 /* LICENSE
 /* .ad
 /* .fi
@@ -30,6 +32,11 @@ extern const char *midna_domain_suffix_to_utf8(const char *);
 /*	IBM T.J. Watson Research
 /*	P.O. Box 704
 /*	Yorktown Heights, NY 10598, USA
+/*
+/*	Wietse Venema
+/*	Google, Inc.
+/*	111 8th Avenue
+/*	New York, NY 10011, USA
 /*--*/
 
 #endif

postfix/src/util/midna_domain_test.ref

@@ -1,11 +1,11 @@
 ./midna_domain: > # Upper-case greek -> lower-case greek.
 ./midna_domain: > Δημοσθένους.example.com
 ./midna_domain: unconditional conversions:
-./midna_domain: "Δημοσθένους.example.com" ->utf8 "δημοσθένουσ.example.com"
+./midna_domain: "Δημοσθένους.example.com" ->utf8 "δημοσθένους.example.com"
-./midna_domain: "Δημοσθένους.example.com" ->ascii "xn--ixanjetild6aev.example.com"
+./midna_domain: "Δημοσθένους.example.com" ->ascii "xn--ixanjetild1af0a.example.com"
 ./midna_domain: conditional conversions:
-./midna_domain: "Δημοσθένους.example.com" ->ascii "xn--ixanjetild6aev.example.com" ->utf8 "δημοσθένουσ.example.com"
+./midna_domain: "Δημοσθένους.example.com" ->ascii "xn--ixanjetild1af0a.example.com" ->utf8 "δημοσθένους.example.com"
-./midna_domain: warning: "Δημοσθένους.example.com" != "δημοσθένουσ.example.com"
+./midna_domain: warning: "Δημοσθένους.example.com" != "δημοσθένους.example.com"
 ./midna_domain: > # Upper-case ASCII -> lower-case ASCII.
 ./midna_domain: > Hello.example.com
 ./midna_domain: unconditional conversions:
@@ -17,22 +17,22 @@
 ./midna_domain: > # Invalid LDH label('-' at begin or end).
 ./midna_domain: > bad-.example.com
 ./midna_domain: unconditional conversions:
-./midna_domain: warning: midna_domain_to_utf8_create: Problem translating domain "bad-.example.com" to UTF8 form: U_UNSUPPORTED_ERROR
+./midna_domain: warning: midna_domain_to_utf8_create: Problem translating domain "bad-.example.com" to UTF8 form: UIDNA_ERROR_TRAILING_HYPHEN
 ./midna_domain: "bad-.example.com" ->utf8 "(error)"
-./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "bad-.example.com" to ASCII form: U_UNSUPPORTED_ERROR
+./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "bad-.example.com" to ASCII form: UIDNA_ERROR_TRAILING_HYPHEN
 ./midna_domain: "bad-.example.com" ->ascii "(error)"
 ./midna_domain: conditional conversions:
 ./midna_domain: > -bad.example.com
 ./midna_domain: unconditional conversions:
-./midna_domain: warning: midna_domain_to_utf8_create: Problem translating domain "-bad.example.com" to UTF8 form: U_INDEX_OUTOFBOUNDS_ERROR
+./midna_domain: warning: midna_domain_to_utf8_create: Problem translating domain "-bad.example.com" to UTF8 form: UIDNA_ERROR_LEADING_HYPHEN
 ./midna_domain: "-bad.example.com" ->utf8 "(error)"
-./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "-bad.example.com" to ASCII form: U_INDEX_OUTOFBOUNDS_ERROR
+./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "-bad.example.com" to ASCII form: UIDNA_ERROR_LEADING_HYPHEN
 ./midna_domain: "-bad.example.com" ->ascii "(error)"
 ./midna_domain: conditional conversions:
 ./midna_domain: > # Invalid LDH (label > 63 bytes).
 ./midna_domain: > abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789.example.com
 ./midna_domain: unconditional conversions:
-./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789.example.com" to ASCII form: U_MISSING_RESOURCE_ERROR
+./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789.example.com" to ASCII form: UIDNA_ERROR_LABEL_TOO_LONG
 ./midna_domain: "abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789.example.com" ->utf8 "(error)"
 ./midna_domain: "abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789.example.com" ->ascii "(error)"
 ./midna_domain: conditional conversions:
@@ -46,7 +46,7 @@
 ./midna_domain: > # Invalid name (length > 255 bytes).
 ./midna_domain: > abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.example.com
 ./midna_domain: unconditional conversions:
-./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcde" to ASCII form: U_FILE_ACCESS_ERROR
+./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcde" to ASCII form: UIDNA_ERROR_DOMAIN_NAME_TOO_LONG
 ./midna_domain: "abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.example.com" ->utf8 "(error)"
 ./midna_domain: "abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567.example.com" ->ascii "(error)"
 ./midna_domain: conditional conversions:
@@ -82,8 +82,8 @@
 ./midna_domain: > # Bad a-label.
 ./midna_domain: > xn--123456.example.com
 ./midna_domain: unconditional conversions:
-./midna_domain: warning: midna_domain_to_utf8_create: Problem translating domain "xn--123456.example.com" to UTF8 form: [BOGUS UErrorCode]
+./midna_domain: warning: midna_domain_to_utf8_create: Problem translating domain "xn--123456.example.com" to UTF8 form: UIDNA_ERROR_PUNYCODE
 ./midna_domain: "xn--123456.example.com" ->utf8 "(error)"
-./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "xn--123456.example.com" to ASCII form: [BOGUS UErrorCode]
+./midna_domain: warning: midna_domain_to_ascii_create: Problem translating domain "xn--123456.example.com" to ASCII form: UIDNA_ERROR_PUNYCODE
 ./midna_domain: "xn--123456.example.com" ->ascii "(error)"
 ./midna_domain: conditional conversions:

postfix/src/verify/verify.c

@@ -155,6 +155,12 @@
 /* .IP "\fBsmtputf8_autodetect_classes (sendmail, verify)\fR"
 /*	Detect that a message requires SMTPUTF8 support for the specified
 /*	mail origin classes.
+/* .PP
+/*	Available in Postfix version 3.2 and later:
+/* .IP "\fBenable_idna2003_compatibility (no)\fR"
+/*	Enable 'transitional' compatibility between IDNA2003 and IDNA2008,
+/*	when converting UTF-8 domain names to/from the ASCII form that is
+/*	used for DNS lookups.
 /* MISCELLANEOUS CONTROLS
 /* .ad
 /* .fi