mir/postfix
2
0
Fork 0
mirror of https://github.com/vdukhovni/postfix synced 2025-08-31 14:17:41 +00:00
Code Issues Releases Wiki Activity

postfix-2.9-20111213

Browse Source
This commit is contained in:
Wietse Venema
2011-12-13 00:00:00 -05:00
committed by Viktor Dukhovni
parent c538d673ac
commit 9b2bef6acf
23 changed files with 1043 additions and 577 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
postfix/HISTORY
View File

@@ -17283,3 +17283,22 @@ Apologies for any names omitted.
src/postlog/postlog.c, src/postmap/postmap.c, src/postlog/postlog.c, src/postmap/postmap.c,
src/postmulti/postmulti.c, src/postqueue/postqueue.c, src/postmulti/postmulti.c, src/postqueue/postqueue.c,
src/postsuper/postsuper.c, src/sendmail/sendmail.c. src/postsuper/postsuper.c, src/sendmail/sendmail.c.
20111211
Feature: first/next (sequence) support in the proxymap
protocol. This is needed for cache cleanup of a proxied
postscreen or verify persistent cache. Files:
global/dict_proxy.[hc], proxymap/proxymap.c.
Feature: memcache client support without libmemcache
dependencies. Files: global/memcache_proto.[hc],
global/dict_memcache.c.
Feature: support for persistent backup database in the
memcache client. The database can be shared with the proxymap
service, but it needs to be listed as "proxy:maptype:mapname"
in the proxy_read_maps or proxy_write_maps parameter value
(depending on whether the access is read-only or read-write).
Support for proxymap-over-tcp (proxy:maptype:mapname@host:port)
is under development. File: global/dict_memcache.c.

75
postfix/README_FILES/MEMCACHE_README
View File

@@ -4,63 +4,36 @@ PPoossttffiixx mmeemmccaacchhee cclliieenntt HHoowwtto
IInnttrroodduuccttiioonn IInnttrroodduuccttiioonn
The Postfix memcache client type allows you to hook up Postfix to a memcache The Postfix memcache client allows you to hook up Postfix to a memcache server.
server. This implementation supports multiple memcache servers for redundancy, The current implementation supports one memcache server per Postfix table, with
and multiple memcache clients that you can use for different table lookups. The one optional Postfix database that provides persistent backup. The Postfix
Postfix memcache client supports both lookup and update operations. memcache client supports the lookup, update, delete and sequence operations.
The sequence (i.e. first/next) operation requires a backup database that
supports this operation.
Typically, a memcache map is used to reduce query load on a database server, or Typically, the Postfix memcache client is used to reduce query load on a
to share a low-latency database among different Postfix instances. persistent database, but it may also be used to query a memory-only database
for low-value, easy-to-create, information such as a reputation cache for
postscreen(8), verify(8) or greylisting.
LLiimmiittaattiioonnss LLiimmiittaattiioonnss
* The Postfix memcache client is based on libmemcache, which will terminate
its process after a memcache server goes down. To avoid this, set up
redundant memcache servers that have no common source of failure.
* The Postfix memcache client cannot be used for security-sensitive tables * The Postfix memcache client cannot be used for security-sensitive tables
such as alias_maps (these may contain "|command" and "/file/name" such as alias_maps (these may contain "|command" and "/file/name"
destinations), or virtual_uid_maps and virtual_gid_maps (these specify UNIX destinations), or virtual_uid_maps, virtual_gid_maps and
process privileges). Typically, a memcache database is shared via a TCP virtual_mailbox_maps (these specify UNIX process privileges or "/file/name"
socket, and is writable not only by Postfix, but by any process that can destinations). Typically, a memcache database is writable by any process
talk to the memcache server. that can talk to the memcache server; in contrast, security-sensitive
tables must not be writable by the unprivileged Postfix user.
* The Postfix memcache client requires additional configuration when used * The Postfix memcache client requires additional configuration when used as
with the postscreen(8) and verify(8) daemons. For details see the ttl postscreen(8) or verify(8) cache. For details see the backup and ttl
parameter discussion in the memcache_table(5) manual page. parameter discussions in the memcache_table(5) manual page.
* The Postfix memcache client is supported only with libmemcache version
1.4.0. Some libmemcache features are documented by reading libmemcache
source code, instead of a proper API.
BBuuiillddiinngg PPoossttffiixx wwiitthh mmeemmccaacchhee ssuuppppoorrtt BBuuiillddiinngg PPoossttffiixx wwiitthh mmeemmccaacchhee ssuuppppoorrtt
To build Postfix with memcache client support, specify -DHAS_MEMCACHE, the The Postfix memcache client has no external dependencies, and is therefore
location of the libmemcache include files, and the location of the libmemcache built into Postfix by default.
object library.
For example:
% make -f Makefile.init makefiles \
'CCARGS=-DHAS_MEMCACHE -I/usr/local/include' \
'AUXLIBS=-L/usr/local/lib -lmemcache'
Then run 'make'.
If the build fails with "undefined reference to `mcm_buf_len'" (and with a
similar error message for mcm_buf_remain_off), then you need to edit
libmemcache source code.
The following instructions apply to libmemcache 1.4.0.rc2.
* Open the libmemcache source file include/memcache/buffer.h.
* Delete the "inline" words before the functions that were reported in the
"undefined reference" error messages.
* Recompile and reinstall libmemcache.
Then, continue building Postfix by running 'make'.
CCoonnffiigguurriinngg mmeemmccaacchhee llooookkuupp ttaabblleess CCoonnffiigguurriinngg mmeemmccaacchhee llooookkuupp ttaabblleess
@@ -68,8 +41,10 @@ Configuration is described in the memcache_table(5) manpage.
CCrreeddiittss CCrreeddiittss
The first memcache client for Postfix was written by Omar Kilani. The first memcache client for Postfix was written by Omar Kilani, and was based
on the libmemcache library.
Wietse wrote a new memcache client from the ground up. Besides also using Wietse wrote the current memcache client from the ground up. This
libmemcache, the current implementation bears no resemblance to Omar's work. implementation does not use libmemcache, and bears no resemblance to earlier
work.

27
postfix/RELEASE_NOTES
View File

@@ -14,6 +14,33 @@ specifies the release date of a stable release or snapshot release.
If you upgrade from Postfix 2.7 or earlier, read RELEASE_NOTES-2.8 If you upgrade from Postfix 2.7 or earlier, read RELEASE_NOTES-2.8
before proceeding. before proceeding.
Major changes with snapshot 20111213
====================================
Support for a persistent backup database in the memcache client.
The memcache client updates the memcache whenever it looks up or
modifies information in the persistent database.
The persistent database can be shared with the proxymap service,
but it needs to be listed as "proxy:maptype:mapname" in the
proxy_read_maps or proxy_write_maps parameter value (depending on
whether the access is read-only or read-write).
Support for proxymap-over-tcp (proxy:maptype:mapname@host:port) is
under development.
Elimination of dependencies on the libmemcache library. Postfix
memcache support is now compiled in by default.
Major changes with snapshot 20111209
====================================
memcache lookup and update support. This provides a way to share
postscreen(8) or verify(8) caches between Postfix instances. The
Postfix memcache client can't be used for security-sensitive
information, and it supports only libmemcache version 1.4.0. See
MEMCACHE_README and memcache_table(5) for details and limitations.
Incompatible changes with snapshot 20111205 Incompatible changes with snapshot 20111205
=========================================== ===========================================

3
postfix/WISHLIST
View File

@@ -6,6 +6,9 @@ Wish list:
or require that they reset dict_errno on entry, either exit or require that they reset dict_errno on entry, either exit
with a fatal error or set dict_errno on error. with a fatal error or set dict_errno on error.
dict_memcache: treat "bad" key as cache miss, i.e. read/write
the database as if the cache did not exist.
Is it possible to replace msg_fatal calls in match_ops.c Is it possible to replace msg_fatal calls in match_ops.c
by msg_warn and longjmp? The callers will have to specify by msg_warn and longjmp? The callers will have to specify
if they want the code to return instead of terminate. if they want the code to return instead of terminate.

93
postfix/html/MEMCACHE_README.html
View File

@@ -19,82 +19,44 @@
<h2>Introduction</h2> <h2>Introduction</h2>
<p>The Postfix memcache client type allows you to hook up Postfix to <p>The Postfix memcache client allows you to hook up Postfix to a
a memcache server. This implementation supports multiple memcache memcache server. The current implementation supports one memcache
servers for redundancy, and multiple memcache clients that you can server per Postfix table, with one optional Postfix database that
use for different table lookups. The Postfix memcache client provides persistent backup. The Postfix memcache client supports
supports both lookup and update operations. </p> the lookup, update, delete and sequence operations. The sequence
(i.e. first/next) operation requires a backup database that supports
this operation. </p>
<p> Typically, a memcache map is used to reduce query load on a <p> Typically, the Postfix memcache client is used to reduce query
database server, or to share a low-latency database among different load on a persistent database, but it may also be used to query a
Postfix instances. </p> memory-only database for low-value, easy-to-create, information
such as a reputation cache for <a href="postscreen.8.html">postscreen(8)</a>, <a href="verify.8.html">verify(8)</a> or greylisting.
</p>
<h2>Limitations</h2> <h2>Limitations</h2>
<ul> <ul>
<li> <p> The Postfix memcache client is based on libmemcache, which
will terminate its process after a memcache server goes down. To
avoid this, set up redundant memcache servers that have no common
source of failure. </p>
<li> <p> The Postfix memcache client cannot be used for security-sensitive <li> <p> The Postfix memcache client cannot be used for security-sensitive
tables such as <tt><a href="postconf.5.html#alias_maps">alias_maps</a></tt> (these may contain "<tt>|command</tt>" tables such as <tt><a href="postconf.5.html#alias_maps">alias_maps</a></tt> (these may contain "<tt>|command</tt>"
and "<tt>/file/name</tt>" destinations), or <tt><a href="postconf.5.html#virtual_uid_maps">virtual_uid_maps</a></tt> and "<tt>/file/name</tt>" destinations), or <tt><a href="postconf.5.html#virtual_uid_maps">virtual_uid_maps</a></tt>,
and <tt><a href="postconf.5.html#virtual_gid_maps">virtual_gid_maps</a></tt> (these specify UNIX process privileges). <tt><a href="postconf.5.html#virtual_gid_maps">virtual_gid_maps</a></tt> and <tt><a href="postconf.5.html#virtual_mailbox_maps">virtual_mailbox_maps</a></tt> (these
Typically, a memcache database is shared via a TCP socket, and is specify UNIX process privileges or "<tt>/file/name</tt>" destinations).
writable not only by Postfix, but by any process that can talk to Typically, a memcache database is writable by any process that can
the memcache server. </p> talk to the memcache server; in contrast, security-sensitive tables
must not be writable by the unprivileged Postfix user. </p>
<li> <p> The Postfix memcache client requires additional configuration <li> <p> The Postfix memcache client requires additional configuration
when used with the <a href="postscreen.8.html">postscreen(8)</a> and <a href="verify.8.html">verify(8)</a> daemons. For details when used as <a href="postscreen.8.html">postscreen(8)</a> or <a href="verify.8.html">verify(8)</a> cache. For details see the
see the <tt>ttl</tt> parameter discussion in the <a href="memcache_table.5.html">memcache_table(5)</a> <tt>backup</tt> and <tt>ttl</tt> parameter discussions in the
manual page. </p> <a href="memcache_table.5.html">memcache_table(5)</a> manual page. </p>
<li> <p> The Postfix memcache client is supported only with libmemcache
version 1.4.0. Some libmemcache features are documented by reading
libmemcache source code, instead of a proper API. </p>
</ul> </ul>
<h2>Building Postfix with memcache support</h2> <h2>Building Postfix with memcache support</h2>
<p>To build Postfix with memcache client support, specify <p>The Postfix memcache client has no external dependencies,
<tt>-DHAS_MEMCACHE</tt>, the location of the libmemcache include and is therefore built into Postfix by default. </p>
files, and the location of the libmemcache object library. </p>
<p> For example: </p>
<blockquote>
<pre>
% make -f Makefile.init makefiles \
'CCARGS=-DHAS_MEMCACHE -I/usr/local/include' \
'AUXLIBS=-L/usr/local/lib -lmemcache'
</pre>
</blockquote>
<p> Then run 'make'. </p>
<p> If the build fails with "<tt>undefined reference to `mcm_buf_len'</tt>"
(and with a similar error message for <tt>mcm_buf_remain_off</tt>),
then you need to edit libmemcache source code. </p>
<p> The following instructions apply to libmemcache 1.4.0.rc2. </p>
<ul>
<li> <p> Open the libmemcache source file
<tt>include/memcache/buffer.h</tt>. </p>
<li> <p> Delete the "<tt>inline</tt>" words before the functions
that were reported in the "<tt>undefined reference</tt>" error
messages. </p>
<li> <p> Recompile and reinstall libmemcache. </p>
</ul>
<p> Then, continue building Postfix by running 'make'. </p>
<h2>Configuring memcache lookup tables</h2> <h2>Configuring memcache lookup tables</h2>
@@ -102,11 +64,12 @@ messages. </p>
<h2>Credits</h2> <h2>Credits</h2>
<p> The first memcache client for Postfix was written by Omar Kilani. </p> <p> The first memcache client for Postfix was written by Omar Kilani,
and was based on the libmemcache library. </p>
<p> Wietse wrote a new memcache client from the ground up. Besides <p> Wietse wrote the current memcache client from the ground up.
also using libmemcache, the current implementation bears no resemblance This implementation does not use libmemcache, and bears no resemblance
to Omar's work. </p> to earlier work. </p>
</body> </body>

108
postfix/html/memcache_table.5.html
View File

@@ -20,8 +20,8 @@ MEMCACHE_TABLE(5) MEMCACHE_TABLE(5)
or <b>db</b> format. or <b>db</b> format.
Alternatively, lookup tables can be specified as memcache Alternatively, lookup tables can be specified as memcache
instances. In order to use memcache lookups, define a instances. To use memcache lookups, define a memcache
memcache source as a lookup table in <a href="postconf.5.html">main.cf</a>, for example: source as a lookup table in <a href="postconf.5.html">main.cf</a>, for example:
<a href="postconf.5.html#virtual_alias_maps">virtual_alias_maps</a> = <a href="memcache_table.5.html">memcache</a>:/etc/postfix/memcache-aliases.cf <a href="postconf.5.html#virtual_alias_maps">virtual_alias_maps</a> = <a href="memcache_table.5.html">memcache</a>:/etc/postfix/memcache-aliases.cf
@@ -29,18 +29,51 @@ MEMCACHE_TABLE(5) MEMCACHE_TABLE(5)
format as the Postfix <a href="postconf.5.html">main.cf</a> file, and specifies the format as the Postfix <a href="postconf.5.html">main.cf</a> file, and specifies the
parameters described below. parameters described below.
The Postfix memcache client supports the lookup and update The Postfix memcache client supports the lookup, update,
operations. delete and sequence (first/next) operations. The sequence
operation requires a backup database that supports the
operation.
<b>MEMCACHE PARAMETERS</b> <b>MEMCACHE PARAMETERS</b>
<b>hosts (default: localhost:11211)</b> <b>backup</b> An optional Postfix database that provides persis-
The memcache servers that Postfix will try to con- tent backup for the memcache database. The Postfix
nect to. Specify a hostname or address, optionally memcache client will update the memcache database
followed by ":" and a port name or number. The whenever it looks up or changes information in the
default port is 11211. Examples: persistent database. Specify a Postfix "<a href="DATABASE_README.html">type:table</a>"
database. Example:
hosts = memcache01.example.com backup = btree:/var/lib/postfix/<a href="postconf.5.html#postscreen_cache_map">postscreen_cache_map</a>
memcache02.example.com
Access to remote proxymap servers is under develop-
ment.
NOTE 1: When using memcache with persistent backup
as <a href="postscreen.8.html"><b>postscreen</b>(8)</a> or <a href="verify.8.html"><b>verify</b>(8)</a> cache, disable auto-
matic cache cleanup (*_cache_cleanup_interval = 0)
in all Postfix instances except for one instance
that will be responsible for cache cleanup.
NOTE 2: In the case of a proxied database, the full
database name (including the "<a href="proxymap.8.html">proxy</a>:" prefix) must
be specified in the proxymap server's
<a href="postconf.5.html#proxy_read_maps">proxy_read_maps</a> or <a href="postconf.5.html#proxy_write_maps">proxy_write_maps</a> setting
(depending on whether the access is read-only or
read-write).
<b>memcache (default: inet:localhost:11211)</b>
The memcache server (note: singular) that Postfix
will try to connect to. For a TCP server specify
"inet:" followed by a hostname or address, ":", and
a port name or number. For a UNIX-domain server
specify "unix:" followed by the socket pathname.
Examples:
memcache = inet:memcache.example.com
memcache = unix:/path/to/socket
NOTE: In the case of a UNIX-domain socket, it must
be accessible by the unprivileged postfix user and
by the memcached process.
<b>key_format (default: %s)</b> <b>key_format (default: %s)</b>
Format of the lookup and update keys in memcache Format of the lookup and update keys in memcache
@@ -111,45 +144,37 @@ MEMCACHE_TABLE(5) MEMCACHE_TABLE(5)
Optional flags that should be stored along with a Optional flags that should be stored along with a
memcache update. memcache update.
<b>ttl (default: 604800)</b> <b>ttl (default: 3600)</b>
The expiration time in seconds of memcache updates. The expiration time in seconds of memcache updates.
The default is one week.
When using memcache tables with <a href="postscreen.8.html"><b>postscreen</b>(8)</a> or NOTE 1: When using a memcache table as
<a href="verify.8.html"><b>verify</b>(8)</a>, specify a zero *_cache_cleanup_interval <a href="postscreen.8.html"><b>postscreen</b>(8)</a> or <a href="verify.8.html"><b>verify</b>(8)</a> cache without persistent
value, and specify the largest <a href="postscreen.8.html"><b>postscreen</b>(8)</a> *_ttl backup, specify a zero *_cache_cleanup_interval
value with all Postfix instances that use the mem-
cache, and specify the largest <a href="postscreen.8.html"><b>postscreen</b>(8)</a> *_ttl
value or <a href="verify.8.html"><b>verify</b>(8)</a> *_expire_time value as the mem- value or <a href="verify.8.html"><b>verify</b>(8)</a> *_expire_time value as the mem-
cache map's <b>ttl</b> value. cache table's <b>ttl</b> value.
Note: according to memcache protocol documentation, NOTE 2: According to memcache protocol documenta-
a value greater than 30 days (2592000 seconds) tion, a value greater than 30 days (2592000 sec-
specifies absolute UNIX time. Smaller values are onds) specifies absolute UNIX time. Smaller values
relative to the time of the update. are relative to the time of the update.
<b>BUGS</b> <b>BUGS</b>
The Postfix memcache client is based on libmemcache, which
will terminate its process after a memcache server goes
down. To avoid this, set up redundant memcache servers
that have no common source of failure.
The Postfix memcache client cannot be used for security- The Postfix memcache client cannot be used for security-
sensitive tables such as <b><a href="postconf.5.html#alias_maps">alias_maps</a></b> (these may contain sensitive tables such as <b><a href="postconf.5.html#alias_maps">alias_maps</a></b> (these may contain
"<i>|command</i> and "<i>/file/name</i>" destinations), or <b><a href="postconf.5.html#virtual_uid_maps">vir</a>-</b> "<i>|command</i> and "<i>/file/name</i>" destinations), or <b><a href="postconf.5.html#virtual_uid_maps">vir</a>-</b>
<b><a href="postconf.5.html#virtual_uid_maps">tual_uid_maps</a></b> and <b><a href="postconf.5.html#virtual_gid_maps">virtual_gid_maps</a></b> (these specify UNIX <b><a href="postconf.5.html#virtual_uid_maps">tual_uid_maps</a></b>, <b><a href="postconf.5.html#virtual_gid_maps">virtual_gid_maps</a></b> and <b><a href="postconf.5.html#virtual_mailbox_maps">virtual_mailbox_maps</a></b>
process privileges). In a typical deployment a memcache (these specify UNIX process privileges or "<i>/file/name</i>"
database is shared via a TCP socket, and is therefore destinations). In a typical deployment a memcache data-
writable not only by Postfix, but by any process that can base is writable by any process that can talk to the mem-
talk to the memcache server. cache server; in contrast, security-sensitive tables must
not be writable by the unprivileged Postfix user.
The Postfix memcache client requires additional configura- The Postfix memcache client requires additional configura-
tion when used with the <a href="postscreen.8.html"><b>postscreen</b>(8)</a> and <a href="verify.8.html"><b>verify</b>(8)</a> dae- tion when used as <a href="postscreen.8.html"><b>postscreen</b>(8)</a> or <a href="verify.8.html"><b>verify</b>(8)</a> cache. For
mons. For details see the <b>ttl</b> parameter discussion at the details see the <b>backup</b> and <b>ttl</b> parameter discussions in
end of the MEMCACHE PARAMETERS section in this document. the MEMCACHE PARAMETERS section above.
The Postfix memcache client is supported only with libmem-
cache version 1.4.0. Some libmemcache features are docu-
mented by reading libmemcache source code, instead a
proper API.
<b>SEE ALSO</b> <b>SEE ALSO</b>
<a href="postmap.1.html">postmap(1)</a>, Postfix lookup table manager <a href="postmap.1.html">postmap(1)</a>, Postfix lookup table manager
@@ -165,8 +190,9 @@ MEMCACHE_TABLE(5) MEMCACHE_TABLE(5)
<b>HISTORY</b> <b>HISTORY</b>
The first memcache client for Postfix was written by Omar The first memcache client for Postfix was written by Omar
Kilani. Besides being implemented on libmemcache, this Kilani, and was based on libmemcache. The Postfix imple-
implementation bears no resemblance to his work. mentation does not use libmemcache, and bears no resem-
blance to earlier work.
<b>AUTHOR(S)</b> <b>AUTHOR(S)</b>
Wietse Venema Wietse Venema

8
postfix/html/proxymap.8.html
View File

@@ -79,6 +79,14 @@ PROXYMAP(8) PROXYMAP(8)
This request is supported in Postfix 2.5 and later. This request is supported in Postfix 2.5 and later.
<b>sequence</b> <i>maptype:mapname flags function</i>
Iterate over the specified database. The <i>function</i>
is one of DICT_SEQ_FUN_FIRST or DICT_SEQ_FUN_NEXT.
The reply is the request completion status code and
a lookup key and result value, if found.
This request is supported in Postfix 2.9 and later.
The request completion status is one of OK, RETRY, NOKEY The request completion status is one of OK, RETRY, NOKEY
(lookup failed because the key was not found), BAD (mal- (lookup failed because the key was not found), BAD (mal-
formed request) or DENY (the table is not approved for formed request) or DENY (the table is not approved for

104
postfix/man/man5/memcache_table.5
View File

@@ -19,8 +19,8 @@ rewriting or mail routing. These tables are usually in
\fBdbm\fR or \fBdb\fR format. \fBdbm\fR or \fBdb\fR format.
Alternatively, lookup tables can be specified as memcache Alternatively, lookup tables can be specified as memcache
instances. In order to use memcache lookups, define a instances. To use memcache lookups, define a memcache
memcache source as a lookup table in main.cf, for example: source as a lookup table in main.cf, for example:
.nf .nf
virtual_alias_maps = memcache:/etc/postfix/memcache-aliases.cf virtual_alias_maps = memcache:/etc/postfix/memcache-aliases.cf
@@ -30,23 +30,53 @@ The file /etc/postfix/memcache-aliases.cf has the same
format as the Postfix main.cf file, and specifies the format as the Postfix main.cf file, and specifies the
parameters described below. parameters described below.
The Postfix memcache client supports the lookup and update The Postfix memcache client supports the lookup, update,
operations. delete and sequence (first/next) operations. The sequence
operation requires a backup database that supports the
operation.
.SH "MEMCACHE PARAMETERS" .SH "MEMCACHE PARAMETERS"
.na .na
.nf .nf
.ad .ad
.fi .fi
.IP "\fBhosts (default: localhost:11211)\fR" .IP \fBbackup\fR
The memcache servers that Postfix will try to connect to. An optional Postfix database that provides persistent backup
Specify a hostname or address, optionally followed by ":" for the memcache database. The Postfix memcache client will
and a port name or number. The default port is 11211. update the memcache database whenever it looks up or changes
Examples: information in the persistent database. Specify a Postfix
"type:table" database. Example:
.nf .nf
hosts = memcache01.example.com backup = btree:/var/lib/postfix/postscreen_cache_map
memcache02.example.com
.fi .fi
Access to remote proxymap servers is under development.
NOTE 1: When using memcache with persistent backup as
\fBpostscreen\fR(8) or \fBverify\fR(8) cache, disable
automatic cache cleanup (*_cache_cleanup_interval = 0) in
all Postfix instances except for one instance that will be
responsible for cache cleanup.
NOTE 2: In the case of a proxied database, the full database
name (including the "proxy:" prefix) must be specified in
the proxymap server's proxy_read_maps or proxy_write_maps
setting (depending on whether the access is read-only or
read-write).
.IP "\fBmemcache (default: inet:localhost:11211)\fR"
The memcache server (note: singular) that Postfix will try
to connect to. For a TCP server specify "inet:" followed by
a hostname or address, ":", and a port name or number.
For a UNIX-domain server specify "unix:" followed by the
socket pathname. Examples:
.nf
memcache = inet:memcache.example.com
memcache = unix:/path/to/socket
.fi
NOTE: In the case of a UNIX-domain socket, it must be accessible
by the unprivileged postfix user and by the memcached process.
.IP "\fBkey_format (default: %s)\fB" .IP "\fBkey_format (default: %s)\fB"
Format of the lookup and update keys in memcache queries. Format of the lookup and update keys in memcache queries.
By default, these are the same as the lookup and update By default, these are the same as the lookup and update
@@ -110,45 +140,38 @@ are skipped with a warning). Example:
.IP "\fBflags (default: 0)\fR" .IP "\fBflags (default: 0)\fR"
Optional flags that should be stored along with a memcache Optional flags that should be stored along with a memcache
update. update.
.IP "\fBttl (default: 604800)\fR" .IP "\fBttl (default: 3600)\fR"
The expiration time in seconds of memcache updates. The expiration time in seconds of memcache updates.
The default is one week.
When using memcache tables with \fBpostscreen\fR(8) or NOTE 1: When using a memcache table as \fBpostscreen\fR(8)
\fBverify\fR(8), specify a zero *_cache_cleanup_interval or \fBverify\fR(8) cache without persistent backup, specify
value, and specify the largest \fBpostscreen\fR(8) *_ttl a zero *_cache_cleanup_interval value with all Postfix
value or \fBverify\fR(8) *_expire_time value as the memcache instances that use the memcache, and specify the largest
map's \fBttl\fR value. \fBpostscreen\fR(8) *_ttl value or \fBverify\fR(8) *_expire_time
value as the memcache table's \fBttl\fR value.
Note: according to memcache protocol documentation, a value NOTE 2: According to memcache protocol documentation, a
greater than 30 days (2592000 seconds) specifies absolute UNIX value greater than 30 days (2592000 seconds) specifies
absolute UNIX
time. Smaller values are relative to the time of the update. time. Smaller values are relative to the time of the update.
.SH BUGS .SH BUGS
.ad .ad
.fi .fi
The Postfix memcache client is based on libmemcache, which
will terminate its process after a memcache server goes
down. To avoid this, set up redundant memcache servers that
have no common source of failure.
The Postfix memcache client cannot be used for security-sensitive The Postfix memcache client cannot be used for security-sensitive
tables such as \fBalias_maps\fR (these may contain tables such as \fBalias_maps\fR (these may contain
"\fI|command\fR and "\fI/file/name\fR" destinations), or "\fI|command\fR and "\fI/file/name\fR" destinations), or
\fBvirtual_uid_maps\fR and \fBvirtual_gid_maps\fR (these \fBvirtual_uid_maps\fR, \fBvirtual_gid_maps\fR and
specify UNIX process privileges). In a typical deployment \fBvirtual_mailbox_maps\fR (these specify UNIX process
a memcache database is shared via a TCP socket, and is privileges or "\fI/file/name\fR" destinations). In a typical
therefore writable not only by Postfix, but by any process deployment a memcache database is writable by any process
that can talk to the memcache server. that can talk to the memcache server; in contrast,
security-sensitive tables must not be writable by the
unprivileged Postfix user.
The Postfix memcache client requires additional configuration The Postfix memcache client requires additional configuration
when used with the \fBpostscreen\fR(8) and \fBverify\fR(8) when used as \fBpostscreen\fR(8) or \fBverify\fR(8) cache.
daemons. For details see the \fBttl\fR parameter discussion For details see the \fBbackup\fR and \fBttl\fR parameter
at the end of the MEMCACHE PARAMETERS section in this discussions in the MEMCACHE PARAMETERS section above.
document.
The Postfix memcache client is supported only with libmemcache
version 1.4.0. Some libmemcache features are documented
by reading libmemcache source code, instead a proper API.
.SH "SEE ALSO" .SH "SEE ALSO"
.na .na
.nf .nf
@@ -177,8 +200,9 @@ The Secure Mailer license must be distributed with this software.
.ad .ad
.fi .fi
The first memcache client for Postfix was written by Omar The first memcache client for Postfix was written by Omar
Kilani. Besides being implemented on libmemcache, this Kilani, and was based on libmemcache.
implementation bears no resemblance to his work. The Postfix implementation does not use libmemcache, and
bears no resemblance to earlier work.
.SH "AUTHOR(S)" .SH "AUTHOR(S)"
.na .na
.nf .nf

7
postfix/man/man8/proxymap.8
View File

@@ -75,6 +75,13 @@ The \fImaptype:mapname\fR and \fIflags\fR are the same
as with the \fBopen\fR request. as with the \fBopen\fR request.
.sp .sp
This request is supported in Postfix 2.5 and later. This request is supported in Postfix 2.5 and later.
.IP "\fBsequence\fR \fImaptype:mapname flags function\fR"
Iterate over the specified database. The \fIfunction\fR
is one of DICT_SEQ_FUN_FIRST or DICT_SEQ_FUN_NEXT.
The reply is the request completion status code and
a lookup key and result value, if found.
.sp
This request is supported in Postfix 2.9 and later.
.PP .PP
The request completion status is one of OK, RETRY, NOKEY The request completion status is one of OK, RETRY, NOKEY
(lookup failed because the key was not found), BAD (malformed (lookup failed because the key was not found), BAD (malformed

93
postfix/proto/MEMCACHE_README.html
View File

@@ -19,82 +19,44 @@
<h2>Introduction</h2> <h2>Introduction</h2>
<p>The Postfix memcache client type allows you to hook up Postfix to <p>The Postfix memcache client allows you to hook up Postfix to a
a memcache server. This implementation supports multiple memcache memcache server. The current implementation supports one memcache
servers for redundancy, and multiple memcache clients that you can server per Postfix table, with one optional Postfix database that
use for different table lookups. The Postfix memcache client provides persistent backup. The Postfix memcache client supports
supports both lookup and update operations. </p> the lookup, update, delete and sequence operations. The sequence
(i.e. first/next) operation requires a backup database that supports
this operation. </p>
<p> Typically, a memcache map is used to reduce query load on a <p> Typically, the Postfix memcache client is used to reduce query
database server, or to share a low-latency database among different load on a persistent database, but it may also be used to query a
Postfix instances. </p> memory-only database for low-value, easy-to-create, information
such as a reputation cache for postscreen(8), verify(8) or greylisting.
</p>
<h2>Limitations</h2> <h2>Limitations</h2>
<ul> <ul>
<li> <p> The Postfix memcache client is based on libmemcache, which
will terminate its process after a memcache server goes down. To
avoid this, set up redundant memcache servers that have no common
source of failure. </p>
<li> <p> The Postfix memcache client cannot be used for security-sensitive <li> <p> The Postfix memcache client cannot be used for security-sensitive
tables such as <tt>alias_maps</tt> (these may contain "<tt>|command</tt>" tables such as <tt>alias_maps</tt> (these may contain "<tt>|command</tt>"
and "<tt>/file/name</tt>" destinations), or <tt>virtual_uid_maps</tt> and "<tt>/file/name</tt>" destinations), or <tt>virtual_uid_maps</tt>,
and <tt>virtual_gid_maps</tt> (these specify UNIX process privileges). <tt>virtual_gid_maps</tt> and <tt>virtual_mailbox_maps</tt> (these
Typically, a memcache database is shared via a TCP socket, and is specify UNIX process privileges or "<tt>/file/name</tt>" destinations).
writable not only by Postfix, but by any process that can talk to Typically, a memcache database is writable by any process that can
the memcache server. </p> talk to the memcache server; in contrast, security-sensitive tables
must not be writable by the unprivileged Postfix user. </p>
<li> <p> The Postfix memcache client requires additional configuration <li> <p> The Postfix memcache client requires additional configuration
when used with the postscreen(8) and verify(8) daemons. For details when used as postscreen(8) or verify(8) cache. For details see the
see the <tt>ttl</tt> parameter discussion in the memcache_table(5) <tt>backup</tt> and <tt>ttl</tt> parameter discussions in the
manual page. </p> memcache_table(5) manual page. </p>
<li> <p> The Postfix memcache client is supported only with libmemcache
version 1.4.0. Some libmemcache features are documented by reading
libmemcache source code, instead of a proper API. </p>
</ul> </ul>
<h2>Building Postfix with memcache support</h2> <h2>Building Postfix with memcache support</h2>
<p>To build Postfix with memcache client support, specify <p>The Postfix memcache client has no external dependencies,
<tt>-DHAS_MEMCACHE</tt>, the location of the libmemcache include and is therefore built into Postfix by default. </p>
files, and the location of the libmemcache object library. </p>
<p> For example: </p>
<blockquote>
<pre>
% make -f Makefile.init makefiles \
'CCARGS=-DHAS_MEMCACHE -I/usr/local/include' \
'AUXLIBS=-L/usr/local/lib -lmemcache'
</pre>
</blockquote>
<p> Then run 'make'. </p>
<p> If the build fails with "<tt>undefined reference to `mcm_buf_len'</tt>"
(and with a similar error message for <tt>mcm_buf_remain_off</tt>),
then you need to edit libmemcache source code. </p>
<p> The following instructions apply to libmemcache 1.4.0.rc2. </p>
<ul>
<li> <p> Open the libmemcache source file
<tt>include/memcache/buffer.h</tt>. </p>
<li> <p> Delete the "<tt>inline</tt>" words before the functions
that were reported in the "<tt>undefined reference</tt>" error
messages. </p>
<li> <p> Recompile and reinstall libmemcache. </p>
</ul>
<p> Then, continue building Postfix by running 'make'. </p>
<h2>Configuring memcache lookup tables</h2> <h2>Configuring memcache lookup tables</h2>
@@ -102,11 +64,12 @@ messages. </p>
<h2>Credits</h2> <h2>Credits</h2>
<p> The first memcache client for Postfix was written by Omar Kilani. </p> <p> The first memcache client for Postfix was written by Omar Kilani,
and was based on the libmemcache library. </p>
<p> Wietse wrote a new memcache client from the ground up. Besides <p> Wietse wrote the current memcache client from the ground up.
also using libmemcache, the current implementation bears no resemblance This implementation does not use libmemcache, and bears no resemblance
to Omar's work. </p> to earlier work. </p>
</body> </body>

104
postfix/proto/memcache_table
View File

@@ -13,8 +13,8 @@
# \fBdbm\fR or \fBdb\fR format. # \fBdbm\fR or \fBdb\fR format.
# #
# Alternatively, lookup tables can be specified as memcache # Alternatively, lookup tables can be specified as memcache
# instances. In order to use memcache lookups, define a # instances. To use memcache lookups, define a memcache
# memcache source as a lookup table in main.cf, for example: # source as a lookup table in main.cf, for example:
# #
# .nf # .nf
# virtual_alias_maps = memcache:/etc/postfix/memcache-aliases.cf # virtual_alias_maps = memcache:/etc/postfix/memcache-aliases.cf
@@ -24,21 +24,51 @@
# format as the Postfix main.cf file, and specifies the # format as the Postfix main.cf file, and specifies the
# parameters described below. # parameters described below.
# #
# The Postfix memcache client supports the lookup and update # The Postfix memcache client supports the lookup, update,
# operations. # delete and sequence (first/next) operations. The sequence
# operation requires a backup database that supports the
# operation.
# MEMCACHE PARAMETERS # MEMCACHE PARAMETERS
# .ad # .ad
# .fi # .fi
# .IP "\fBhosts (default: localhost:11211)\fR" # .IP \fBbackup\fR
# The memcache servers that Postfix will try to connect to. # An optional Postfix database that provides persistent backup
# Specify a hostname or address, optionally followed by ":" # for the memcache database. The Postfix memcache client will
# and a port name or number. The default port is 11211. # update the memcache database whenever it looks up or changes
# Examples: # information in the persistent database. Specify a Postfix
# "type:table" database. Example:
# #
# .nf # .nf
# hosts = memcache01.example.com # backup = btree:/var/lib/postfix/postscreen_cache_map
# memcache02.example.com
# .fi # .fi
#
# Access to remote proxymap servers is under development.
#
# NOTE 1: When using memcache with persistent backup as
# \fBpostscreen\fR(8) or \fBverify\fR(8) cache, disable
# automatic cache cleanup (*_cache_cleanup_interval = 0) in
# all Postfix instances except for one instance that will be
# responsible for cache cleanup.
#
# NOTE 2: In the case of a proxied database, the full database
# name (including the "proxy:" prefix) must be specified in
# the proxymap server's proxy_read_maps or proxy_write_maps
# setting (depending on whether the access is read-only or
# read-write).
# .IP "\fBmemcache (default: inet:localhost:11211)\fR"
# The memcache server (note: singular) that Postfix will try
# to connect to. For a TCP server specify "inet:" followed by
# a hostname or address, ":", and a port name or number.
# For a UNIX-domain server specify "unix:" followed by the
# socket pathname. Examples:
#
# .nf
# memcache = inet:memcache.example.com
# memcache = unix:/path/to/socket
# .fi
#
# NOTE: In the case of a UNIX-domain socket, it must be accessible
# by the unprivileged postfix user and by the memcached process.
# .IP "\fBkey_format (default: %s)\fB" # .IP "\fBkey_format (default: %s)\fB"
# Format of the lookup and update keys in memcache queries. # Format of the lookup and update keys in memcache queries.
# By default, these are the same as the lookup and update # By default, these are the same as the lookup and update
@@ -102,43 +132,36 @@
# .IP "\fBflags (default: 0)\fR" # .IP "\fBflags (default: 0)\fR"
# Optional flags that should be stored along with a memcache # Optional flags that should be stored along with a memcache
# update. # update.
# .IP "\fBttl (default: 604800)\fR" # .IP "\fBttl (default: 3600)\fR"
# The expiration time in seconds of memcache updates. # The expiration time in seconds of memcache updates.
# The default is one week.
# #
# When using memcache tables with \fBpostscreen\fR(8) or # NOTE 1: When using a memcache table as \fBpostscreen\fR(8)
# \fBverify\fR(8), specify a zero *_cache_cleanup_interval # or \fBverify\fR(8) cache without persistent backup, specify
# value, and specify the largest \fBpostscreen\fR(8) *_ttl # a zero *_cache_cleanup_interval value with all Postfix
# value or \fBverify\fR(8) *_expire_time value as the memcache # instances that use the memcache, and specify the largest
# map's \fBttl\fR value. # \fBpostscreen\fR(8) *_ttl value or \fBverify\fR(8) *_expire_time
# value as the memcache table's \fBttl\fR value.
# #
# Note: according to memcache protocol documentation, a value # NOTE 2: According to memcache protocol documentation, a
# greater than 30 days (2592000 seconds) specifies absolute UNIX # value greater than 30 days (2592000 seconds) specifies
# absolute UNIX
# time. Smaller values are relative to the time of the update. # time. Smaller values are relative to the time of the update.
# BUGS # BUGS
# The Postfix memcache client is based on libmemcache, which
# will terminate its process after a memcache server goes
# down. To avoid this, set up redundant memcache servers that
# have no common source of failure.
#
# The Postfix memcache client cannot be used for security-sensitive # The Postfix memcache client cannot be used for security-sensitive
# tables such as \fBalias_maps\fR (these may contain # tables such as \fBalias_maps\fR (these may contain
# "\fI|command\fR and "\fI/file/name\fR" destinations), or # "\fI|command\fR and "\fI/file/name\fR" destinations), or
# \fBvirtual_uid_maps\fR and \fBvirtual_gid_maps\fR (these # \fBvirtual_uid_maps\fR, \fBvirtual_gid_maps\fR and
# specify UNIX process privileges). In a typical deployment # \fBvirtual_mailbox_maps\fR (these specify UNIX process
# a memcache database is shared via a TCP socket, and is # privileges or "\fI/file/name\fR" destinations). In a typical
# therefore writable not only by Postfix, but by any process # deployment a memcache database is writable by any process
# that can talk to the memcache server. # that can talk to the memcache server; in contrast,
# security-sensitive tables must not be writable by the
# unprivileged Postfix user.
# #
# The Postfix memcache client requires additional configuration # The Postfix memcache client requires additional configuration
# when used with the \fBpostscreen\fR(8) and \fBverify\fR(8) # when used as \fBpostscreen\fR(8) or \fBverify\fR(8) cache.
# daemons. For details see the \fBttl\fR parameter discussion # For details see the \fBbackup\fR and \fBttl\fR parameter
# at the end of the MEMCACHE PARAMETERS section in this # discussions in the MEMCACHE PARAMETERS section above.
# document.
#
# The Postfix memcache client is supported only with libmemcache
# version 1.4.0. Some libmemcache features are documented
# by reading libmemcache source code, instead a proper API.
# SEE ALSO # SEE ALSO
# postmap(1), Postfix lookup table manager # postmap(1), Postfix lookup table manager
# postconf(5), configuration parameters # postconf(5), configuration parameters
@@ -159,8 +182,9 @@
# .ad # .ad
# .fi # .fi
# The first memcache client for Postfix was written by Omar # The first memcache client for Postfix was written by Omar
# Kilani. Besides being implemented on libmemcache, this # Kilani, and was based on libmemcache.
# implementation bears no resemblance to his work. # The Postfix implementation does not use libmemcache, and
# bears no resemblance to earlier work.
# AUTHOR(S) # AUTHOR(S)
# Wietse Venema # Wietse Venema
# IBM T.J. Watson Research # IBM T.J. Watson Research

26
postfix/src/global/Makefile.in
View File

@@ -31,7 +31,7 @@ SRCS = abounce.c anvil_clnt.c been_here.c bounce.c bounce_log.c \
fold_addr.c header_body_checks.c mkmap_proxy.c data_redirect.c \ fold_addr.c header_body_checks.c mkmap_proxy.c data_redirect.c \
match_service.c mail_conf_nint.c addr_match_list.c mail_conf_nbool.c \ match_service.c mail_conf_nint.c addr_match_list.c mail_conf_nbool.c \
smtp_reply_footer.c safe_ultostr.c verify_sender_addr.c \ smtp_reply_footer.c safe_ultostr.c verify_sender_addr.c \
dict_memcache.c mail_version.c dict_memcache.c mail_version.c memcache_proto.c
OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \ OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \
canon_addr.o cfg_parser.o cleanup_strerror.o cleanup_strflags.o \ canon_addr.o cfg_parser.o cleanup_strerror.o cleanup_strflags.o \
clnt_stream.o conv_time.o db_common.o debug_peer.o debug_process.o \ clnt_stream.o conv_time.o db_common.o debug_peer.o debug_process.o \
@@ -64,7 +64,7 @@ OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \
fold_addr.o header_body_checks.o mkmap_proxy.o data_redirect.o \ fold_addr.o header_body_checks.o mkmap_proxy.o data_redirect.o \
match_service.o mail_conf_nint.o addr_match_list.o mail_conf_nbool.o \ match_service.o mail_conf_nint.o addr_match_list.o mail_conf_nbool.o \
smtp_reply_footer.o safe_ultostr.o verify_sender_addr.o \ smtp_reply_footer.o safe_ultostr.o verify_sender_addr.o \
dict_memcache.o mail_version.o dict_memcache.o mail_version.o memcache_proto.o
HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \ HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \
canon_addr.h cfg_parser.h cleanup_user.h clnt_stream.h config.h \ canon_addr.h cfg_parser.h cleanup_user.h clnt_stream.h config.h \
conv_time.h db_common.h debug_peer.h debug_process.h defer.h \ conv_time.h db_common.h debug_peer.h debug_process.h defer.h \
@@ -90,7 +90,7 @@ HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \
verp_sender.h wildcard_inet_addr.h xtext.h delivered_hdr.h \ verp_sender.h wildcard_inet_addr.h xtext.h delivered_hdr.h \
fold_addr.h header_body_checks.h data_redirect.h match_service.h \ fold_addr.h header_body_checks.h data_redirect.h match_service.h \
addr_match_list.h smtp_reply_footer.h safe_ultostr.h \ addr_match_list.h smtp_reply_footer.h safe_ultostr.h \
verify_sender_addr.h dict_memcache.h verify_sender_addr.h dict_memcache.h memcache_proto.h
TESTSRC = rec2stream.c stream2rec.c recdump.c TESTSRC = rec2stream.c stream2rec.c recdump.c
DEFS = -I. -I$(INC_DIR) -D$(SYSTYPE) DEFS = -I. -I$(INC_DIR) -D$(SYSTYPE)
CFLAGS = $(DEBUG) $(OPT) $(DEFS) CFLAGS = $(DEBUG) $(OPT) $(DEFS)
@@ -866,7 +866,7 @@ dict_ldap.o: dict_ldap.h
dict_ldap.o: mail_conf.h dict_ldap.o: mail_conf.h
dict_ldap.o: string_list.h dict_ldap.o: string_list.h
dict_memcache.o: ../../include/argv.h dict_memcache.o: ../../include/argv.h
dict_memcache.o: ../../include/binhash.h dict_memcache.o: ../../include/auto_clnt.h
dict_memcache.o: ../../include/dict.h dict_memcache.o: ../../include/dict.h
dict_memcache.o: ../../include/match_list.h dict_memcache.o: ../../include/match_list.h
dict_memcache.o: ../../include/match_ops.h dict_memcache.o: ../../include/match_ops.h
@@ -881,6 +881,7 @@ dict_memcache.o: cfg_parser.h
dict_memcache.o: db_common.h dict_memcache.o: db_common.h
dict_memcache.o: dict_memcache.c dict_memcache.o: dict_memcache.c
dict_memcache.o: dict_memcache.h dict_memcache.o: dict_memcache.h
dict_memcache.o: memcache_proto.h
dict_memcache.o: string_list.h dict_memcache.o: string_list.h
dict_mysql.o: ../../include/argv.h dict_mysql.o: ../../include/argv.h
dict_mysql.o: ../../include/dict.h dict_mysql.o: ../../include/dict.h
@@ -1470,6 +1471,7 @@ mail_trigger.o: ../../include/vstream.h
mail_trigger.o: mail_params.h mail_trigger.o: mail_params.h
mail_trigger.o: mail_proto.h mail_trigger.o: mail_proto.h
mail_trigger.o: mail_trigger.c mail_trigger.o: mail_trigger.c
mail_version.o: ../../include/msg.h
mail_version.o: ../../include/mymalloc.h mail_version.o: ../../include/mymalloc.h
mail_version.o: ../../include/split_at.h mail_version.o: ../../include/split_at.h
mail_version.o: ../../include/stringops.h mail_version.o: ../../include/stringops.h
@@ -1547,6 +1549,14 @@ mbox_open.o: dsn_buf.h
mbox_open.o: mbox_conf.h mbox_open.o: mbox_conf.h
mbox_open.o: mbox_open.c mbox_open.o: mbox_open.c
mbox_open.o: mbox_open.h mbox_open.o: mbox_open.h
memcache_proto.o: ../../include/msg.h
memcache_proto.o: ../../include/sys_defs.h
memcache_proto.o: ../../include/vbuf.h
memcache_proto.o: ../../include/vstream.h
memcache_proto.o: ../../include/vstring.h
memcache_proto.o: ../../include/vstring_vstream.h
memcache_proto.o: memcache_proto.c
memcache_proto.o: memcache_proto.h
mime_state.o: ../../include/msg.h mime_state.o: ../../include/msg.h
mime_state.o: ../../include/mymalloc.h mime_state.o: ../../include/mymalloc.h
mime_state.o: ../../include/sys_defs.h mime_state.o: ../../include/sys_defs.h
@@ -1560,15 +1570,7 @@ mime_state.o: mail_params.h
mime_state.o: mime_state.c mime_state.o: mime_state.c
mime_state.o: mime_state.h mime_state.o: mime_state.h
mime_state.o: rec_type.h mime_state.o: rec_type.h
mkmap_cdb.o: ../../include/argv.h
mkmap_cdb.o: ../../include/dict.h
mkmap_cdb.o: ../../include/dict_cdb.h
mkmap_cdb.o: ../../include/mymalloc.h
mkmap_cdb.o: ../../include/sys_defs.h mkmap_cdb.o: ../../include/sys_defs.h
mkmap_cdb.o: ../../include/vbuf.h
mkmap_cdb.o: ../../include/vstream.h
mkmap_cdb.o: ../../include/vstring.h
mkmap_cdb.o: mkmap.h
mkmap_cdb.o: mkmap_cdb.c mkmap_cdb.o: mkmap_cdb.c
mkmap_db.o: ../../include/argv.h mkmap_db.o: ../../include/argv.h
mkmap_db.o: ../../include/dict.h mkmap_db.o: ../../include/dict.h

497
postfix/src/global/dict_memcache.c
View File

@@ -2,7 +2,7 @@
/* NAME /* NAME
/* dict_memcache 3 /* dict_memcache 3
/* SUMMARY /* SUMMARY
/* dictionary interface to memcache databases /* dictionary interface to memcaches
/* SYNOPSIS /* SYNOPSIS
/* #include <dict_memcache.h> /* #include <dict_memcache.h>
/* /*
@@ -11,7 +11,7 @@
/* int open_flags; /* int open_flags;
/* int dict_flags; /* int dict_flags;
/* DESCRIPTION /* DESCRIPTION
/* dict_memcache_open() opens a memcache database, providing /* dict_memcache_open() opens a memcache, providing
/* a dictionary interface for Postfix key->value mappings. /* a dictionary interface for Postfix key->value mappings.
/* The result is a pointer to the installed dictionary. /* The result is a pointer to the installed dictionary.
/* /*
@@ -19,9 +19,7 @@
/* /*
/* Arguments: /* Arguments:
/* .IP name /* .IP name
/* Either the path to the Postfix memcache configuration file /* The path to the Postfix memcache configuration file.
/* (if it starts with '/' or '.'), or the parameter name prefix
/* which will be used to obtain main.cf configuration parameters.
/* .IP open_flags /* .IP open_flags
/* O_RDONLY or O_RDWR. This function ignores flags that don't /* O_RDONLY or O_RDWR. This function ignores flags that don't
/* specify a read, write or append mode. /* specify a read, write or append mode.
@@ -29,14 +27,13 @@
/* See dict_open(3). /* See dict_open(3).
/* SEE ALSO /* SEE ALSO
/* dict(3) generic dictionary manager /* dict(3) generic dictionary manager
/* BUGS
/* This code requires libmemcache 1.4.0, because some parts
/* of their API are documented by looking at the implementation.
/* HISTORY /* HISTORY
/* The first memcache client for Postfix was written by: /* .ad
/* Omar Kilani /* .fi
/* omar@tinysofa.com /* The first memcache client for Postfix was written by Omar
/* This implementation bears no resemblance to his work. /* Kilani, and was based on libmemcache. The current
/* implementation implements the memcache protocol directly,
/* and bears no resemblance to earlier work.
/* AUTHOR(S) /* AUTHOR(S)
/* Wietse Venema /* Wietse Venema
/* IBM T.J. Watson Research /* IBM T.J. Watson Research
@@ -46,15 +43,10 @@
/* System library. */ /* System library. */
#include "sys_defs.h" #include <sys_defs.h>
#ifdef HAS_MEMCACHE
#include <string.h> #include <string.h>
#include <memcache.h> #include <ctype.h>
#include <stdio.h> /* XXX sscanf() */
#if !defined(MEMCACHE_VERNUM) || MEMCACHE_VERNUM != 10400
#error "Postfix memcache supports only libmemcache version 1.4.0"
#endif
/* Utility library. */ /* Utility library. */
@@ -63,50 +55,39 @@
#include <dict.h> #include <dict.h>
#include <vstring.h> #include <vstring.h>
#include <stringops.h> #include <stringops.h>
#include <binhash.h> #include <auto_clnt.h>
#include <vstream.h>
/* Global library. */ /* Global library. */
#include <cfg_parser.h> #include <cfg_parser.h>
#include <db_common.h> #include <db_common.h>
#include <memcache_proto.h>
/* Application-specific. */ /* Application-specific. */
#include <dict_memcache.h> #include <dict_memcache.h>
/*
* Robustness tests (with a single memcache server) proved disappointing.
*
* After failure to connect to the memcache server, libmemcache reports the
* error once. From then on it silently discards all updates and always
* reports "not found" for all lookups, without ever reporting an error. To
* avoid this, we destroy the memcache client and create a new one after
* libmemcache reports an error.
*
* Even more problematic is that libmemcache will terminate the process when
* the memcache server connection is lost (the libmemcache error message is:
* "read(2) failed: Socket is already connected"). Unfortunately, telling
* libmemcache not to terminate the process will result in an assertion
* failure followed by core dump.
*
* Conclusion: if we want robust code, then we should use our own memcache
* protocol implementation instead of libmemcache.
*/
/* /*
* Structure of one memcache dictionary handle. * Structure of one memcache dictionary handle.
*/ */
typedef struct { typedef struct {
DICT dict; /* parent class */ DICT dict; /* parent class */
struct memcache_ctxt *mc_ctxt; /* libmemcache context */
struct memcache *mc; /* libmemcache object */
CFG_PARSER *parser; /* common parameter parser */ CFG_PARSER *parser; /* common parameter parser */
void *dbc_ctxt; /* db_common context */ void *dbc_ctxt; /* db_common context */
char *key_format; /* query key translation */ char *key_format; /* query key translation */
int timeout; /* client timeout */
int mc_ttl; /* memcache expiration */ int mc_ttl; /* memcache expiration */
int mc_flags; /* memcache flags */ int mc_flags; /* memcache flags */
int mc_pause; /* sleep between errors */
int mc_maxtry; /* number of tries */
char *memcache; /* memcache server spec */
AUTO_CLNT *clnt; /* memcache client stream */
VSTRING *clnt_buf; /* memcache client buffer */
VSTRING *key_buf; /* lookup key */ VSTRING *key_buf; /* lookup key */
VSTRING *res_buf; /* lookup result */ VSTRING *res_buf; /* lookup result */
int mc_errno; /* memcache dict_errno */
DICT *backup; /* persistent backup */
} DICT_MC; } DICT_MC;
/* /*
@@ -114,23 +95,13 @@ typedef struct {
*/ */
#define DICT_MC_DEF_HOST "localhost" #define DICT_MC_DEF_HOST "localhost"
#define DICT_MC_DEF_PORT "11211" #define DICT_MC_DEF_PORT "11211"
#define DICT_MC_DEF_HOST_PORT DICT_MC_DEF_HOST ":" DICT_MC_DEF_PORT #define DICT_MC_DEF_MEMCACHE "inet:" DICT_MC_DEF_HOST ":" DICT_MC_DEF_PORT
#define DICT_MC_DEF_KEY_FMT "%s" #define DICT_MC_DEF_KEY_FMT "%s"
#define DICT_MC_DEF_TTL (7 * 86400) #define DICT_MC_DEF_MC_TTL 3600
#define DICT_MC_DEF_FLAGS 0 #define DICT_MC_DEF_MC_TIMEOUT 2
#define DICT_MC_DEF_MC_FLAGS 0
/* #define DICT_MC_DEF_MC_MAXTRY 2
* libmemcache can report errors through an application call-back function, #define DICT_MC_DEF_MC_PAUSE 1
* but there is no support for passing application context to the call-back.
* The call-back API has two documented arguments: pointer to memcache_ctxt,
* and pointer to memcache_ectxt. The memcache_ctxt data structure has no
* space for application context, and the mcm_err() function zero-fills the
* memcache_ectxt data structure, making it useless for application context.
*
* We use our own hash table to find our dictionary handle, so that we can
* report errors in the proper context.
*/
static BINHASH *dict_mc_hash;
/* /*
* SLMs. * SLMs.
@@ -138,77 +109,94 @@ static BINHASH *dict_mc_hash;
#define STR(x) vstring_str(x) #define STR(x) vstring_str(x)
#define LEN(x) VSTRING_LEN(x) #define LEN(x) VSTRING_LEN(x)
/*#define msg_verbose 1*/ #define msg_verbose 1
/* dict_memcache_error_cb - error call-back */ /* dict_memcache_set - set memcache key/value */
static int dict_memcache_error_cb(MCM_ERR_FUNC_ARGS) static void dict_memcache_set(DICT_MC *dict_mc, const char *value, int ttl)
{ {
const char *myname = "dict_memcache_error_cb"; VSTREAM *fp;
const struct memcache_ctxt *ctxt; int count;
struct memcache_err_ctxt *ectxt;
DICT_MC *dict_mc;
void (*log_fn) (const char *,...);
/* #define MC_LINE_LIMIT 1024
* Play by the rules of the libmemcache API.
*/
MCM_ERR_INIT_CTXT(ctxt, ectxt);
/* dict_mc->mc_errno = DICT_ERR_RETRY;
* Locate our own dictionary handle for error reporting context. for (count = 0; count < dict_mc->mc_maxtry; count++) {
* Unfortunately, the ctxt structure does not store application context, if (count > 0)
* and mcm_err() zero-fills the ectxt structure, making it useless for sleep(1);
* storing application context. We use our own hash table instead. if ((fp = auto_clnt_access(dict_mc->clnt)) != 0) {
*/ if (memcache_printf(fp, "set %s %d %d %ld",
if ((dict_mc = (DICT_MC *) binhash_find(dict_mc_hash, (char *) &ctxt, STR(dict_mc->key_buf), dict_mc->mc_flags,
sizeof(ctxt))) == 0) ttl, strlen(value)) < 0
msg_panic("%s: can't locate DICT_MC database handle", myname); || memcache_fwrite(fp, value, strlen(value)) < 0
|| memcache_get(fp, dict_mc->clnt_buf, MC_LINE_LIMIT) < 0) {
/* if (count > 0)
* Report the error in our context, and set dict_errno for possible msg_warn("database %s:%s: I/O error: %m",
* errors. We override dict_errno when an error was recoverable. DICT_TYPE_MEMCACHE, dict_mc->dict.name);
*/ auto_clnt_recover(dict_mc->clnt);
switch (ectxt->severity) { } else if (strcmp(STR(dict_mc->clnt_buf), "STORED") != 0) {
default: if (count > 0)
#ifdef DICT_MC_RECOVER_FROM_DISCONNECT msg_warn("database %s:%s: update failed: %.30s",
/* Code below causes an assert failure and core dump. */ DICT_TYPE_MEMCACHE, dict_mc->dict.name,
if (ectxt->errcode == MCM_ERR_SYS_READ) STR(dict_mc->clnt_buf));
/* Also: MCM_ERR_SYS_WRITEV, MCM_ERR_SYS_SETSOCKOPT */ auto_clnt_recover(dict_mc->clnt);
ectxt->cont = 'y'; } else {
#endif /* Victory! */
/* FALLTHROUGH */ dict_mc->mc_errno = 0;
case MCM_ERR_LVL_NOTICE:
log_fn = msg_warn;
dict_errno = 1;
break;
case MCM_ERR_LVL_INFO:
log_fn = msg_info;
break; break;
} }
log_fn((ectxt)->errnum ? "database %s:%s: libmemcache error: %s: %m" : }
"database %s:%s: libmemcache error: %s", }
DICT_TYPE_MEMCACHE, dict_mc->dict.name, (ectxt)->errstr);
return (0);
} }
static void dict_memcache_mc_free(DICT_MC *); /* dict_memcache_get - get memcache key/value */
static void dict_memcache_mc_init(DICT_MC *);
/* dict_memcache_recover - recover after libmemcache error */ static const char *dict_memcache_get(DICT_MC *dict_mc)
static void dict_memcache_recover(DICT_MC *dict_mc)
{ {
int saved_dict_errno; VSTREAM *fp;
long todo;
const char *retval;
int count;
/* dict_mc->mc_errno = DICT_ERR_RETRY;
* XXX If we don't try to recover from the first error, libmemcache will retval = 0;
* silently skip all subsequent database operations. for (count = 0; count < dict_mc->mc_maxtry; count++) {
*/ if (count > 0)
saved_dict_errno = dict_errno; sleep(1);
dict_memcache_mc_free(dict_mc); if ((fp = auto_clnt_access(dict_mc->clnt)) != 0) {
dict_memcache_mc_init(dict_mc); if (memcache_printf(fp, "get %s", STR(dict_mc->key_buf)) < 0
dict_errno = saved_dict_errno; || memcache_get(fp, dict_mc->clnt_buf, MC_LINE_LIMIT) < 0) {
if (count > 0)
msg_warn("database %s:%s: I/O error: %m",
DICT_TYPE_MEMCACHE, dict_mc->dict.name);
auto_clnt_recover(dict_mc->clnt);
} else if (strcmp(STR(dict_mc->clnt_buf), "END") == 0) {
/* Not found. */
dict_mc->mc_errno = 0;
break;
} else if (sscanf(STR(dict_mc->clnt_buf),
"VALUE %*s %*s %ld", &todo) != 1 || todo < 0) {
if (count > 0)
msg_warn("%s: unexpected memcache server reply: %.30s",
dict_mc->dict.name, STR(dict_mc->clnt_buf));
auto_clnt_recover(dict_mc->clnt);
} else if (memcache_fread(fp, dict_mc->res_buf, todo) < 0) {
if (count > 0)
msg_warn("%s: EOF receiving memcache server reply",
dict_mc->dict.name);
auto_clnt_recover(dict_mc->clnt);
} else {
/* Victory! */
retval = STR(dict_mc->res_buf);
dict_mc->mc_errno = 0;
if (memcache_get(fp, dict_mc->clnt_buf, MC_LINE_LIMIT) < 0
|| strcmp(STR(dict_mc->clnt_buf), "END") != 0)
auto_clnt_recover(dict_mc->clnt);
break;
}
}
}
return (retval);
} }
/* dict_memcache_prepare_key - prepare lookup key */ /* dict_memcache_prepare_key - prepare lookup key */
@@ -249,189 +237,204 @@ static int dict_memcache_prepare_key(DICT_MC *dict_mc, const char *name)
return (LEN(dict_mc->key_buf)); return (LEN(dict_mc->key_buf));
} }
/* dict_memcache_update - update memcache database */ /* dict_memcache_valid_key - validate key */
static int dict_memcache_valid_key(DICT_MC *dict_mc,
const char *name,
const char *operation,
void (*log_func) (const char *,...))
{
unsigned char *cp;
#define DICT_MC_SKIP(why) do { \
if (msg_verbose || log_func != msg_info) \
log_func("%s: skipping %s for name \"%s\": %s", \
dict_mc->dict.name, operation, name, (why)); \
return(0); \
} while (0)
if (*name == 0)
DICT_MC_SKIP("empty lookup key");
if (db_common_check_domain(dict_mc->dbc_ctxt, name) == 0)
DICT_MC_SKIP("domain mismatch");
if (dict_memcache_prepare_key(dict_mc, name) == 0)
DICT_MC_SKIP("empty lookup key expansion");
for (cp = (unsigned char *) STR(dict_mc->key_buf); *cp; cp++)
if (isascii(*cp) && isspace(*cp))
DICT_MC_SKIP("name contains space");
return (1);
}
/* dict_memcache_update - update memcache */
static void dict_memcache_update(DICT *dict, const char *name, static void dict_memcache_update(DICT *dict, const char *name,
const char *value) const char *value)
{ {
const char *myname = "dict_memcache_update"; const char *myname = "dict_memcache_update";
DICT_MC *dict_mc = (DICT_MC *) dict; DICT_MC *dict_mc = (DICT_MC *) dict;
int backup_errno = 0;
/* /*
* Skip updates with a null key, noisily. This would result in loss of * Skip updates with an inapplicable key, noisily. This results in loss
* information. * of information.
*/ */
if (dict_memcache_prepare_key(dict_mc, name) == 0) { dict_errno = DICT_ERR_RETRY;
dict_errno = 1; if (dict_memcache_valid_key(dict_mc, name, "update", msg_warn) == 0)
msg_warn("database %s:%s: name \"%s\" expands to empty lookup key "
"-- skipping update", DICT_TYPE_MEMCACHE,
dict_mc->dict.name, name);
return; return;
/*
* Update the backup database first.
*/
if (dict_mc->backup) {
dict_errno = 0;
dict_mc->backup->update(dict_mc->backup, name, value);
backup_errno = dict_errno;
} }
/* /*
* Our error call-back routine will report errors and set dict_errno. * Update the memcache last.
*/ */
dict_errno = (mcm_set(dict_mc->mc_ctxt, dict_mc->mc, STR(dict_mc->key_buf), dict_memcache_set(dict_mc, value, dict_mc->mc_ttl);
LEN(dict_mc->key_buf), value, strlen(value),
dict_mc->mc_ttl, dict_mc->mc_flags) != 0);
if (msg_verbose) if (msg_verbose)
msg_info("%s: %s: update key \"%s\" => \"%s\" %s", msg_info("%s: %s: update key \"%s\" => \"%s\" %s",
myname, dict_mc->dict.name, STR(dict_mc->key_buf), value, myname, dict_mc->dict.name, STR(dict_mc->key_buf), value,
dict_errno ? "(error)" : "(no error)"); dict_mc->mc_errno ? "(memcache error)" :
backup_errno ? "(backup error)" : "(no error)");
/* dict_errno = (backup_errno ? backup_errno : dict_mc->mc_errno);
* Recover after server failure.
*/
if (dict_errno)
dict_memcache_recover(dict_mc);
} }
/* dict_memcache_lookup - lookup memcache database */ /* dict_memcache_lookup - lookup memcache */
static const char *dict_memcache_lookup(DICT *dict, const char *name) static const char *dict_memcache_lookup(DICT *dict, const char *name)
{ {
const char *myname = "dict_memcache_lookup"; const char *myname = "dict_memcache_lookup";
DICT_MC *dict_mc = (DICT_MC *) dict; DICT_MC *dict_mc = (DICT_MC *) dict;
struct memcache_req *req;
struct memcache_res *res;
const char *retval; const char *retval;
int backup_errno = 0;
/* /*
* Skip lookups with a null key, silently. This is just asking for * Skip lookups with an inapplicable key, silently. This is just asking
* information that cannot exist. * for information that cannot exist.
*/ */
#define DICT_MC_SKIP(why, map_name, key) do { \ dict_errno = 0;
if (msg_verbose) \ if (dict_memcache_valid_key(dict_mc, name, "lookup", msg_info) == 0)
msg_info("%s: %s: skipping lookup of key \"%s\": %s", \ return (0);
myname, (map_name), (key), (why)); \
return (0); \
} while (0)
if (*name == 0)
DICT_MC_SKIP("empty lookup key", dict_mc->dict.name, name);
if (db_common_check_domain(dict_mc->dbc_ctxt, name) == 0)
DICT_MC_SKIP("domain mismatch", dict_mc->dict.name, name);
if (dict_memcache_prepare_key(dict_mc, name) == 0)
DICT_MC_SKIP("empty lookup key expansion", dict_mc->dict.name, name);
/* /*
* Our error call-back routine will report errors and set dict_errno. We * Search the memcache first.
* reset dict_errno after an error turns out to be recoverable.
*/ */
if ((req = mcm_req_new(dict_mc->mc_ctxt)) == 0) retval = dict_memcache_get(dict_mc);
msg_fatal("%s: can't create new request: %m", myname); /* XXX */
/* Not: mcm_req_add(), because that makes unnecessary copy of the key. */
if ((res = mcm_req_add_ref(dict_mc->mc_ctxt, req, STR(dict_mc->key_buf),
LEN(dict_mc->key_buf))) == 0)
msg_fatal("%s: can't create new result: %m", myname); /* XXX */
dict_errno = 0; /*
mcm_get(dict_mc->mc_ctxt, dict_mc->mc, req); * Search the backup database last. Update the memcache if the data is
if (mcm_res_found(dict_mc->mc_ctxt, res) && res->bytes) { * found.
vstring_strncpy(dict_mc->res_buf, res->val, res->bytes); */
retval = STR(dict_mc->res_buf); if (retval == 0 && dict_mc->backup) {
dict_errno = 0; retval = dict_mc->backup->lookup(dict_mc->backup, name);
} else { backup_errno = dict_errno;
retval = 0; /* Update the cache. */
if (retval != 0)
dict_memcache_set(dict_mc, retval, dict_mc->mc_ttl);
} }
mcm_res_free(dict_mc->mc_ctxt, req, res);
mcm_req_free(dict_mc->mc_ctxt, req);
if (msg_verbose) if (msg_verbose)
msg_info("%s: %s: key %s => %s", msg_info("%s: %s: key %s => %s",
myname, dict_mc->dict.name, STR(dict_mc->key_buf), myname, dict_mc->dict.name, STR(dict_mc->key_buf),
retval ? STR(dict_mc->res_buf) : retval ? retval :
dict_errno ? "(error)" : "(not found)"); dict_mc->mc_errno ? "(memcache error)" :
backup_errno ? "(backup error)" : "(not found)");
/*
* Recover after server failure.
*/
if (dict_errno)
dict_memcache_recover(dict_mc);
return (retval); return (retval);
} }
/* dict_memcache_mc_free - destroy libmemcache objects */ /* dict_memcache_delete - delete memcache entry */
static void dict_memcache_mc_free(DICT_MC *dict_mc) static int dict_memcache_delete(DICT *dict, const char *name)
{ {
binhash_delete(dict_mc_hash, (char *) &dict_mc->mc_ctxt, const char *myname = "dict_memcache_delete";
sizeof(dict_mc->mc_ctxt), (void (*) (char *)) 0); DICT_MC *dict_mc = (DICT_MC *) dict;
mcm_free(dict_mc->mc_ctxt, dict_mc->mc); const char *retval;
mcMemFreeCtxt(dict_mc->mc_ctxt); int backup_errno = 0;
} int del_res = 0;
/* dict_memcache_mc_init - create libmemcache objects */
static void dict_memcache_mc_init(DICT_MC *dict_mc)
{
const char *myname = "dict_memcache_mc_init";
char *servers;
char *server;
char *cp;
/* /*
* Create the libmemcache objects. * Skip lookups with an inapplicable key, silently. This is just deleting
* information that cannot exist.
*/ */
dict_mc->mc_ctxt = dict_errno = 0;
mcMemNewCtxt((mcFreeFunc) myfree, (mcMallocFunc) mymalloc, if (dict_memcache_valid_key(dict_mc, name, "delete", msg_info) == 0)
(mcMallocFunc) mymalloc, (mcReallocFunc) myrealloc); return (1);
if (dict_mc->mc_ctxt == 0)
msg_fatal("error creating memcache context: %m"); /* XXX */
dict_mc->mc = mcm_new(dict_mc->mc_ctxt);
if (dict_mc->mc == 0)
msg_fatal("error creating memcache object: %m"); /* XXX */
/* /*
* Set up call-back info for error reporting. * Update the persistent database first.
*/ */
if (dict_mc_hash == 0) if (dict_mc->backup) {
dict_mc_hash = binhash_create(1); dict_errno = 0;
binhash_enter(dict_mc_hash, (char *) &dict_mc->mc_ctxt, del_res = dict_mc->backup->delete(dict_mc->backup, name);
sizeof(dict_mc->mc_ctxt), (char *) dict_mc); backup_errno = dict_errno;
mcErrSetupCtxt(dict_mc->mc_ctxt, dict_memcache_error_cb);
/*
* Add the server list.
*/
cp = servers = cfg_get_str(dict_mc->parser, "hosts",
DICT_MC_DEF_HOST_PORT, 0, 0);
while ((server = mystrtok(&cp, " ,\t\r\n")) != 0) {
if (msg_verbose)
msg_info("%s: database %s:%s: adding server %s",
myname, DICT_TYPE_MEMCACHE, dict_mc->dict.name, server);
if (mcm_server_add4(dict_mc->mc_ctxt, dict_mc->mc, server) < 0)
msg_warn("database %s:%s: error adding server %s",
DICT_TYPE_MEMCACHE, dict_mc->dict.name, server);
} }
myfree(servers);
/*
* Update the memcache last. There is no memcache delete operation.
* Instead, we set a short expiration time if the data exists.
*/
if ((retval = dict_memcache_get(dict_mc)) != 0)
dict_memcache_set(dict_mc, retval, 1);
if (msg_verbose)
msg_info("%s: %s: delete key %s => %s",
myname, dict_mc->dict.name, STR(dict_mc->key_buf),
dict_mc->mc_errno ? "(memcache error)" :
backup_errno ? "(backup error)" : "(no error)");
dict_errno = (backup_errno ? backup_errno : dict_mc->mc_errno);
return (del_res);
} }
/* dict_memcache_close - close memcache database */ /* dict_memcache_sequence - first/next lookup */
static int dict_memcache_sequence(DICT *dict, int function, const char **key,
const char **value)
{
DICT_MC *dict_mc = (DICT_MC *) dict;
if (dict_mc->backup == 0)
msg_fatal("database %s:%s: first/next support requires backup database",
DICT_TYPE_MEMCACHE, dict_mc->dict.name);
return (dict_mc->backup->sequence(dict_mc->backup, function, key, value));
}
/* dict_memcache_close - close memcache */
static void dict_memcache_close(DICT *dict) static void dict_memcache_close(DICT *dict)
{ {
DICT_MC *dict_mc = (DICT_MC *) dict; DICT_MC *dict_mc = (DICT_MC *) dict;
dict_memcache_mc_free(dict_mc);
cfg_parser_free(dict_mc->parser); cfg_parser_free(dict_mc->parser);
db_common_free_ctx(dict_mc->dbc_ctxt); db_common_free_ctx(dict_mc->dbc_ctxt);
vstring_free(dict_mc->key_buf);
vstring_free(dict_mc->res_buf);
if (dict_mc->key_format) if (dict_mc->key_format)
myfree(dict_mc->key_format); myfree(dict_mc->key_format);
myfree(dict_mc->memcache);
auto_clnt_free(dict_mc->clnt);
vstring_free(dict_mc->clnt_buf);
vstring_free(dict_mc->key_buf);
vstring_free(dict_mc->res_buf);
if (dict->fold_buf) if (dict->fold_buf)
vstring_free(dict->fold_buf); vstring_free(dict->fold_buf);
if (dict_mc->backup)
dict_close(dict_mc->backup);
dict_free(dict); dict_free(dict);
} }
/* dict_memcache_open - open memcache database */ /* dict_memcache_open - open memcache */
DICT *dict_memcache_open(const char *name, int open_flags, int dict_flags) DICT *dict_memcache_open(const char *name, int open_flags, int dict_flags)
{ {
DICT_MC *dict_mc; DICT_MC *dict_mc;
char *backup;
/* /*
* Sanity checks. * Sanity checks.
@@ -450,8 +453,11 @@ DICT *dict_memcache_open(const char *name, int open_flags, int dict_flags)
dict_mc = (DICT_MC *) dict_alloc(DICT_TYPE_MEMCACHE, name, dict_mc = (DICT_MC *) dict_alloc(DICT_TYPE_MEMCACHE, name,
sizeof(*dict_mc)); sizeof(*dict_mc));
dict_mc->dict.lookup = dict_memcache_lookup; dict_mc->dict.lookup = dict_memcache_lookup;
if (open_flags == O_RDWR) if (open_flags == O_RDWR) {
dict_mc->dict.update = dict_memcache_update; dict_mc->dict.update = dict_memcache_update;
dict_mc->dict.delete = dict_memcache_delete;
}
dict_mc->dict.sequence = dict_memcache_sequence;
dict_mc->dict.close = dict_memcache_close; dict_mc->dict.close = dict_memcache_close;
dict_mc->dict.flags = dict_flags; dict_mc->dict.flags = dict_flags;
dict_mc->key_buf = vstring_alloc(10); dict_mc->key_buf = vstring_alloc(10);
@@ -463,15 +469,34 @@ DICT *dict_memcache_open(const char *name, int open_flags, int dict_flags)
dict_mc->parser = cfg_parser_alloc(name); dict_mc->parser = cfg_parser_alloc(name);
dict_mc->key_format = cfg_get_str(dict_mc->parser, "key_format", dict_mc->key_format = cfg_get_str(dict_mc->parser, "key_format",
DICT_MC_DEF_KEY_FMT, 0, 0); DICT_MC_DEF_KEY_FMT, 0, 0);
dict_mc->timeout = cfg_get_int(dict_mc->parser, "timeout",
DICT_MC_DEF_MC_TIMEOUT, 0, 0);
dict_mc->mc_ttl = cfg_get_int(dict_mc->parser, "ttl", dict_mc->mc_ttl = cfg_get_int(dict_mc->parser, "ttl",
DICT_MC_DEF_TTL, 0, 0); DICT_MC_DEF_MC_TTL, 0, 0);
dict_mc->mc_flags = cfg_get_int(dict_mc->parser, "flags", dict_mc->mc_flags = cfg_get_int(dict_mc->parser, "flags",
DICT_MC_DEF_FLAGS, 0, 0); DICT_MC_DEF_MC_FLAGS, 0, 0);
dict_mc->mc_pause = cfg_get_int(dict_mc->parser, "error_pause",
DICT_MC_DEF_MC_PAUSE, 1, 0);
dict_mc->mc_maxtry = cfg_get_int(dict_mc->parser, "maxtry",
DICT_MC_DEF_MC_MAXTRY, 1, 0);
dict_mc->memcache = cfg_get_str(dict_mc->parser, "memcache",
DICT_MC_DEF_MEMCACHE, 0, 0);
/* /*
* Initialize the memcache objects. * Initialize the memcache client.
*/ */
dict_memcache_mc_init(dict_mc); dict_mc->clnt = auto_clnt_create(dict_mc->memcache, dict_mc->timeout, 0, 0);
dict_mc->clnt_buf = vstring_alloc(100);
/*
* Open the optional backup database.
*/
backup = cfg_get_str(dict_mc->parser, "backup", (char *) 0, 0, 0);
if (backup) {
dict_mc->backup = dict_open(backup, open_flags, dict_flags);
myfree(backup);
} else
dict_mc->backup = 0;
/* /*
* Parse templates and common database parameters. Maps that use * Parse templates and common database parameters. Maps that use
@@ -489,5 +514,3 @@ DICT *dict_memcache_open(const char *name, int open_flags, int dict_flags)
return (&dict_mc->dict); return (&dict_mc->dict);
} }
#endif

103
postfix/src/global/dict_proxy.c
View File

@@ -11,11 +11,11 @@
/* int open_flags; /* int open_flags;
/* int dict_flags; /* int dict_flags;
/* DESCRIPTION /* DESCRIPTION
/* dict_proxy_open() relays read-only operations through /* dict_proxy_open() relays read-only or read-write operations
/* the Postfix proxymap server. /* through the Postfix proxymap server.
/* /*
/* The \fIopen_flags\fR argument must specify O_RDONLY /* The \fIopen_flags\fR argument must specify O_RDONLY
/* or O_RDWR|O_CREAT. Depending on this, the client /* or O_RDWR. Depending on this, the client
/* connects to the proxymap multiserver or to the /* connects to the proxymap multiserver or to the
/* proxywrite single updater. /* proxywrite single updater.
/* /*
@@ -73,6 +73,7 @@ typedef struct {
CLNT_STREAM *clnt; /* client handle (shared) */ CLNT_STREAM *clnt; /* client handle (shared) */
const char *service; /* service name */ const char *service; /* service name */
int in_flags; /* caller-specified flags */ int in_flags; /* caller-specified flags */
VSTRING *reskey; /* result key storage */
VSTRING *result; /* storage */ VSTRING *result; /* storage */
} DICT_PROXY; } DICT_PROXY;
@@ -88,6 +89,86 @@ typedef struct {
static CLNT_STREAM *proxymap_stream; /* read-only maps */ static CLNT_STREAM *proxymap_stream; /* read-only maps */
static CLNT_STREAM *proxywrite_stream; /* read-write maps */ static CLNT_STREAM *proxywrite_stream; /* read-write maps */
/* dict_proxy_sequence - find first/next entry */
static int dict_proxy_sequence(DICT *dict, int function,
const char **key, const char **value)
{
const char *myname = "dict_proxy_sequence";
DICT_PROXY *dict_proxy = (DICT_PROXY *) dict;
VSTREAM *stream;
int status;
int count = 0;
int request_flags;
/*
* The client and server live in separate processes that may start and
* terminate independently. We cannot rely on a persistent connection,
* let alone on persistent state (such as a specific open table) that is
* associated with a specific connection. Each lookup needs to specify
* the table and the flags that were specified to dict_proxy_open().
*/
VSTRING_RESET(dict_proxy->reskey);
VSTRING_TERMINATE(dict_proxy->reskey);
VSTRING_RESET(dict_proxy->result);
VSTRING_TERMINATE(dict_proxy->result);
request_flags = (dict_proxy->in_flags & DICT_FLAG_RQST_MASK)
| (dict->flags & DICT_FLAG_RQST_MASK);
for (;;) {
stream = clnt_stream_access(dict_proxy->clnt);
errno = 0;
count += 1;
if (attr_print(stream, ATTR_FLAG_NONE,
ATTR_TYPE_STR, MAIL_ATTR_REQ, PROXY_REQ_SEQUENCE,
ATTR_TYPE_STR, MAIL_ATTR_TABLE, dict->name,
ATTR_TYPE_INT, MAIL_ATTR_FLAGS, request_flags,
ATTR_TYPE_INT, MAIL_ATTR_FUNC, function,
ATTR_TYPE_END) != 0
|| vstream_fflush(stream)
|| attr_scan(stream, ATTR_FLAG_STRICT,
ATTR_TYPE_INT, MAIL_ATTR_STATUS, &status,
ATTR_TYPE_STR, MAIL_ATTR_KEY, dict_proxy->reskey,
ATTR_TYPE_STR, MAIL_ATTR_VALUE, dict_proxy->result,
ATTR_TYPE_END) != 3) {
if (msg_verbose || count > 1 || (errno && errno != EPIPE && errno != ENOENT))
msg_warn("%s: service %s: %m", myname, VSTREAM_PATH(stream));
} else {
if (msg_verbose)
msg_info("%s: table=%s flags=%s func=%d -> status=%d key=%s val=%s",
myname, dict->name, dict_flags_str(request_flags),
function, status, STR(dict_proxy->reskey),
STR(dict_proxy->result));
switch (status) {
case PROXY_STAT_BAD:
msg_fatal("%s sequence failed for table \"%s\" function %d: "
"invalid request",
dict_proxy->service, dict->name, function);
case PROXY_STAT_DENY:
msg_fatal("%s service is not configured for table \"%s\"",
dict_proxy->service, dict->name);
case PROXY_STAT_OK:
*key = STR(dict_proxy->reskey);
*value = STR(dict_proxy->result);
return (0);
case PROXY_STAT_NOKEY:
dict_errno = 0;
*key = *value = 0;
return (1);
case PROXY_STAT_RETRY:
dict_errno = DICT_ERR_RETRY;
*key = *value = 0;
return (1);
default:
msg_warn("%s sequence failed for table \"%s\" function %d: "
"unexpected reply status %d",
dict_proxy->service, dict->name, function, status);
}
}
clnt_stream_recover(dict_proxy->clnt);
sleep(1); /* XXX make configurable */
}
}
/* dict_proxy_lookup - find table entry */ /* dict_proxy_lookup - find table entry */
static const char *dict_proxy_lookup(DICT *dict, const char *key) static const char *dict_proxy_lookup(DICT *dict, const char *key)
@@ -212,6 +293,9 @@ static void dict_proxy_update(DICT *dict, const char *key, const char *value)
dict_proxy->service, dict->name); dict_proxy->service, dict->name);
case PROXY_STAT_OK: case PROXY_STAT_OK:
return; return;
case PROXY_STAT_RETRY:
dict_errno = DICT_ERR_RETRY;
return;
default: default:
msg_warn("%s update failed for table \"%s\" key \"%s\": " msg_warn("%s update failed for table \"%s\" key \"%s\": "
"unexpected reply status %d", "unexpected reply status %d",
@@ -275,8 +359,12 @@ static int dict_proxy_delete(DICT *dict, const char *key)
dict_proxy->service, dict->name); dict_proxy->service, dict->name);
case PROXY_STAT_OK: case PROXY_STAT_OK:
return 0; return 0;
case PROXY_STAT_RETRY:
dict_errno = DICT_ERR_RETRY;
return (-1);
case PROXY_STAT_NOKEY: case PROXY_STAT_NOKEY:
return 1; dict_errno = 0;
return (1);
default: default:
msg_warn("%s delete failed for table \"%s\" key \"%s\": " msg_warn("%s delete failed for table \"%s\" key \"%s\": "
"unexpected reply status %d", "unexpected reply status %d",
@@ -294,6 +382,7 @@ static void dict_proxy_close(DICT *dict)
{ {
DICT_PROXY *dict_proxy = (DICT_PROXY *) dict; DICT_PROXY *dict_proxy = (DICT_PROXY *) dict;
vstring_free(dict_proxy->reskey);
vstring_free(dict_proxy->result); vstring_free(dict_proxy->result);
dict_free(dict); dict_free(dict);
} }
@@ -334,11 +423,11 @@ DICT *dict_proxy_open(const char *map, int open_flags, int dict_flags)
if (open_flags == O_RDONLY) { if (open_flags == O_RDONLY) {
pstream = &proxymap_stream; pstream = &proxymap_stream;
service = var_proxymap_service; service = var_proxymap_service;
} else if (open_flags == (O_RDWR | O_CREAT)) { } else if ((open_flags & O_RDWR) == O_RDWR) {
pstream = &proxywrite_stream; pstream = &proxywrite_stream;
service = var_proxywrite_service; service = var_proxywrite_service;
} else } else
msg_fatal("%s: %s map open requires O_RDONLY or O_RDWR|O_CREAT mode", msg_fatal("%s: %s map open requires O_RDONLY or O_RDWR mode",
map, DICT_TYPE_PROXY); map, DICT_TYPE_PROXY);
if (*pstream == 0) { if (*pstream == 0) {
@@ -364,8 +453,10 @@ DICT *dict_proxy_open(const char *map, int open_flags, int dict_flags)
dict_proxy->dict.lookup = dict_proxy_lookup; dict_proxy->dict.lookup = dict_proxy_lookup;
dict_proxy->dict.update = dict_proxy_update; dict_proxy->dict.update = dict_proxy_update;
dict_proxy->dict.delete = dict_proxy_delete; dict_proxy->dict.delete = dict_proxy_delete;
dict_proxy->dict.sequence = dict_proxy_sequence;
dict_proxy->dict.close = dict_proxy_close; dict_proxy->dict.close = dict_proxy_close;
dict_proxy->in_flags = dict_flags; dict_proxy->in_flags = dict_flags;
dict_proxy->reskey = vstring_alloc(10);
dict_proxy->result = vstring_alloc(10); dict_proxy->result = vstring_alloc(10);
dict_proxy->clnt = *pstream; dict_proxy->clnt = *pstream;
dict_proxy->service = service; dict_proxy->service = service;

1
postfix/src/global/dict_proxy.h
View File

@@ -30,6 +30,7 @@ extern DICT *dict_proxy_open(const char *, int, int);
#define PROXY_REQ_LOOKUP "lookup" #define PROXY_REQ_LOOKUP "lookup"
#define PROXY_REQ_UPDATE "update" #define PROXY_REQ_UPDATE "update"
#define PROXY_REQ_DELETE "delete" #define PROXY_REQ_DELETE "delete"
#define PROXY_REQ_SEQUENCE "sequence"
#define PROXY_STAT_OK 0 /* operation succeeded */ #define PROXY_STAT_OK 0 /* operation succeeded */
#define PROXY_STAT_NOKEY 1 /* requested key not found */ #define PROXY_STAT_NOKEY 1 /* requested key not found */

11
postfix/src/global/mail_dict.c
View File

@@ -59,9 +59,7 @@ static const DICT_OPEN_INFO dict_open_info[] = {
#ifdef HAS_SQLITE #ifdef HAS_SQLITE
DICT_TYPE_SQLITE, dict_sqlite_open, DICT_TYPE_SQLITE, dict_sqlite_open,
#endif #endif
#ifdef HAS_MEMCACHE
DICT_TYPE_MEMCACHE, dict_memcache_open, DICT_TYPE_MEMCACHE, dict_memcache_open,
#endif
0, 0,
}; };
@@ -76,12 +74,19 @@ void mail_dict_init(void)
} }
#ifdef TEST #ifdef TEST
/* /*
* Proof-of-concept test program. * Proof-of-concept test program.
*/ */
#include <mail_proto.h>
#include <mail_params.h>
int main(int argc, char **argv) int main(int argc, char **argv)
{ {
var_queue_dir = DEF_QUEUE_DIR;
var_proxymap_service = DEF_PROXYMAP_SERVICE;
var_proxywrite_service = DEF_PROXYWRITE_SERVICE;
var_ipc_timeout = 3600;
mail_dict_init(); mail_dict_init();
dict_test(argc, argv); dict_test(argc, argv);
return (0); return (0);

1
postfix/src/global/mail_proto.h
View File

@@ -141,6 +141,7 @@ extern char *mail_pathname(const char *, const char *);
#define MAIL_ATTR_TTL "ttl" #define MAIL_ATTR_TTL "ttl"
#define MAIL_ATTR_LABEL "label" #define MAIL_ATTR_LABEL "label"
#define MAIL_ATTR_PROP "property" #define MAIL_ATTR_PROP "property"
#define MAIL_ATTR_FUNC "function"
#define MAIL_ATTR_CCERT_SUBJECT "ccert_subject" #define MAIL_ATTR_CCERT_SUBJECT "ccert_subject"
#define MAIL_ATTR_CCERT_ISSUER "ccert_issuer" #define MAIL_ATTR_CCERT_ISSUER "ccert_issuer"
#define MAIL_ATTR_CCERT_FINGERPRINT "ccert_fingerprint" #define MAIL_ATTR_CCERT_FINGERPRINT "ccert_fingerprint"

2
postfix/src/global/mail_version.h
View File

@@ -20,7 +20,7 @@
* Patches change both the patchlevel and the release date. Snapshots have no * Patches change both the patchlevel and the release date. Snapshots have no
* patchlevel; they change the release date only. * patchlevel; they change the release date only.
*/ */
#define MAIL_RELEASE_DATE "20111209" #define MAIL_RELEASE_DATE "20111213"
#define MAIL_VERSION_NUMBER "2.9" #define MAIL_VERSION_NUMBER "2.9"
#ifdef SNAPSHOT #ifdef SNAPSHOT

202
postfix/src/global/memcache_proto.c Normal file
View File

@@ -0,0 +1,202 @@
/*++
/* NAME
/* memcache_proto 3
/* SUMMARY
/* memcache low-level protocol
/* SYNOPSIS
/* #include <memcache_proto.h>
/*
/* int memcache_get(fp, buf, len)
/* VSTREAM *fp;
/* VSTRING *buf;
/* ssize_t len;
/*
/* int memcache_printf(fp, format, ...)
/* VSTREAM *fp;
/* const char *format;
/*
/* int memcache_vprintf(fp, format, ap)
/* VSTREAM *fp;
/* const char *format;
/* va_list ap;
/*
/* int memcache_fread(fp, buf, len)
/* VSTREAM *fp;
/* VSTRING *buf;
/* ssize_t len;
/*
/* int memcache_fwrite(fp, buf, len)
/* VSTREAM *fp;
/* const char *buf;
/* ssize_t len;
/* DESCRIPTION
/* This module implements the low-level memcache protocol.
/* All functions return -1 on error and 0 on succcess.
/* SEE ALSO
/* smtp_proto(3) SMTP low-level protocol.
/* AUTHOR(S)
/* Wietse Venema
/* IBM T.J. Watson Research
/* P.O. Box 704
/* Yorktown Heights, NY 10598, USA
/*--*/
#include <sys_defs.h>
/* Utility library. */
#include <msg.h>
#include <vstream.h>
#include <vstring.h>
#include <vstring_vstream.h>
/* Application-specific. */
#include <memcache_proto.h>
#define STR(x) vstring_str(x)
#define LEN(x) VSTRING_LEN(x)
/* memcache_get - read one line from peer */
int memcache_get(VSTREAM *stream, VSTRING *vp, ssize_t bound)
{
int last_char;
int next_char;
last_char = (bound == 0 ? vstring_get(vp, stream) :
vstring_get_bound(vp, stream, bound));
switch (last_char) {
/*
* Do some repair in the rare case that we stopped reading in the
* middle of the CRLF record terminator.
*/
case '\r':
if ((next_char = VSTREAM_GETC(stream)) == '\n') {
VSTRING_ADDCH(vp, '\n');
/* FALLTRHOUGH */
} else {
if (next_char != VSTREAM_EOF)
vstream_ungetc(stream, next_char);
/*
* Input too long, or EOF
*/
default:
if (msg_verbose)
msg_info("%s got %s", VSTREAM_PATH(stream),
LEN(vp) < bound ? "EOF" : "input too long");
return (-1);
}
/*
* Strip off the record terminator: either CRLF or just bare LF.
*/
case '\n':
vstring_truncate(vp, VSTRING_LEN(vp) - 1);
if (VSTRING_LEN(vp) > 0 && vstring_end(vp)[-1] == '\r')
vstring_truncate(vp, VSTRING_LEN(vp) - 1);
VSTRING_TERMINATE(vp);
if (msg_verbose)
msg_info("%s got: %s", VSTREAM_PATH(stream), STR(vp));
return (0);
}
}
/* memcache_fwrite - write one blob to peer */
int memcache_fwrite(VSTREAM *stream, const char *cp, ssize_t todo)
{
/*
* Sanity check.
*/
if (todo < 0)
msg_panic("memcache_fwrite: negative todo %ld", (long) todo);
/*
* Do the I/O.
*/
if (msg_verbose)
msg_info("%s write: %.*s", VSTREAM_PATH(stream), (int) todo, cp);
if (vstream_fwrite(stream, cp, todo) != todo
|| vstream_fputs("\r\n", stream) == VSTREAM_EOF)
return (-1);
else
return (0);
}
/* memcache_fread - read one blob from peer */
int memcache_fread(VSTREAM *stream, VSTRING *buf, ssize_t todo)
{
/*
* Sanity check.
*/
if (todo < 0)
msg_panic("memcache_fread: negative todo %ld", (long) todo);
/*
* Do the I/O.
*/
VSTRING_SPACE(buf, todo);
VSTRING_AT_OFFSET(buf, todo);
if (vstream_fread(stream, STR(buf), todo) != todo
|| VSTREAM_GETC(stream) != '\r'
|| VSTREAM_GETC(stream) != '\n') {
if (msg_verbose)
msg_info("%s read: error", VSTREAM_PATH(stream));
return (-1);
} else {
vstring_truncate(buf, todo);
VSTRING_TERMINATE(buf);
if (msg_verbose)
msg_info("%s read: %s", VSTREAM_PATH(stream), STR(buf));
return (0);
}
}
/* memcache_vprintf - write one line to peer */
int memcache_vprintf(VSTREAM *stream, const char *fmt, va_list ap)
{
/*
* Do the I/O.
*/
vstream_vfprintf(stream, fmt, ap);
vstream_fputs("\r\n", stream);
if (vstream_ferror(stream))
return (-1);
else
return (0);
}
/* memcache_printf - write one line to peer */
int memcache_printf(VSTREAM *stream, const char *fmt,...)
{
va_list ap;
int ret;
if (msg_verbose) {
VSTRING *buf = vstring_alloc(100);
va_start(ap, fmt);
vstring_vsprintf(buf, fmt, ap);
va_end(ap);
msg_info("%s write: %s", VSTREAM_PATH(stream), STR(buf));
vstring_free(buf);
}
/*
* Do the I/O.
*/
va_start(ap, fmt);
ret = memcache_vprintf(stream, fmt, ap);
va_end(ap);
return (ret);
}

34
postfix/src/global/memcache_proto.h Normal file
View File

@@ -0,0 +1,34 @@
#ifndef _MEMCACHE_PROTO_H_INCLUDED_
#define _MEMCACHE_PROTO_H_INCLUDED_
/*++
/* NAME
/* memcache_proto 3h
/* SUMMARY
/* memcache low-level protocol
/* SYNOPSIS
/* #include <memcache_proto.h>
/* DESCRIPTION
/* .nf
/*
* External interface.
*/
extern int memcache_get(VSTREAM *, VSTRING *, ssize_t);
extern int memcache_vprintf(VSTREAM *, const char *, va_list);
extern int memcache_printf(VSTREAM *, const char *fmt,...);
extern int memcache_fread(VSTREAM *, VSTRING *, ssize_t);
extern int memcache_fwrite(VSTREAM *, const char *, ssize_t);
/* LICENSE
/* .ad
/* .fi
/* The Secure Mailer license must be distributed with this software.
/* AUTHOR(S)
/* Wietse Venema
/* IBM T.J. Watson Research
/* P.O. Box 704
/* Yorktown Heights, NY 10598, USA
/*--*/
#endif

70
postfix/src/proxymap/proxymap.c
View File

@@ -69,6 +69,13 @@
/* as with the \fBopen\fR request. /* as with the \fBopen\fR request.
/* .sp /* .sp
/* This request is supported in Postfix 2.5 and later. /* This request is supported in Postfix 2.5 and later.
/* .IP "\fBsequence\fR \fImaptype:mapname flags function\fR"
/* Iterate over the specified database. The \fIfunction\fR
/* is one of DICT_SEQ_FUN_FIRST or DICT_SEQ_FUN_NEXT.
/* The reply is the request completion status code and
/* a lookup key and result value, if found.
/* .sp
/* This request is supported in Postfix 2.9 and later.
/* .PP /* .PP
/* The request completion status is one of OK, RETRY, NOKEY /* The request completion status is one of OK, RETRY, NOKEY
/* (lookup failed because the key was not found), BAD (malformed /* (lookup failed because the key was not found), BAD (malformed
@@ -235,6 +242,7 @@
* aren't too broken. The fix is to gather all parameter default settings in * aren't too broken. The fix is to gather all parameter default settings in
* one place. * one place.
*/ */
char *var_alias_maps;
char *var_local_rcpt_maps; char *var_local_rcpt_maps;
char *var_virt_alias_maps; char *var_virt_alias_maps;
char *var_virt_alias_doms; char *var_virt_alias_doms;
@@ -329,6 +337,54 @@ static DICT *proxy_map_find(const char *map_type_name, int request_flags,
return (dict); return (dict);
} }
/* proxymap_sequence_service - remote sequence service */
static void proxymap_sequence_service(VSTREAM *client_stream)
{
int request_flags;
DICT *dict;
int request_func;
const char *reply_key;
const char *reply_value;
int reply_status;
/*
* Process the request.
*/
if (attr_scan(client_stream, ATTR_FLAG_STRICT,
ATTR_TYPE_STR, MAIL_ATTR_TABLE, request_map,
ATTR_TYPE_INT, MAIL_ATTR_FLAGS, &request_flags,
ATTR_TYPE_INT, MAIL_ATTR_FUNC, &request_func,
ATTR_TYPE_END) != 3
|| (request_func != DICT_SEQ_FUN_FIRST
&& request_func != DICT_SEQ_FUN_NEXT)) {
reply_status = PROXY_STAT_BAD;
reply_key = reply_value = "";
} else if ((dict = proxy_map_find(STR(request_map), request_flags,
&reply_status)) == 0) {
reply_key = reply_value = "";
} else if (dict->flags = ((dict->flags & ~DICT_FLAG_RQST_MASK)
| (request_flags & DICT_FLAG_RQST_MASK)),
dict_seq(dict, request_func, &reply_key, &reply_value) == 0) {
reply_status = PROXY_STAT_OK;
} else if (dict_errno == 0) {
reply_status = PROXY_STAT_NOKEY;
reply_key = reply_value = "";
} else {
reply_status = PROXY_STAT_RETRY;
reply_key = reply_value = "";
}
/*
* Respond to the client.
*/
attr_print(client_stream, ATTR_FLAG_NONE,
ATTR_TYPE_INT, MAIL_ATTR_STATUS, reply_status,
ATTR_TYPE_STR, MAIL_ATTR_KEY, reply_key,
ATTR_TYPE_STR, MAIL_ATTR_VALUE, reply_value,
ATTR_TYPE_END);
}
/* proxymap_lookup_service - remote lookup service */ /* proxymap_lookup_service - remote lookup service */
static void proxymap_lookup_service(VSTREAM *client_stream) static void proxymap_lookup_service(VSTREAM *client_stream)
@@ -407,8 +463,9 @@ static void proxymap_update_service(VSTREAM *client_stream)
dict->flags = ((dict->flags & ~DICT_FLAG_RQST_MASK) dict->flags = ((dict->flags & ~DICT_FLAG_RQST_MASK)
| (request_flags & DICT_FLAG_RQST_MASK) | (request_flags & DICT_FLAG_RQST_MASK)
| DICT_FLAG_SYNC_UPDATE | DICT_FLAG_DUP_REPLACE); | DICT_FLAG_SYNC_UPDATE | DICT_FLAG_DUP_REPLACE);
dict_errno = 0;
dict_put(dict, STR(request_key), STR(request_value)); dict_put(dict, STR(request_key), STR(request_value));
reply_status = PROXY_STAT_OK; reply_status = (dict_errno ? PROXY_STAT_RETRY : PROXY_STAT_OK);
} }
/* /*
@@ -425,6 +482,7 @@ static void proxymap_delete_service(VSTREAM *client_stream)
{ {
int request_flags; int request_flags;
DICT *dict; DICT *dict;
int dict_status;
int reply_status; int reply_status;
/* /*
@@ -450,8 +508,11 @@ static void proxymap_delete_service(VSTREAM *client_stream)
dict->flags = ((dict->flags & ~DICT_FLAG_RQST_MASK) dict->flags = ((dict->flags & ~DICT_FLAG_RQST_MASK)
| (request_flags & DICT_FLAG_RQST_MASK) | (request_flags & DICT_FLAG_RQST_MASK)
| DICT_FLAG_SYNC_UPDATE); | DICT_FLAG_SYNC_UPDATE);
reply_status = dict_errno = 0;
dict_del(dict, STR(request_key)) ? PROXY_STAT_OK : PROXY_STAT_NOKEY; dict_status = dict_del(dict, STR(request_key));
reply_status = (dict_status == 0 ? PROXY_STAT_OK :
dict_status > 0 ? PROXY_STAT_NOKEY :
PROXY_STAT_RETRY);
} }
/* /*
@@ -524,6 +585,8 @@ static void proxymap_service(VSTREAM *client_stream, char *unused_service,
proxymap_update_service(client_stream); proxymap_update_service(client_stream);
} else if (VSTREQ(request, PROXY_REQ_DELETE)) { } else if (VSTREQ(request, PROXY_REQ_DELETE)) {
proxymap_delete_service(client_stream); proxymap_delete_service(client_stream);
} else if (VSTREQ(request, PROXY_REQ_SEQUENCE)) {
proxymap_sequence_service(client_stream);
} else if (VSTREQ(request, PROXY_REQ_OPEN)) { } else if (VSTREQ(request, PROXY_REQ_OPEN)) {
proxymap_open_service(client_stream); proxymap_open_service(client_stream);
} else { } else {
@@ -620,6 +683,7 @@ MAIL_VERSION_STAMP_DECLARE;
int main(int argc, char **argv) int main(int argc, char **argv)
{ {
static const CONFIG_STR_TABLE str_table[] = { static const CONFIG_STR_TABLE str_table[] = {
VAR_ALIAS_MAPS, DEF_ALIAS_MAPS, &var_alias_maps, 0, 0,
VAR_LOCAL_RCPT_MAPS, DEF_LOCAL_RCPT_MAPS, &var_local_rcpt_maps, 0, 0, VAR_LOCAL_RCPT_MAPS, DEF_LOCAL_RCPT_MAPS, &var_local_rcpt_maps, 0, 0,
VAR_VIRT_ALIAS_MAPS, DEF_VIRT_ALIAS_MAPS, &var_virt_alias_maps, 0, 0, VAR_VIRT_ALIAS_MAPS, DEF_VIRT_ALIAS_MAPS, &var_virt_alias_maps, 0, 0,
VAR_VIRT_ALIAS_DOMS, DEF_VIRT_ALIAS_DOMS, &var_virt_alias_doms, 0, 0, VAR_VIRT_ALIAS_DOMS, DEF_VIRT_ALIAS_DOMS, &var_virt_alias_doms, 0, 0,

4
postfix/src/util/dict.c
View File

@@ -107,7 +107,9 @@
/* modified, or if the result is to survive multiple dict_lookup() calls. /* modified, or if the result is to survive multiple dict_lookup() calls.
/* /*
/* dict_delete() removes the named member from the named dictionary. /* dict_delete() removes the named member from the named dictionary.
/* The result value is zero when the member was found. /* The result value is zero when the member was found, > 0 if
/* it was not found, and < 0 in case of error (a database may
/* not return after error).
/* /*
/* dict_sequence() steps through the named dictionary and returns /* dict_sequence() steps through the named dictionary and returns
/* keys and values in some implementation-defined order. The func /* keys and values in some implementation-defined order. The func

6
postfix/src/util/dict_test.c
View File

@@ -87,7 +87,7 @@ void dict_test(int argc, char **argv)
if (*bufp == '#') if (*bufp == '#')
continue; continue;
if ((cmd = mystrtok(&bufp, " ")) == 0) { if ((cmd = mystrtok(&bufp, " ")) == 0) {
vstream_printf("usage: del key|get key|put key=value|first|next\n"); vstream_printf("usage: verbose|del key|get key|put key=value|first|next\n");
vstream_fflush(VSTREAM_OUT); vstream_fflush(VSTREAM_OUT);
continue; continue;
} }
@@ -95,7 +95,9 @@ void dict_test(int argc, char **argv)
msg_warn("dictionary has changed"); msg_warn("dictionary has changed");
key = *bufp ? vstring_str(unescape(keybuf, mystrtok(&bufp, " ="))) : 0; key = *bufp ? vstring_str(unescape(keybuf, mystrtok(&bufp, " ="))) : 0;
value = mystrtok(&bufp, " ="); value = mystrtok(&bufp, " =");
if (strcmp(cmd, "del") == 0 && key && !value) { if (strcmp(cmd, "verbose") == 0 && !key) {
msg_verbose++;
} else if (strcmp(cmd, "del") == 0 && key && !value) {
if (dict_del(dict, key)) if (dict_del(dict, key))
vstream_printf("%s: not found\n", key); vstream_printf("%s: not found\n", key);
else else