Vsevolod Stakhov, Notes

Rspamd

2009-04-13T16:48:58Z

First public version of rspamd is available for beta-testing. Rspamd is fast, modular and lightweight spam filter. It is designed to work with big ammount of mail and can be easily extended with own filters written in lua or perl.
Rspamd can be found here: http://cebka.pp.ru/trac

2008-08-12T14:23:57Z

Here is mercurial repository of some of my software.
- http://cebka.pp.ru/hg/rmilter/ - mail filter for sendmail/postfix, that can check mail with spamassassin/clamav/dcc/spf, check limits and do greylisting (via memcached), and check regexps.
- http://cebka.pp.ru/hg/nginx-smtp-policy/ - async (libevent based) DNS resolver with patch to nginx (versions up to 0.6.11) mail module (originnally written by Max Dounin http://mdounin.ru/) that can do load balancing of incoming mail (weight based) check rbls and monitor state of mx's (with nginx-smtp-watchdog). Nginx-smtp-policy designed for huge mail traffic and can help to low load on backend mx's.
- http://cebka.pp.ru/hg/libevent/ - libevent patches for faster DNS search (hashed list instead of linear list) and for allowing TXT requests (for RBL checks).

Migration from jabberd2 to openfire

2008-02-05T17:26:23Z

Here is script that allows to convert jabberd2 database items to xml format that can be used by openfire import/export plugin.

jabber_to_xml.pl

Memcached UDP

2007-12-13T14:37:22Z

This patch fixes memcached support of udp protocol. Now UDP packets with wrong header can break memcached server down. Also I've made some fixes in cleaning UDP sessions after closing.
Results of testing (UDP vs TCP):

UDP:
Results of memcached stress test:
Total number of connections: 10000
Number of seconds for test: 2.23
Number of successfull connections: 10000
Connections per second: 4494.25
TCP:
Results of memcached stress test:
Total number of connections: 10000
Number of seconds for test: 4.38
Number of successfull connections: 10000
Connections per second: 2280.87

patch-memcached-udp

XCLIENT in Exim

2007-12-12T14:51:01Z

I wrote small patch for exim 4.68 that allows using of XCLIENT command in exim. The description of this command you can find here: http://www.postfix.org/XCLIENT_README.html
By default xclient is not allowed from any hosts, but you can specify hosts in option xclient_allow_hosts in configuration file:

xclient_allow_hosts = 127.0.0.1 : 192.168.1.1

Example of smtp dialog with XCLIENT:
Connected to localhost.
Escape character is '^]'.
220 dhcp-ng2 ESMTP Exim 4.68 Mon, 10 Dec 2007 19:26:44 +0300
XCLIENT NAME=spike.porcupine.org ADDR=168.100.189.2 HELO=blah
220 XCLIENT success
MAIL FROM:
250 OK
RCPT TO:
550 relay not permitted

patch-exim-xclient

Determining order of passing packets throught packet filters in FreeBSD

2007-12-12T14:50:01Z

The order of loading kernel modules of packet filters affects the order of registering hooks in pfil interface.
So this code shows how pfil calls specified hook:

for (pfh = pfil_hook_get(dir, ph);
        pfh != NULL;
        pfh = TAILQ_NEXT(pfh, pfil_link)) {
            if (pfh->pfil_func != NULL) {
                rv = (*pfh->pfil_func)(pfh->pfil_arg, &m, ifp, dir, inp);
                if (rv != 0 || m == NULL)
                       break;
            }
}

So we check all hooks begining from TAILQ_FIRST and pass packet to each hook as parameter.
If hook drops packet, we do not check other hooks.

This code register hooks in pfil interface:

    /*
     * insert the input list in reverse order of the output list
     * so that the same path is followed in or out of the kernel.
     */
    if (flags & PFIL_IN)
        TAILQ_INSERT_HEAD(list, pfh1, pfil_link);
    else
        TAILQ_INSERT_TAIL(list, pfh1, pfil_link);

So for inbound traffic module that adds its hook last would process packets first, and for outbound traffic module that adds its hook first would process packets first.
Now let's determine load order of kernel modules.
Modules on load attach to subsystems, that are described in sys/kernel.h:

    SI_SUB_PROTO_IF     = 0x8400000,    /* interfaces*/
    SI_SUB_PROTO_DOMAIN = 0x8800000,    /* domains (address families?)*/
    SI_SUB_PROTO_IFATTACHDOMAIN = 0x8800001,    /* domain dependent data init*/
Subsystems are loading in order of their number, first are loaded subsystems with lesser number.
Module on load can be attached to subsystem in different module too, this order is declared in macro DECLARE_MODULE:

enum sysinit_elem_order {
    SI_ORDER_FIRST      = 0x0000000,    /* first*/
    SI_ORDER_SECOND     = 0x0000001,    /* second*/
    SI_ORDER_THIRD      = 0x0000002,    /* third*/
    SI_ORDER_MIDDLE     = 0x1000000,    /* somewhere in the middle */
    SI_ORDER_ANY        = 0xfffffff /* last*/
};
So we should check subsystem of module and order of attachement to subsystem for each packet filter module:
pf:
DECLARE_MODULE(pf, pf_mod, SI_SUB_PROTO_IFATTACHDOMAIN, SI_ORDER_FIRST);
ipfw:
DECLARE_MODULE(ipfw, ipfwmod, SI_SUB_PROTO_IFATTACHDOMAIN, SI_ORDER_ANY);
ipfilter:
DECLARE_MODULE(ipfilter, ipfiltermod, SI_SUB_PROTO_DOMAIN, SI_ORDER_ANY);

Subsystems loads in this order: SI_SUB_PROTO_DOMAIN -> SI_SUB_PROTO_IFATTACHDOMAIN (ipfilter -> (ipfw, pf)), according to order of attachement of modules to subsystems the chain of loading must look like ipfilter -> pf -> ipfw.
Process order for inbound traffic:
ipfw -> pf -> ipfilter -> stack
and for outbound:
stack -> ipfilter -> pf -> ipfw

On the other hand ps is often loaded as kld, and is not statically compiled in the kernel, so if you can see pf.ko in loaded modules list (kldstat), pf is loaded after ipfw, not before. So traffic pass chains must look like this:
pf -> ipfw -> ipfilter -> stack - for inbound
stack -> ipfilter -> ipfw -> pf - for outbound

The order of passing trafic throught packet filters is very important in case of NAT, as next filter in chain can receive modified packet.