[AusNOG] Data Retention - are you kidding me??

[russell at central-data.net]

Another option in my case postfix you can modify the config to you get one nice line in the logs which makes it very easy to search and pull out reports

In postfix.conf add 
header_checks = regexp:/etc/postfix/header_checks

in header_checks add
/^Subject:/      WARN

This will put a single line in your logs like the following

warning: header Subject: "Subject line"[src ip addr]; from=<src at address.com> to=<dst at address.com> proto=ESMTP helo=<src server>




Kind Regards, 

Russell Brooks 


Central Data Systems Pty Ltd 
88 Havelock Street, West Perth, WA 6005 
Phone: 08 9481 4010 

www.central-data.net 

----- Original Message -----
From: "Ross Wheeler" <ausnog at rossw.net>
To: "Mike O'Connor" <mike at pineview.net>
Cc: "<ausnog at lists.ausnog.net>" <ausnog at lists.ausnog.net>
Sent: Tuesday, November 22, 2016 10:46:41 AM
Subject: Re: [AusNOG] Data Retention - are you kidding me??



On Tue, 22 Nov 2016, Mike O'Connor wrote:

> On 17/11/2016 10:28 AM, Ross Wheeler wrote:

> My problem is I have no idea how I would filter the results to just one 
> customer say in the case of mail logs. I was looking at the output of our 
> mail server and the details are split over a number of lines and in most 
> cases the information is not there.

Certainly in the case of sendmail (and I suspect most MTAs), it would 
require a small amount of scripting - but still easy enough to achieve.

An example:

# grep mike at pineview.net /var/log/maillog

Nov 22 12:55:45 ali-syd-1 sm-mta[76593]: uAM1tibu076593:
    from=<mike at pineview.net>, size=1534, class=0, nrcpts=1,
    msgid=<dccb7790-48e1-fccd-0b44-776623461f1e at pineview.net>,
    bodytype=8BITMIME, proto=ESMTP, daemon=IPv4, relay=mail.pineview.net
    [203.33.246.11]

The "uAM1tibu076593" identifier is unique for this transaction.
A second pass will find all the lines for this transaction, including 
lines that didn't contain the original search term (mike@)


# grep uAM1tibu076593 /var/log/maillog

Nov 22 12:55:45 ali-syd-1 milter-greylist: uAM1tibu076593: skipping
    greylist because recipient <ausnog at rossw.net> is whitelisted,
    (from=<mike at pineview.net>, rcpt=<ausnog at rossw.net>,
    addr=mail.pineview.net[203.33.246.11])

Nov 22 12:55:45 ali-syd-1 sm-mta[76593]: uAM1tibu076593:
    from=<mike at pineview.net>, size=1534, class=0, nrcpts=1,
    msgid=<dccb7790-48e1-fccd-0b44-776623461f1e at pineview.net>,
    bodytype=8BITMIME, proto=ESMTP, daemon=IPv4, relay=mail.pineview.net
    [203.33.246.11]

Nov 22 12:55:45 ali-syd-1 sm-mta[76593]: uAM1tibu076593: Milter add:
    header: X-Greylist: Recipient e-mail whitelisted, not delayed by
    milter-greylist-3.0 (mail.albury.net.au [202.3.36.15]); Tue, 22 Nov
    2016 12:55:45 +1100 (EST)

Nov 22 12:55:45 ali-syd-1 sm-mta[76724]: uAM1tibu076593:
    to=*******,*******, delay=00:00:00, xdelay=00:00:00, mailer=local,
    pri=61930, relay=local, dsn=2.0.0, stat=Sent



The process of finding all the unique IDs for mail to or from a given user 
is quite straightforward using CLI tools available on every system I know 
of that you're likely to be running a mail server on :)

Eg, a totally brute-force and ugly version:

for id in `grep "whoever at example.com" /var/log/maillog|awk '{print $6}'|sort|uniq`; do grep $id /var/log/maillog; done

_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net
http://lists.ausnog.net/mailman/listinfo/ausnog