[AusNOG] Data Retention - are you kidding me??

Ross Wheeler ausnog at rossw.net
Tue Nov 22 13:46:41 EST 2016



On Tue, 22 Nov 2016, Mike O'Connor wrote:

> On 17/11/2016 10:28 AM, Ross Wheeler wrote:

> My problem is I have no idea how I would filter the results to just one 
> customer say in the case of mail logs. I was looking at the output of our 
> mail server and the details are split over a number of lines and in most 
> cases the information is not there.

Certainly in the case of sendmail (and I suspect most MTAs), it would 
require a small amount of scripting - but still easy enough to achieve.

An example:

# grep mike at pineview.net /var/log/maillog

Nov 22 12:55:45 ali-syd-1 sm-mta[76593]: uAM1tibu076593:
    from=<mike at pineview.net>, size=1534, class=0, nrcpts=1,
    msgid=<dccb7790-48e1-fccd-0b44-776623461f1e at pineview.net>,
    bodytype=8BITMIME, proto=ESMTP, daemon=IPv4, relay=mail.pineview.net
    [203.33.246.11]

The "uAM1tibu076593" identifier is unique for this transaction.
A second pass will find all the lines for this transaction, including 
lines that didn't contain the original search term (mike@)


# grep uAM1tibu076593 /var/log/maillog

Nov 22 12:55:45 ali-syd-1 milter-greylist: uAM1tibu076593: skipping
    greylist because recipient <ausnog at rossw.net> is whitelisted,
    (from=<mike at pineview.net>, rcpt=<ausnog at rossw.net>,
    addr=mail.pineview.net[203.33.246.11])

Nov 22 12:55:45 ali-syd-1 sm-mta[76593]: uAM1tibu076593:
    from=<mike at pineview.net>, size=1534, class=0, nrcpts=1,
    msgid=<dccb7790-48e1-fccd-0b44-776623461f1e at pineview.net>,
    bodytype=8BITMIME, proto=ESMTP, daemon=IPv4, relay=mail.pineview.net
    [203.33.246.11]

Nov 22 12:55:45 ali-syd-1 sm-mta[76593]: uAM1tibu076593: Milter add:
    header: X-Greylist: Recipient e-mail whitelisted, not delayed by
    milter-greylist-3.0 (mail.albury.net.au [202.3.36.15]); Tue, 22 Nov
    2016 12:55:45 +1100 (EST)

Nov 22 12:55:45 ali-syd-1 sm-mta[76724]: uAM1tibu076593:
    to=*******,*******, delay=00:00:00, xdelay=00:00:00, mailer=local,
    pri=61930, relay=local, dsn=2.0.0, stat=Sent



The process of finding all the unique IDs for mail to or from a given user 
is quite straightforward using CLI tools available on every system I know 
of that you're likely to be running a mail server on :)

Eg, a totally brute-force and ugly version:

for id in `grep "whoever at example.com" /var/log/maillog|awk '{print $6}'|sort|uniq`; do grep $id /var/log/maillog; done



More information about the AusNOG mailing list