<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap:break-word"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Late to the conversation.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">There’s been great replies, I’d just like to add to these if I can.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Yes compress prior to encrypt and sending but also when looking at compression you mentioned log-files so if these are large and in a standard format (i.e traffic logs/apache/nginx etc etc) it’s worth looking at compression with a specific specification of deduplication.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Many standard compression formats deal with compressing text files by removing whitespace and converting it to a hashing sequence but fail to do dedup’ing. </div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">I’m sure many on here know what deduplication is (eg. many sans feature it) but if you don’t read up on wikipedia: <a href="https://en.wikipedia.org/wiki/Data_deduplication">https://en.wikipedia.org/wiki/Data_deduplication</a></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Whilst you can save considerable transmission and storage space with just removing the excess whitespace in a text file you can save incredible amounts of space by indexing and removing multiple sequences of data that repeat.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Ultimately it really depends on your requirements and how much you want to get this down by. Best of luck! </div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">—Karl.</div> <br> <div id="bloop_sign_1479540578295293952" class="bloop_sign">Kind Regards,<div>Karl Kloppenborg</div><div><br></div></div> <br><p class="airmail_on">On 18 November 2016 at 5:18:23 pm, Mark Smith (<a href="mailto:markzzzsmith@gmail.com">markzzzsmith@gmail.com</a>) wrote:</p> <blockquote type="cite" class="clean_bq"><span><div><div></div><div>
<title></title>
<p dir="ltr">Compress and then encrypt. If your encryption is any
good there will be no repeating patterns worth compressing.</p>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 18 Nov. 2016 16:29, "Ross Wheeler"
<<a href="mailto:ausnog@rossw.net">ausnog@rossw.net</a>>
wrote:<br type="attribution">
<blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Esteemed geeks, I seek your considered input.<br>
<br>
As part of "that which we may not discuss", I am seeking arguments
as to precedence.<br>
<br>
If I have a plain-text log file on machine (a), that I wish to
store securely for some period on machine (b), is it better to
encrypt the file first and then compress it, or to compress it
first and then encrypt it?<br>
Either way, it will be encrypted before it leaves machine (a) over
an ssh link, and will be stored in the (compressed and encrypted)
form on the storage device.<br>
<br>
I think compression is likely to be vastly superior for text files
than binary files, so compression first, then encrypt the (binary)
file, and indeed a couple of quick tests shows files are 10-15
times larger if I encrypt first. (330KB vs 5500KB, 125KB vs
1611KB). Given there will be a lot of files to copy daily, the
savings in transmission time, storage and possibly CPU (must be
easier to encrypt a small file than a large one), this looks like
the clear winner.<br>
<br>
I'm not sure there's much difference either way when extracting
data, but if anyone knows of a pitfall in this process, I'd rather
hear about it before I get too far down the road!<br>
<br>
Thanks guys, and have a good weekend... it's almost beer
oclock!<br>
<br>
R.<br>
______________________________<wbr>_________________<br>
AusNOG mailing list<br>
<a href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
<a href="http://lists.ausnog.net/mailman/listinfo/ausnog" rel="noreferrer" target="_blank">http://lists.ausnog.net/mailma<wbr>n/listinfo/ausnog</a><br>
</blockquote>
</div>
<br></div>
_______________________________________________
<br>AusNOG mailing list
<br><a href="mailto:AusNOG@lists.ausnog.net">AusNOG@lists.ausnog.net</a>
<br><a href="http://lists.ausnog.net/mailman/listinfo/ausnog">http://lists.ausnog.net/mailman/listinfo/ausnog</a>
<br></div></div></span></blockquote></body></html>