Originally published March 1, 2019 @ 11:54 am
Most log files do not contain personally-identifiable information or other sensitive data. And even if they do, encryption of all personal data is not mandatory under GDPR. Still, on occasion, for testing and troubleshooting purposes you may want to log potentially sensitive information. It would be a very good idea not to let these logs get away from you.
You can certainly run the encryption process separately from log rotation via cron or some such. But this would be an extra step with extra problems. You can actually use logrotate directly to handle encryption. Basically, you will tell logrotate use gpg instead of the usual gzip.
Here’s an example of my /etc/logrotate.d/squid:
Expand
/var/log/squid/*.log {
daily
rotate 5
compress
compresscmd /usr/bin/gpg
compressoptions --always-trust --encrypt --recipient your_email@domain.com
compressext .gpg
notifempty
missingok
sharedscripts
postrotate
# Asks squid to reopen its logs. (log_rotate 0 is set in squid.conf)
# errors redirected to make it silent if squid is not running
/usr/sbin/squid -k rotate 2>/dev/null
# Wait a little to allow Squid to catch up before the logs is compressed
sleep 1
endscript
} Older kernels warning
Expand
Kernels with logrotate version prior to 3.8 have an issue with the compressoptions not taking spaces. There’s a simple workaround, though: create a script that contains the command (such us gpg) and all the options and put it in /usr/bin/gpg-logrotate (don’t forget to make it executable, obviously):
#!/bin/bash
/usr/bin/gpg --always-trust --encrypt --recipient your_email@domain.com
However, since log rotation runs via cron you would either need to source root profile (along with GPG keys), or temporarily extract the relevant GPG key ID to be used for encryption. From a security standpoint, the latter is safer. Here’s what your gpg-logrotate script may look like in this case:
#!/bin/bash
d=$(mktemp -d)
/usr/bin/gpg --homedir ${d} --import /root/.gnupg/your_public_key.pub
keyid="$(/usr/bin/gpg --homedir=${d} --keyid-format 0xLONG -k | grep -m1 ^pub | awk -F'/' '{print $2}' | awk '{print $1}')"
/usr/bin/gpg --homedir=${d} --always-trust --encrypt --recipient ${keyid}
/bin/rm -rf ${d}
And then the logrotate config file from the example above would look like this:
/var/log/squid/*.log {
daily
rotate 5
compress
compresscmd /usr/bin/gpg-logrotate
compressext .gpg
notifempty
missingok
sharedscripts
postrotate
# Asks squid to reopen its logs. (log_rotate 0 is set in squid.conf)
# errors redirected to make it silent if squid is not running
/usr/sbin/squid -k rotate 2>/dev/null
# Wait a little to allow Squid to catch up before the logs is compressed
sleep 1
endscript
} Following log rotation, you should see something like this in the log destination:
ls -als /var/log/squid
total 232
4 drwxr-x--- 2 squid squid 4096 Feb 28 16:06 .
4 drwxr-xr-x. 27 root root 4096 Feb 24 03:10 ..
204 -rw-r----- 1 squid squid 202423 Feb 28 16:56 access.log
4 -rw-r----- 1 squid squid 1535 Feb 28 16:06 access.log.1.gpg
8 -rw-r----- 1 squid squid 6238 Feb 28 16:09 cache.log
4 -rw-r----- 1 squid squid 519 Feb 28 16:06 cache.log.1.gpg
4 -rw-r--r-- 1 root root 187 Feb 6 11:06 squid.out
Log rotation frequency
By default, log rotation process would run once a day. This is fine if you specified daily or weekly in your config. If you want to run the process hourly, you would need to copy /etc/cron.daily/logrotate into /etc/cron.hourly/ directory. Naturally, in your logrotate config file you would also need to specify hourly with the desired retention period.
And, I guess, a couple of words on how to set up gpg and keys. If you have an existing key, copy the public portion of it to your server and add it to the keyring:
gpg --import F656FEB192A41CA82B6F73834011ACA1D7833129.asc
You can then list all the keys on your server:
gpg --list-keys
/root/.gnupg/pubring.gpg
------------------------
pub 2048R/5303E22B 2017-07-14
uid Your Name <your_email@domain.com>
sub 2048R/8A10B563 2017-07-14
If you don’t already have a key, you should create a passphrase-protected key. It is also a good idea not to keep the private half of the key on the server.
Generating new key
In the following example we create a new passphrase-protected non-expiring private key.
cd ~ && gpg --gen-key
gpg (GnuPG) 2.0.14; Copyright (C) 2009 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 2
DSA keys may be between 1024 and 3072 bits long.
What keysize do you want? (2048) 4096
DSA keysizes must be in the range 1024-3072
What keysize do you want? (2048) 3072
Requested keysize is 3072 bits
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y
GnuPG needs to construct a user ID to identify your key.
Real name: Your Name
Email address: your_email@domain.com
Comment: delete me
You selected this USER-ID:
"Your Name (delete me) <your_email@domain.com>"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
You need a Passphrase to protect your secret key.
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: WARNING: some OpenPGP programs can't handle a DSA key with this digest size
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: key AADDBFF5 marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 2 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 2u
pub 3072D/AADDBFF5 2019-02-28
Key fingerprint = FE29 2FF3 8653 F2C2 4B9A B9C5 FE39 CF03 AADD BFF5
uid Your Name (delete me) <your_email@domain.com>
sub 3072g/6DACF5EB 2019-02-28
You see the key you just created:
gpg --list-secret-keys
/root/.gnupg/secring.gpg
------------------------
sec 3072D/AADDBFF5 2019-02-28
uid Your Name (delete me) <your_email@domain.com>
ssb 3072g/6DACF5EB 2019-02-28
As I mentioned, you shouldn’t be keeping your private key on the server. Generate the key on a less-exposed machine, extract the public key, and import it on your server. Here’s an example:
Expand
# Generate passphrase-protected private key as shown above
# on a secure machine
# Extract the public key
gpg --armor --export your_email@domain.com > ~/your_email@domain.com.asc
# Move the public key to the server and import it
gpg --import ~/your_email@domain.com.asc
Experienced Unix/Linux System Administrator with 20-year background in Systems Analysis, Problem Resolution and Engineering Application Support in a large distributed Unix and Windows server environment. Strong problem determination skills. Good knowledge of networking, remote diagnostic techniques, firewalls and network security. Extensive experience with engineering application and database servers, high-availability systems, high-performance computing clusters, and process automation.
