Renaming Files with exiftool

Originally published October 28, 2020 @ 5:44 pm

Recently, I exported a whole bunch of files from Lightroom to a NAS share, and the filenames like 2E570434-67B7E0489CA2-39354-000017CF24DD8ACD.jpg are not very informative. It would’ve been nice if the filename contained some useful information like date, camera model, and maybe even location.

The exiftool by Phil Harvey has been described as the Swiss Army knife for file metadata manipulation, and it certainly is that. For my purposes, I only needed a small subset of the functionality offered by exiftool (which, for some reason, I keep misspelling as ‘exitfool’…)

Anyway, I needed the filename to contain the timestamp when the photo was originally taken, the camera model (as short as possible – just enough for me to identify the equipment I used), and the location where the photo was taken (provided the camera supported geotagging).

Now, since multiple photos could’ve been taken at roughly the same time, with the same camera, and at the same location, the filename would also need to contain some sort of numerical auto-incremental field. Here’s an example of my “perfect” filename for photos:

20200822-1845-000-philadelphia_pa_us-xt3.jpg

I thought including city, state, and country in the filename was sufficient for my needs. You don’t really want to make that filename too long.

The original filename contained no useful information, so I did not want it to be a part of the new filename. Having said that, I thought it might be a good idea to add the original filename as an Exif tag just for record-keeping purposes, and so I can find it in the Lightroom catalog, should the need arise.

It should be noted that adding a tag to a file requires rewriting the entire file, which adds quite a bit of time to the renaming process. Feel free to drop this step if the original filename is not something you care to remember. And, by the way, if you want to see the available tags in a file, use this command: exiftool -a -G1 -s <FILENAME>

# Add a tag to the file with that file's filename
exiftool -P -overwrite_original_in_place '-XMP-xmpMM:PreservedFileName<${filename;s/\.[^.]*$//}' "${file_name}"

# Same as above but for all files in the current folder
exiftool -P -overwrite_original_in_place '-XMP-xmpMM:PreservedFileName<${filename;s/\.[^.]*$//}' .

# Same as above but for all file in the current folder and all subfolders
exiftool -P -overwrite_original_in_place '-XMP-xmpMM:PreservedFileName<${filename;s/\.[^.]*$//}' -r .

If your camera does not support geotagging or if you just don’t care to include location information in the filename, the renaming syntax is simple:

# Rename based on time and camera model. Do not preserve original filename
exiftool '-filename<${CreateDate}_${model;s/[- ]//g;tr/A-Z/a-z/}.%le' -d '%Y%m%d-%H%M%%-03.c' .

# Sample new filename:
20200813-1826-000_xt3.jpg

# Same as above but with the original filename included
# (but converted to lower-case)
exiftool '-filename<${CreateDate}_${model;s/[- ]//g;tr/A-Z/a-z/}_${filename;tr/A-Z/a-z/}' -d '%Y%m%d-%H%M%%-03.c' .

# Sample new filename:
20200813-1826-000_xt3_8f582e2b-3da0a2bf05d0-29166-000011acb22d2b3a.jpg

Now we get to the more complicated part: geotagging. You can extract the GPS coordinates from the photo using exiftool like so:

exiftool -q -m -n -p '$GPSLatitude,$GPSLongitude' "${file_name}"

# Sample output
38.3312111111112,-76.4904321111112

Converting latitude and longitude to a geographic name requires some sort of geolocation database. Unless you have one handy, I suggest you obtain an API key from one of the providers. Google is a popular choice, but I found Geocodio an easier process. With a free account, you can do up to 2,500 daily lookups, and you don’t need to provide payment information or any personal details when you sign up for a free account.

Here’s a sample reverse geolocation lookup where you send the GPS coordinates, and the API responds with the geographic name for that location:

# Using free API from https://dash.geocod.io API key 
# up to 2,500 free API queries per day
api_key='*************************************'
v='v1.6' # The version of the API
apibase="https://api.geocod.io/${v}"

# Here I am using `jq` to extract just the City, State, and Country fields
# and converting them to lowercase.
# I am also replacing spaces with underscores and removing any oddball
# characters that should not be a part of a filename. You can sanitize
# the filename with sed:
curl -s0 -q -k "${apibase}/reverse?q=38.3312111111112,-76.4904321111112&api_key=${api_key}&limit=1" | \
jq -r '.results[]|.address_components|"\(.city) \(.state) \(.country)"' 2>/dev/null | \
sed -e 's/\(.*\)/\L/' -e 's/[^A-Za-z0-9._-]/_/g'

# ... or with detox (apt install detox):
curl -s0 -q -k "${apibase}/reverse?q=38.3312111111112,-76.4904321111112&api_key=${api_key}&limit=1" | \
jq -r '.results[]|.address_components|"\(.city) \(.state) \(.country)"' 2>/dev/null |  \
detox --inline --remove-trailing  | sed -e 's/\(.*\)/\L/'

# Sample output
california_md_us

So now that we have this, how to tie it together with exiftool file renaming process? Not complicated at all, actually.

First, we rewrite the previous curl command to include the exiftool syntax that will dynamically read the GPS coordinates from the file:

curl -s0 -q -k "${apibase}/reverse?q=$(exiftool -q -m -n -p '$GPSLatitude,$GPSLongitude' "${file_name}")&api_key=${api_key}&limit=1" | \
jq -r '.results[]|.address_components|"\(.city) \(.state) \(.country)"' 2>/dev/null | \
sed -e 's/\(.*\)/\L/' -e 's/[^A-Za-z0-9._-]/_/g'

And now, we just take the previous renaming example, and change single quotes to double quotes for the -d section, and right after the 03.c bit insert the $(longuglycommand), where the command would be that curl syntax above.

# Just to show you where the curl command goes
exiftool '-filename<${CreateDate}-${model;s/[- ]//g;tr/A-Z/a-z/}.%le' -d "%Y%m%d-%H%M%%-03.c-$(longuglycommand)"

# And here's the real thing
exiftool '-filename<${CreateDate}-${model;s/[- ]//g;tr/A-Z/a-z/}.%le' -d "%Y%m%d-%H%M%%-03.c-$(curl -s0 -q -k "${apibase}/reverse?q=$(exiftool -q -m -n -p '$GPSLatitude,$GPSLongitude' "${file_name}")&api_key=${api_key}&limit=1" | jq -r '.results[]|.address_components|"\(.city) \(.state) \(.country)"' 2>/dev/null | sed -e 's/\(.*\)/\L/' -e 's/[^A-Za-z0-9._-]/_/g')" "${file_name}"

# Sample filename result
20200123-1359-000-wilmington_de_us-iphone8plus.jpg

The final step is to produce a loop to rename the files of your choice. Even though exiftool has the recursive option, for better control, I would suggest using find. Here’s an example:

find . -mindepth 1 -maxdepth 1 -type f -name "*\.jpg" | while read file_name; do
echo "Saving original filename as a tag in ${file_name}"
exiftool -P -overwrite_original_in_place '-XMP-xmpMM:PreservedFileName<${filename;s/\.[^.]*$//}' "${file_name}"
echo "Renaming ${file_name}"
exiftool '-filename<${CreateDate}-${model;s/[- ]//g;tr/A-Z/a-z/}.%le' -d "%Y%m%d-%H%M%%-03.c-$(curl -s0 -q -k "${apibase}/reverse?q=$(exiftool -q -m -n -p '$GPSLatitude,$GPSLongitude' "${file_name}")&api_key=${api_key}&limit=1" | jq -r '.results[]|.address_components|"\(.city) \(.state) \(.country)"' 2>/dev/null | sed -e 's/\(.*\)/\L/' -e 's/[^A-Za-z0-9._-]/_/g')" "${file_name}"
done

The time-consuming part here is the `-overwrite_original_in_place` that saves the original filename as a tag inside the file. As I previously mentioned, if you don’t want it – you don’t have to keep it. If you do want it, however, it may be possible to speed up the process by taking advantage of your computer’s multiple CPU cores.

Below is an example using xargs. Remember that the free geotagging API you might use may have rate-limit access, so you should probably do this parallel processing only if you have a paid account. You can also add the convert_function to your .bashrc so you can use it to rename individual files manually.

# First we rewrite the conversion command as a function that 
# will accept arguments
convert_function() {
  echo "Saving original filename as a tag in "
  exiftool -P -overwrite_original_in_place '-XMP-xmpMM:PreservedFileName<${filename;s/\.[^.]*$//}' ""
  echo "Renaming "
  exiftool '-filename<${CreateDate}-${model;s/[- ]//g;tr/A-Z/a-z/}.%le' -d "%Y%m%d-%H%M%%-03.c-$(curl -s0 -q -k "${apibase}/reverse?q=$(exiftool -q -m -n -p '$GPSLatitude,$GPSLongitude' "")&api_key=${api_key}&limit=1" | jq -r '.results[]|.address_components|"\(.city) \(.state) \(.country)"' 2>/dev/null | sed -e 's/\(.*\)/\L/' -e 's/[^A-Za-z0-9._-]/_/g')" ""
}
export -f convert_function

# And now we can use that function with the `find` command
# and limiting the number of parallel processes to match
# the number of processor cores (or threads)
find . -mindepth 1 -maxdepth 1 -type f -name "*\.jpg" -print0 | xargs -r0 -n1 -P$(grep -c proc /proc/cpuinfo) -I {} bash -c 'convert_function "$@"' _ {}

If you find yourself with just too many files in a single folder, you can use exiftool to build a year/month/day folder structure and move the file into those folders based on when each photo was taken. Here’s an example:

# Take the files in the current folder and move them to
# year/Year-month-day subfolders
exiftool "-Directory<DateTimeOriginal" -d "%Y/%Y-%m-%d" .

# Sample folder structure
.
└── 2020
    ├── 2020-07-18
    ├── 2020-07-21
    └── 2020-07-31

# Personally, I find this folder structure more useful 
# as I don't take that many photos every day:
exiftool "-Directory<DateTimeOriginal" -d "%Y/%Y-%m" .

# Sample folder structure
.
└── 2020
    └── 2020-07

It may also help to create a monthly contact sheet inside the year folder. Here’s an example that uses the montage command (a part of the imagemagick package).

# Sample folder structure where you are at the '.' level, obviously
.
└── 2020
    └── 2020-07

# Creare a contact sheet in '2020' folder for each 'year-month' subfolder:

s="$(pwd)"
find . -mindepth 2 -maxdepth 2 -type d | while read f; do
  cd "${f}"
  montage -verbose -label '%f' -font Helvetica -pointsize 12 -background '#000000' \
  -fill 'gray' -define jpeg:size=300x300 -geometry 300x300+6+6 -tile 6x -auto-orient \
  $(find . -type f) ./"$(echo ${f} | awk -F'/' '{print $2}')/$(echo ${f} | awk -F'/' '{print $NF}')_contact.jpg"
  cd "${s}"
done

And here I am, hopefully on the way to organizing my mess of a photo archive.