-
Convert file extensions to lower case (rename, Linux shell script)
Posted on November 20th, 2012 5 commentsI am using Unison more and more, and recently I decided to try to use it to keep my digital image archive synchronized. It seems that at some time I have not explicitly instructed the image importing program of the day (Shotwell being the program I mostly use) to rename the image files in order to consequently use a lower case file extension.
Thus I felt the need to rename all image files (in the shotwell folder) to a lower case file extension. There may or may not be a program out there to do this, but I wrote my own shell script for this purpose. My experience is that I learn more by doing it myself, and a lot of the tricks I learn can be reused the next time.
This is how I did it, I cannot promise that it will work for anyone else (indentation lost in WordPress):
#!/bin/sh
imgdir=‘pwd‘
imgucext=$(find ${imgdir} –type f –name ‘.’ | awk –F’.’ ‘NF > 0 { print $NF }’ | sort | uniq | LC_ALL=C egrep ”[[:upper:]]”)
for ucext in ${imgucext}
do
lcext=$(echo ${ucext}|awk ’{print tolower($0)}’)
find ${imgdir} –type f –name “.${ucext}” –print | \
sed ’{s/ /\\ /g}’ – | \
sed ’{s/(/\\(/g}’ – | \
sed ’{s/)/\\)/g}’ – | \
sed ”{s/\(.\.\)${ucext}$/mv –n –v \1${ucext} \1${lcext}/g}” – | \
sh
done
Starting in the current folder/directory, I check what upper case extensions are present, find all files using them, rewrite spaces and parenthesis for the shell, and create a rename command for each file, which is then piped to the shell. For debugging purposes the sh close to the end would be substituted for a cat.5 responses to “Convert file extensions to lower case (rename, Linux shell script)”
-
Johan Adler
November 21st, 2012 at 18:30
It turned out that Unison is not what I need in this case.
What I want to accomplish is to transfer photos from cameras, downloaded to my desktop box or one of the laptop boxes, to a unified structure on my NAS (managed by an Alix 2D13 (http://pcengines.ch/alix2d13.htm) from PC Engines.)
What I need is mostly a one way transfer of any files not already present in the NAS image storage structure. Unison is great for two way synchronization, but in this case it seems that direct invocation of rsync is superior, and I am creating another shell script using rsync for doing what I want.
-
Crap! Files from some sources or eras have uppercase file name and lowercase extension, but not consistently, different boxes and software seems to have handled this in different ways.
Some of the images that exist in more than one copy, each with different combinations of uppcase and lowercase letters in its name, have been rotated, and tags may have been added, and Shotwell or whatever has been instructed (by me, mea culpa) to write these changes/additions to the file metadata (EXIF or similar). This makes a simple search for duplicates based on filesize and/or checksum less than perfect.
I have decided that from now on any digital photos (and other files from digital cameras, such as movies) shall use fully lowercase filename and extension. But I cannot just have a script rename all files with any uppercase part of filename/extension since there in quite a few cases exists a file with just that (fully lowercase) name.
So one current project is a script that will identify files with any uppercase character in either filename or extension (but not the path, since ${HOME}/Pictures/shotwell/ will always contain an uppercase character), then for each uppercase name containing file (“ucf”) look for files in the same directory that do not have the identical name, create a backup directory structure that the extra versions will then be moved to.
I know—I should check filesize, modification date and probably MD5 sum for each of those files, delete the identical ones, keep the newest or largest in the main directory structure and move the rest to the backup directory structure. I am not quite there yet…
The last (close to) eight hours I have been using rsync, without the update option, to transfer all (relevant, in this directory structure) files from the NAS to my desktop. With a 38 GB image repository this is bound to take some time. (I am currently at March last year.)
Next step could either be running fslint to find obviuos duplicates and delete them, or running my script mentioned above that will convert any and all files to lowercase while making a backup of any files that would be in the way for the former. In know, fslint can scan more than one place, so I could move the extra files first, if I want.
Also, the script above is not perfect, I rewrote it and hope this meant some improvement.
-
Rsync stats, no joke:
Number of files: 28064
Number of files transferred: 19497
Total file size: 37.66G bytes
Total transferred file size: 28.39G bytes
Literal data: 22.68G bytes
Matched data: 5.71G bytes
File list size: 562.58K
File list generation time: 0.004 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 44.57M
Total bytes received: 22.35Gsent 44.57M bytes received 22.35G bytes 753.52K bytes/sec
total size is 37.66G speedup is 1.68 -
I have been exploring for a bit for any high-quality articles or
weblog posts on this sort of area . Exploring in Yahoo I at last stumbled upon
this web site. Studying this info So i am glad to exhibit that I have an incredibly good uncanny feeling
I found out exactly what I needed. I most definitely will
make sure to don?t put out of your mind this
web site and give it a glance regularly. -
Unquestionably believe that which you said. Your favourite justification appeared to be at
the web the simplest thing to remember of.
I say to you, I definitely get annoyed even as
folks consider worries that they plainly do not recognise about.
You managed to hit the nail upon the top as smartly as outlined out the entire
thing without having side-effects , folks could take a signal.
Will likely be again to get more. Thanks
Leave a reply
-
creeper
- Riksdagen 2013-05-17
- Försvarsmakten 2013-04-29
- Riksdagen 2013-04-26
- Åklagarmyndigheten 2013-04-24
- Försäkringskassan 2013-04-09
- Försäkringskassan 2013-02-06
- Försäkringskassan 2013-02-06
- Försvarsmakten 2013-01-30
- Kriminalvården 2013-01-23
- Rikspolisstyrelsen 2013-01-16