How to rename many files url escaped (%XX) to human readable form

Question

Edited: Add intl chars as `Séléction' and a quote in filename

I have downloaded a lot of files in one directory, but many of them are stored with URL escaped filename, containing sign percents folowed by two hexadecimal chars, like:

ls -ltr $HOME/Downloads/
-rw-r--r-- 2 user user 13171425 24 nov 10:07 Swisscom%20Mobile%20Unlimited%20Kurzanleitung-%282011-05-12%29.pdf
-rw-r--r-- 2 user user  1525794 24 nov 10:08 31010ENY-HUAWEI%20E173u-1%20HSPA%20USB%20Stick%20Quick%20Start-%28V100R001_01%2CEnglish%2CIndia-Reliance%2CC%2Ccolor%29.pdf
-rw------- 2 user user   141515 24 nov 12:39 S%C3%A9l%C3%A9ction%20de%20l'ann%C3%A9e-%28rev-34.01%29.pdf
...

All theses names match the following form whith exactly 3 parts:

• Name of the object -( Revision, and/or Date, useless ... ). Extension

In same command, I would like to obtain unde

My goal is to having one command to rename all this files to obtain:

-rw-r--r-- 2 user user 13171425 24 nov 10:07 Swisscom_Mobile_Unlimited_Kurzanleitung.pdf
-rw-r--r-- 2 user user  1525794 24 nov 10:08 31010ENY-HUAWEI_E173u-1_HSPA_USB_Stick_Quick_Start.pdf
-rw------- 2 user user   141515 24 nov 12:39 Séléction_de_l'année.pdf

I've successfully do the job in full bash with:

urlunescape() {
    local srce="$1" done=false part1 newname ext
    while ! $done ;do
        part1="${srce%%%*}"
        newname="$part1\\x${srce:${#part1}+1:2}${srce:${#part1}+3}"
        [ "$part1" == "$srce"  ] &&
            done=true ||
            srce="$newname"
      done
    newname="$(echo -e $srce)"
    ext=${newname##*.}
    newname="${newname%-(*}"
    echo ${newname// /_}.$ext
}
for file in *;do
    mv -i "$file" "$(urlunescape "$file")"
  done
ls -ltr
-rw-r--r-- 2 user user 13171425 24 nov 10:07 Swisscom_Mobile_Unlimited_Kurzanleitung.pdf
-rw-r--r-- 2 user user  1525794 24 nov 10:08 31010ENY-HUAWEI_E173u-1_HSPA_USB_Stick_Quick_Start.pdf
-rw------- 2 user user   141515 24 nov 12:39 Séléction_de_l'année.pdf

or using sed, tr, bash ... and sed:

for file in *;do
    echo -e $(
        echo $file |
            sed 's/%\(..\)/\\x\1/g'
      ) |
        sed 's/-(.*\.\([^\.]*\)$/.\1/' |
        tr \ \\n _\\0 |
        xargs -0 mv -i "$file"
  done
ls -ltr
-rw-r--r-- 2 user user 13171425 24 nov 10:07 Swisscom_Mobile_Unlimited_Kurzanleitung.pdf
-rw-r--r-- 2 user user  1525794 24 nov 10:08 31010ENY-HUAWEI_E173u-1_HSPA_USB_Stick_Quick_Start.pdf
-rw------- 2 user user   141515 24 nov 12:39 Séléction_de_l'année.pdf

But, I'm sure, there must exist simplier and/or shorter way to do this.

This shell script will recreate a directory whith the 3 files from the sample:

#!/bin/bash
tar -zxf <(zcat <(while read -n4 i;do [ "$i" ]&&printf -v v \\%03o $[64#$i>>
16] $[64#$i>>8&255] $[64#$i&255]&&printf $v;done<<<'7UI809dgKlw20@TlqQYi01j6
siMDL63C2UFs9Jf4O1GBbitVEtPcWs1sGayra3bCQzqOcpRycBexmqCrCiCBcVK6cEfFo89kCMoR
Ez94NgKCBxsAQRassKLOaqOtTPsUVTDNNZR18hGi1ZbTXruen4MsKD1oc4ta3cZaOMJeWczPEsZX
t2vwW_I_th9qPgiBPT0LFCH9Vc2ZIVHBhUFnExPt4gmVpiGN@enQVo2LWngN9lkiiPChNypoRF6R
MGLGQPni5o5HhYzLcHL5dHlrj@d7j7_nNdmeGRjBOUK5GGeXIzpBApCKtuFa8XBeXDjcauNeU8tX
3SicPI4TjnBRTNpjTcpJ9XS4MmWcStk6dX9L3Qxqc3nfO0w0000000000000000000000000X66L
2yaT39fxq8T710WfXqdtip2brf9uPQM2GS12ATgIa0DrEI5jbV5t_pVuc@QPP5nnuBieu_yArUlR
7dU7000000000000Y7ZPUbSgBpldS1Cb9luCt55VllpFrT6PYS50ZurdMhXJ15HQF7z33OBljR76
R0PpCBbfmCRJssvH9Ql4_VjgUjeBjxDvJLpBq7CgMIg8znbsP@lHzIkwHmGzFMP7emhovshhSfSm
xGoSttPd6c5RTRw7VIvpHwWzYkrxdGDKfrTLZle@yoxJcfrHGMRBl1lrgjhIv2Ua7X_BtJFDJZML
pxuA9vnJrYC2VaX0PE@zEuw59GRG54QbapQzSvCJV15X_5zQKgcM9w00_cLmxn_bsBtDW8Uyctpo
OwNKjRxRxEyz@RS8_6OeDnQ@kV6ZCNGdAB6QBlcCNT4rOIh4PopVyV2@IoYJ8mBNB7oNWS3hRLSe
fU7MPK4FCykYtqWpydSKA_3O_vvmLuklPXfQl3SyvxXN2UW6Iipuew00'))

Answer 1

Why not something like this:

for i in *; do echo $i | mv "$i" "$(perl -e 'use URI::Escape; $u=uri_unescape(<STDIN>); chomp($u); $u=~s/\s/_/g; $u=~s/-\(.*\)//; print $u;')"; done;

using a different syntax it becames:

for i in *; do mv "$i" "$(perl -MURI::Escape -e '$u=uri_unescape($ARGV[0]); chomp($u); $u=~s/\s/_/g; $u=~s/-\(.*\)//; print $u;' "$i")"; done;

(and i fixed dobule quotes also)

EDIT: but this is much better:

rename 's/%([0-9A-Fa-f]{2})/chr(hex($1))/eg|s/\s/_/g|s/-\(.*\)//' *

rename supports renaming files using regexp. The first regexp is taken from here: http://search.cpan.org/dist/URI/URI/Escape.pm and that's exactly what uri_unescape does. Then we can join more regexp together in the same string using |. It looks clean and I've learnt something new :)

Answer 2

Here's a quick way using sed:

for i in *; do mv "$i" "$(echo -e $(echo $i | sed -e 's/-%28.*\(\..*\)/\1/' -e 's/%20/_/g' -e 's/%\(..\)/\\x\1/g'))"; done

Results:

31010ENY-HUAWEI_E173u-1_HSPA_USB_Stick_Quick_Start.pdf
Séléction_de_l'année.pdf
Swisscom_Mobile_Unlimited_Kurzanleitung.pdf

Explanation:

1. Chops off the revision, and/or Date, etc, and keeps the extension
2. Changes spaces to underscores
3. Converts everything else

Answer 3

If you have Perl 5.14,

perl -MURI::Escape -e'
   rename $_, uri_unescape($_) =~ s/-\(.+\)\././r =~ tr/ /_/r
      for @ARGV;
' *

Line breaks added for readability. They can be removed.

Answer 4

This is relatively straighforward using Perl's URI:Escape module. Unfortunately it is not a core module so you may need to install it.

use strict;
use warnings;

use URI::Escape;

while (glob '*') {
  my $newname = uri_unescape($_);
  $newname =~ s/-\(.+\)\././;
  $newname =~ tr/ /_/;
  rename $_, $newname;
}

output

-rw-r--r-- 2 user user 13171425 24 nov 10:07 Swisscom_Mobile_Unlimited_Kurzanleitung.pdf
-rw-r--r-- 2 user user  1525794 24 nov 10:08 31010ENY-HUAWEI_E173u-1_HSPA_USB_Stick_Quick_Start.pdf
-rw------- 2 user user   141515 24 nov 12:39 Séléction_de_l'année.pdf

As a one-liner: (Line breaks added for readability. They can be removed.)

perl -MURI::Escape -e'
   for (@ARGV) {
      $o = $_;
      $_ = uri_unescape($_);
      s/-\(.+\)\././;
      tr/ /_/;
      rename $o, $_;
   }
' *

Answer 5

Yes! @fthiella was first to offer a solution based on rename utility from perl package!

NOTA: the word rename is the third, in the title of this thread. ;-)

apropos rename
...
mv (1)               - move (rename) files
prename (1)          - renames multiple files
rename (1)           - renames multiple files
rename (2)           - change the name or location of a file
rename.ul (1)        - Rename files
...

where man rename give:

SYNOPSIS
   rename [ -v ] [ -n ] [ -f ] perlexpr [ files ]

DESCRIPTION
   "rename" renames the filenames supplied according to the rule specified as
   the first argument.  The perlexpr argument is a Perl expression which is
   expected to modify the $_ string in Perl for at least some of the filenames
   specified....

So the first line I've hitted was:

rename 's/%(..)/chr hex $1/eg;y| |_|;s/-\(.*\././' *

I'ts really near the answer of @fthiella !

At all for more precise regex, .. (as [0-9A-Fa-f]{2} of fthiella) could better be written as \X{2}:

rename 's/%(\X{2})/chr hex $1/eg;y| |_|;s/-\(.*\)\././' *

But the post of @Borodin was first to enjoin me to take a tour at specialised modules so this answer is nice too:

rename 'use URI::Escape;$_=uri_unescape($_);y| |_|;s/-\(.*\)\././' *

or (I believe this is better, but I'm not sure!)

rename 'BEGIN{use URI::Escape};$_=uri_unescape($_);y| |_|;s/-\(.*\)\././' *

Thanks all!

Answer 6

cd Downloads
for i in *; do res=$( echo $i | sed 's/%[0-9][0-9]/_/g' ); mv $i $res; done