Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have the problem described in this Q&A. Probably from quite old linux distros or from windows I have several files with broken filenames. ls displays a "?" instead of the broken character. I successfully renamed some of these files, but I don't know if I've found all of them.

Is there any method to find all affected files?

share|improve this question
add comment

3 Answers 3

up vote 3 down vote accepted

Assuming you are using utf-8 encoding (the default in Ubuntu), this script should hopefully identify the filenames and rename them for you.

It works by using find with C-encoding (ascii) to locate files with unprintable characters in them. It then tries to determine if these unprintable characters are utf-8 characters or not. If not, it shows you the filenames decoded with each of the encodings listed in the enc array, allowing you to select the one that looks right in order to rename it.

latin1 was commonly used on older linux systems, and windows-1252 is commonly used by windows now adays (I think). iconv -l will show you a list of possible encodings.

#!/bin/bash

# list of encodings to try. (max 10)
enc=( latin1 windows-1252 )

while IFS= read -rd '' file <&3; do
    base=${file##*/} dir=${file%/*}

    # if converting from utf8 to utf8 succeeds, we'll assume the filename is ok.
    iconv -f utf8 <<< "$base" >/dev/null 2>&1 && continue

    # display the filename converted from each enc to utf8
    printf 'In %s:\n' "$dir/"
    for i in "${!enc[@]}"; do
        name=$(iconv -f "${enc[i]}" <<< "$base")
        printf '%2d - %-12s: %s\n' "$i" "${enc[i]}" "$name"
    done
    printf ' s - Skip\n'

    while true; do
        read -p "? " -n1 ans
        printf '\n'
        if [[ $ans = [0-9] && ${enc[ans]} ]]; then
            name=$(iconv -f "${enc[ans]}" <<< "$base")
            mv -iv "$file" "$dir/$name"
            break
        elif [[ $ans = [Ss] ]]; then
            break
        fi
    done
done 3< <(LC_ALL=C find . -depth -name "*[![:print:][:space:]]*" -print0)
share|improve this answer
    
Wow, impressive script - thanks! –  lumbric Mar 16 '12 at 17:42
add comment

Begin with this regex find command and modify it until you hit only those you are interested in: find . | egrep [^a-zA-Z0-9_./-\s].
The one above will find filenames that has a non UTF-8 character(s).

share|improve this answer
    
The above did not work for me... Try this instead if it doesn't work for you either: find / | grep -P "[\x80-\xFF]". This will locate all non-ASCII characters in file and folder names. –  SirCharlo Mar 15 '12 at 14:31
    
@SirCharlo Thanks, this seems to be at least partly successful! But it finds also a lot of filenames which doesn't seem to be affected at first glance. –  lumbric Mar 15 '12 at 16:50
    
@yossile looks like a promising approach, but I stopped trying after having a very long list of excluded symbols. –  lumbric Mar 15 '12 at 16:50
    
@lumbric No problem! Could you give me an example of a seemingly irrelevant file name? –  SirCharlo Mar 15 '12 at 17:28
    
@SirCharlo Sorry, I was wrong. It finds also those filenames with ASCI 7-bit characters - also those where the encoding is correct. But this is exactly what you claimed. I just filtered this output with another |egrep -v [äöüÄÖÜß] (all non-ASCII 7bit characters are german umlauts). Don't you want to convert your comment to an answer, since it was quite helpful too? –  lumbric Mar 16 '12 at 17:17
show 3 more comments

Try this:

find / | grep -P "[\x80-\xFF]"

This will locate all non-ASCII characters in file and folder names, and help you to find the guilty culprits :P

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.