Sign up ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free.

How can I find the line of corrupted characters (like �, ß, ø ...) in text files?

(A list of corrupted characters is here)

share|improve this question
Not easily, but this is obviously caused by mismatched character encodings. Possibly the strings have been input as utf8 strings, but instead of being treated as utf8 strings, they've been treated as latin1 or some other single byte encoding. Figure out what encodings has been involved, and the question should be answerable. – geirha Aug 22 '12 at 18:18
There's no list at the link you gave. Could you edit your link to be more specific? By the way, I agree with Geirha. This is almost certainly related to character encoding. If you can figure out the original encoding and how it's now being interpreted, you should be able to convert the files easily using a tool such as iconv. – Scott Severance Aug 22 '12 at 23:00
@ScottSeverance: On the page I linked is a link to "Corrupted-original characters table (Excel file, 29Kb)" – moose Aug 23 '12 at 7:21
if you have a text file, can you just use: iconv --from-code=UTF-8 (or --from-code=ASCII, etc)? The iconv utility will print "illegal input sequence at position {n}" on the line that has invalid data for the given input. You may also specific a target encoding; if the data can't be converted, you'll also receive a detailed error message. – michael_n Sep 19 '14 at 8:48

1 Answer 1

Though I haven't tested that particular specification inside the excel file I see the sql file mappings. You just asked to locate the bad characters, not fix them like the SQL function does. From stackoverflow to just see the line of non ascii characters:

grep --color='auto' -P -n "[\x80-\xFF]" file.xml

This will give you the line number, and will highlight non-ascii chars in red.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.