2

I have a file with special characters that were created during copying from terminal.

Now I would like to remove them. I tried grep and sed but didn't work maybe I'm doing it wrong.

Special character is an ESC inside a black box, can't copy it but if I do, it turns into a box with numbers inside:

numbers inside that box are:

00

1B

  • 2
    Is this output from the script command? – Jacob Vlijm Dec 22 '16 at 15:45
2

Python can do the job. The process here is simple, we read in all the lines into list while simultaneously replacing the UTF escape character ( which is \u001b ), and then print out lines again, but without the escape character. The < input.txt sends old text to python command , and > new_file.txt sends text to new file.

Script:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import sys
lines=[l.strip().replace(u"\u001b","") for l in sys.stdin]
print("\n".join(lines))

Save it as delete_escape.py, make it executable with chmod +x ./delete_escape.py, and call it like so:

./delete_escape.py < input.txt > output.txt

Results:

enter image description here

  • I execute this code in a python file or can i do it in terminal? – Insanebench420 Dec 22 '16 at 14:23
  • @Insanebench420 you can do either, just remember to properly redirect input and output files. Also, please wait a minute - I'm gonna add python script file to the answer – Sergiy Kolodyazhnyy Dec 22 '16 at 14:24
  • It gives me an error when i try to execute it. Line 3 its pointing at 1.strip().replace ^ – Insanebench420 Dec 22 '16 at 14:39
  • @Insanebench420 did you copy the script exactly as in the answer ? Copy it to paste.ubuntu.com and give me the link plz – Sergiy Kolodyazhnyy Dec 22 '16 at 14:42
  • Oh and please include the error too – Sergiy Kolodyazhnyy Dec 22 '16 at 14:42
2

You can do it in sed, but you need to use the ANSI escape in bash to give it the character:

sed -i 's/'$'\u001b''//g' file

And here it is in action:

screenshot showing the character being removed

Alternatively, in perl:

perl -i -pe 's/'$'\u001b''//g' file

And with tr:

tr -d $'\u001b' < file > newfile 
  • It doesn't delete the ANSI escape inside the .txt file but the output with cat is good. I want it removed from .txt file. – Insanebench420 Dec 22 '16 at 14:29
  • @Insanebench420 it should. And it does in my tests. If it doesn't, that might be the wrong character. Try creating a file with printf ':%s:\n' $'\u001b' > file. That will create a file with the bad character between two :. Then, try the command I gave and see if it removes it. – terdon Dec 22 '16 at 14:42
  • I tried it out and yes if i make one :u0001b: it will delete it but... if i make more then that in one line it wont delete it.. – Insanebench420 Dec 22 '16 at 14:50
  • Instead it will delete only the first one and leave the rest. – Insanebench420 Dec 22 '16 at 14:50
  • @Insanebench420 ah, yes, if you can have more than one you need g. See the updated answer. – terdon Dec 22 '16 at 14:54
0

If I understand correctly it looks like you want to remove the ansi escape sequences present in your text. Try

ansifilter file

Update

To my surprise there is no ansifilter package for ubuntu! (there is for fedora, arch, brew, etc)

Nevertheless we can install from TGZ or from other packages, or wget the rpm, apply alien to build a deb package and install it.

  • 1
    No manpage, not provided by any package. – muru Nov 7 '17 at 8:35
  • @muru, you are absolutely right. (I should perhaps remove the answer). There is the TGZ, packages for fedora, centos, arch, brew, ... but not for ubuntu. – user216043 Nov 7 '17 at 9:51
  • The problem was already solved..but thanks. – Insanebench420 Nov 9 '17 at 16:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.