Is there a way to change the character encoding of a text file? If so, how?

See, I am using a keyboard shortcut to call a shell command which itself uses xclip to copy the contents of a file into the clipboard. I can then use CTRL + V to paste the contents. I am doing this to add the signature to email whenever I need.

This is the script:

# Need to install xclip
cat ~/personal/mail-signature.txt | xclip -i -selection clipboard

It all seems to work correctly, even on the e-mail composition window. However, when the message is sent the carriage return (new line characters) seem to have been doubled as compared to the file. That is, regardless what I use in gedit to add the new line (ENTER, SHIFT+ENTER, or CTRL+ENTER) the all turn this:

My name and address.                      Affiliation
fax:   +1 (5555) 555-5555      Address in this planet
my_email@someaddress.ca    http://www.someaddress.ca/

into this


My name and address.                      Affiliation

fax:   +1 (5555) 555-5555      Address in this planet

my_email@someaddress.ca    http://www.someaddress.ca/


once the message is sent.

This does not seem to happen when I open the signature file in gedit and use CTRL+C and CTRL+V on my e-mail client's window.

I am guessing it must be the way xclip read/interprets the end of line / new line character but for some reason the client's editing window does not show the difference while the server (Gmail's smtp server in this case) does seem see the difference between the end of line characters.

I am using Ubuntu 16.04 and Thunderbird 45.3.0 as the e-mail client.

Update: Just tried xclip with option -noutf8 but did not work either.

  • run file ~/personal/mail-signature.txt and see what the output is; if it's ASCII text, with CRLF line terminators then convert it using dos2unix ~/personal/mail-signature.txt. – michael Sep 27 '16 at 15:53
  • Perfect! That worked. – Juan Antonio Sep 27 '16 at 16:01
  • One question remains, how is it that I have NOT used Windows/dos in years yet the character encoding seems to be stuck in DOS while I am using gedit? – Juan Antonio Sep 27 '16 at 16:02

Although not an answer, more like a workaround, michael_n has provided a way around: put the signature file through dos2unix filter. This fixed the issue but has me asking a bigger question: why is the default character encoding in gedit DOS encoding? For compatibility reasons maybe.

Either way, thanks michael_n.

| improve this answer | |
  • 1
    Technically, it's not "encoding" that you're converting (i.e., it's probably ascii, in either case). Loosely, it's just a "format", specifically, there are extra end-of-line control characters that somehow got in there somewhere. gedit will preserve the line-endings of the original source. If the document was originally created in gedit, you can choose both the encoding (e.g., utf-8, ascii, etc) and the line-ending (windows, unix) when saving the document. – michael Sep 28 '16 at 2:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.