Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

How can I encode and decode percent-encoded (URL encoded) strings on the command line?

I'm looking for a solution that can do this:

$ percent-encode "ændrük"
%C3%A6ndr%C3%BCk
$ percent-decode "%C3%A6ndr%C3%BCk"
ændrük
share|improve this question
    
Do you want to incorporate different encodings too? %E6ndr%FCk doesn't look like (standard) UTF8 to me. Or it's just an example? –  arrange Jul 19 '11 at 21:13
    
@arrange Thanks for catching that. Apparently I chose the bad apple among search results for online converters. –  ændrük Jul 19 '11 at 21:49
    
For file names, see: How to remove URI encoding in file names. –  kenorb Feb 12 at 17:54

4 Answers 4

up vote 15 down vote accepted

These commands do what you want:

python -c "import urllib, sys; print urllib.quote(sys.argv[1])" æ
python -c "import urllib, sys; print urllib.unquote(sys.argv[1])" %C3%A6

If you want to encode spaces as +, replace urllib.quote with urllib.quote_plus.

I'm guessing you will want to alias them ;-)

share|improve this answer

I found a package, renameutils, that contain the utility deurlname that is able to rename a file containing "percent-encoded" characters.

Unfortunately, it does not decode stdin or a command line option, but only rename a file, so you have to create a dummy file to obtain the decoding (the name of the renamed file), but with some bash scripting the process can be automated.

No information about the encoding part, even because it could be questionable which characters to encode. Only non-ASCII?

I think there should be some better tool/method.

share|improve this answer

shell

Try the following command line:

$ echo "%C3%A6ndr%C3%BCk" | sed 's@+@ @g;s@%@\\x@g' | xargs -0 printf "%b"
ændrük

You may define it as alias and add it to your shell rc files:

$ alias urldecode='sed "s@+@ @g;s@%@\\\\x@g" | xargs -0 printf "%b"'

Then every time when you need it, simply go with:

$ echo "http%3A%2F%2Fwww" | urldecode
http://www

bash

When scripting, you can use the following syntax:

input="http%3A%2F%2Fwww"
decoded=$(printf '%b' "${input//%/\\x}")

However above syntax won't handle pluses (+) correctly, so you've to replace them with spaces via sed.

You can also use the following urlencode() and urldecode() functions:

urlencode() {
    # urlencode <string>
    local length="${#1}"
    for (( i = 0; i < length; i++ )); do
        local c="${1:i:1}"
        case $c in
            [a-zA-Z0-9.~_-]) printf "$c" ;;
            *) printf '%%%02X' "'$c"
        esac
    done
}

urldecode() {
    # urldecode <string>

    local url_encoded="${1//+/ }"
    printf '%b' "${url_encoded//%/\\x}"
}

Note that your urldecode() assumes the data contains no backslash.


bash + xxd

Bash function with xxd tool:

urlencode() {
  local length="${#1}"
  for (( i = 0; i < length; i++ )); do
    local c="${1:i:1}"
    case $c in
      [a-zA-Z0-9.~_-]) printf "$c" ;;
    *) printf "$c" | xxd -p -c1 | while read x;do printf "%%%s" "$x";done
  esac
done
}

Found in cdown's gist file, also at stackoverflow.


Python

Try to define the following aliases:

alias urldecode='python -c "import sys, urllib as ul; print ul.unquote_plus(sys.argv[1])"'
alias urlencode='python -c "import sys, urllib as ul; print ul.quote_plus(sys.argv[1])"'

Usage:

$ urlencode "ændrük"
C%26ndrC%3Ck
$ urldecode "%C3%A6ndr%C3%BCk"
ændrük

Source: ruslanspivak


PHP

Using PHP you can try the following command:

$ echo oil+and+gas | php -r 'echo urldecode(fgets(STDIN));' // Or: php://stdin
oil and gas

or just:

php -r 'echo urldecode("oil+and+gas");'

Use -R for multiple line input.


Perl

In Perl you can use URI::Escape.

decoded_url=$(perl -MURI::Escape -e 'print uri_unescape($ARGV[0])' "$encoded_url")

Or to process a file:

perl -i -MURI::Escape -e 'print uri_unescape($ARGV[0])' file

sed

Using sed can be achieved by:

cat file | sed -e's/%\([0-9A-F][0-9A-F]\)/\\\\\x\1/g' | xargs echo -e

awk

Try anon solution:

awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..

See: Using awk printf to urldecode text.


decoding file names

If you need to remove url encoding from the file names, use deurlname tool from renameutils (e.g. deurlname *.*).

See also:


Related:

share|improve this answer

Pure bash solution for decoding only:

$ a='%C3%A6ndr%C3%BCk'
$ echo -e "${a//%/\\x}"
ændrük
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.