Revisiting this again, and trying to use nothing but a Bash shell, another one line solution is:
while read url; do url="${url##*/}" && echo "${url%%\'*}"; done < file.in > file.out
Where file.in contains the 'dirty' url list and file.out will contain the 'clean' URL list. There are no external dependencies and there is no need to spawn any new processes or subshells. The original explanation and a more flexible script follows. There is a good summary of the method here, see example 10-10. This is pattern based parameter substitution in Bash.
Expanding on the idea:
src="define('URL', 'http://url.com');"
src="${src##*/}" # remove the longest string before and including /
echo "${src%%\'*}" # remove the longest string after and including '
Result:
url.com
No need to call any external programs. Furthermore, the following bash script, get_urls.sh
, permits you to read a file directly or from stdin:
#!/usr/bin/env bash
# usage:
# ./get_urls.sh 'file.in'
# grep 'URL' 'file.in' | ./get_urls.sh
# assumptions:
# there is not more than one url per line of text.
# the url of interest is a simple one.
# begin get_urls.sh
# get_url 'string'
function get_url(){
local src="$1"
src="${src##*/}" # remove the longest string before and including /
echo "${src%%\'*}" # remove the longest string after and including '
}
# read each line.
while read line
do
echo "$(get_url "$line")"
done < "${1:-/proc/${$}/fd/0}"
# end get_urls.sh