3

I am trying to extract URLS from a webpage up to user defined n levels using wget. I tried this

 wget -r -l$2 --reject=gif -O out.html www.google.com | sed -n 's/.*href="\([^"]*\).*/\1/p'` "

It is displaying only the first level. it is not parsing any levels how could i rectify it

1

Get rid of the $2 unless you're assigning a variable 2=1 or something like that elsewhere, -l$2 will cause wget to output wget: --level: Invalid number which will not feed into sed very well.

you can do:

wget -l 2 <address>

or

wget --length=2 <address>

you may also have interest in the no-parent option, which will prevent wget from traversing up the directory (n levels). This is an especialy useful option when retrieving recursively, since it guarantees that only the files below a certain hierarchy will be downloaded.

wget -np <address>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.