I'd like to know how to do web scraping using Ubuntu terminal. I've read something about using python with beatifulsoup, urllib, but I was wondering if there is there any way to do it without running a python script?

Let's say I'm searching internet site www.marmara.edu.tr (my university site) and I'd like to get some data, how can i do that by command line?


Combine the curl and text processing utilites like grep, sed awk, etc.

For example:

curl https://askubuntu.com/questions/941409 &| grep -Po '(?<=<title>).*(?=</title>)'

returns the title of your question's page.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.