Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I'm trying to write a Bash script that will extract informations from a HTML page (using wget). I know my informations will be between <h*> tags, but is there a nice way to get those ?

To be more precise let's have an example :

< h1>header1< /h1>
< h2>header2< /h2>
< h2>otherHeader2< /h2>
< h1>lastHeader1< /h1>
< h2>lastHeader2< /h2>

I'd like to extract "otherHeader2", a.k.a. the second (but it could be anywhere) header afer the header1.

share|improve this question
    
It could be done, but it would be somewhat painful, I think. If it were me, I'd use another scripting language, such as perl, python, or PHP, which can be run like a Bash script by using the proper shebang (hashbang); ie #! /usr/bin/python for a python script. You may even find existing code for one of those, and it might be useful to learn the languages. –  Marty Fried Oct 5 '12 at 23:30
    
What about using sed or awk? –  drN Oct 20 '12 at 23:35
add comment

2 Answers 2

up vote 1 down vote accepted

this is a simple python script that will parse your html and put all the values into a list and print it out. You can either write the rest of your script in python, or call this script from python, or plug this short code as a snippet in bash. Check out the examples below.

test.html

< h1>header1< /h1>
< h2>header2< /h2>
< h2>otherHeader2< /h2>
< h1>lastHeader1< /h1>
< h2>lastHeader2< /h2>

parse_header.py

#!/usr/bin/env python
import sys, re
print re.findall(r'< h\d>(.*)< /h\d>', sys.stdin.read())

script can be called from bash

cat test.html | parse_header.py

python code can be put into bash script

cat test.html | python -c "import sys, re;print re.findall(r'< h\d>(.*)< /h\d>', sys.stdin.read())"

the last option I believe is not very readable in your case. It makes more sense if you have some simple code where it's not worth putting it in it's own script.

share|improve this answer
    
Thanks, I'll give Python a try. –  NorTicUs Oct 6 '12 at 9:51
    
Please see stackoverflow.com/a/1732454/1524545 –  geirha yesterday
add comment

You should never parse html with regular expressions. Always use a proper html or xml parser. For example, using xmlstarlet:

$ xmlstarlet sel -t -v '//h2[2]' html.html 
otherHeader2
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.