5

I am trying to fetch a line from a file file.txt which looks like this:

>This is line 1.</li>
>This is line 2.</li>
>This is line 3.</li>
>This is line 4.</li>

I need to fetch content which starts with > and ends in </li> so the output will be This is line 1. This is line 2. and so on. I have looked into this in forum but didnt found solution. This solution also didnt worked.

I ultimately have to fetch some lines from a webpage. So first I will curl webpage and then use grep command to grep that line which starts with > and ends in </li>.

Thanks.!

1
  • I suggest to use an XML/HTML parser (xmlstarlet, xmllint ...).
    – Cyrus
    Commented Aug 12, 2017 at 16:18

2 Answers 2

12

This should be enough:

grep '^>.*</li>$' input-file

The ^ and $ ensure that those parts are anchored at the start and end of the lines respectively.

You can also do:

grep -x '>.*</li>' input-file

-x looks for an exact match: the whole line should match the pattern (which implies ^ and $ is wrapped around the pattern).

5
  • Hi Thanks. It works but is not working when extracing content from a webpage. For example: view-source:http://goodmorningmylove.com/best-romantic-love-quotes-for-her-from-the-heart/ from this page I want to extract this in between text >In a sea of people, my eyes will always be searching for you.</li> . I first did curl webpage > file.txt and stored it in file.txt, now when doing grep it didnt show any result
    – S Andrew
    Commented Aug 12, 2017 at 5:17
  • @SAndrew the "View Source" of a browser shows a cleaned up view of the source (it intends and splits tags over multiple lines). Have you looked at the actual file.txt to check if it matches what you see in the browser? Also, the pages may contain dynamically generated content using JavaScript etc., and you won't see that when curling the page.
    – Olorin
    Commented Aug 12, 2017 at 5:21
  • I checked file.txt and most of the content matches but not sure. Then what could be the best way to extract data(copy a line) from any webpage.
    – S Andrew
    Commented Aug 12, 2017 at 5:23
  • 1
    @SAndrew use a tool that understands HTML, like Pup: github.com/EricChiang/pup
    – Olorin
    Commented Aug 12, 2017 at 5:25
  • i think grep solution mentioned by @Olorin pretty clear, what is wrong with that?
    – solfish
    Commented Aug 12, 2017 at 6:36
0

This is the input file:

$ cat /tmp/tmp.txt
>This is line 1.</li>
invalid line 1
>This is line 2.</li>
>This is line 3.</li>
invalid line 2

>This is line 4.</li>
last invalid line

Using grep and awk to extract the strings you want:

$ cat /tmp/tmp.txt | grep -E '>*</li>' | awk -F\> '{ print $2 }' | awk -F\< '{ print $1 }'
This is line 1.
This is line 2.
This is line 3.
This is line 4.
3
  • 1
    There's no need to use cat and grep if you use awk: awk -F'[<>]' '/^>.*<\/li>$/{print $2}' tmp.txt; and no need to use awk if you use grep: grep -Po '^>\K.*(?=</li>$)' foo
    – muru
    Commented Aug 14, 2017 at 1:20
  • "Simple is better than complex.", @muru (see python.org/dev/peps/pep-0020)
    – boardrider
    Commented Aug 14, 2017 at 22:06
  • 1
    I'd argue that single awk command is simpler than this mess of a pipeline. The Perl lookarounds are a bit much, though, I'll give you that.
    – muru
    Commented Aug 14, 2017 at 23:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .