0

I have a lot of html files in a folder. I used them in a project. I want to build a References.txt file in which the URL from which each html file was saved will be listed with the time stamp on which it was saved. There will be a space between each URL and its corresponding timestamp.

How can I do that from Ubuntu 12.04 using a terminal command or shell script?

One information which can help you is that the URL from which a html file was saved is present in the second line of the html file and that is also visible by opening the file in a browser and then going to source view. But I don't know how to extract it from there using a command.

First two lines from one of those files:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<!-- saved from url=(0084)http://www.forexrealm.com/technical-analysis/technical-indicators/typical-price.html -->
2
  • Please show the first 2 lines of one of these html file – glenn jackman May 31 '13 at 16:40
  • See the edited post. These html files are not any special kind of html files. They are just pages saved from internet. – mgtheboss May 31 '13 at 17:59
1

Run the following command in a directory holding html files.

for f in *.html; do echo $(cat "$f" | sed -n '2p' | grep -o 'http://.*\.html') $(stat -c %y "$f");  done > References.tx

Script:

#!/usr/bin/bash

ref_file='References.txt';
> "$ref_file";

for html in *.html
do
    mtime=$(stat -c %y "$html")
    address=$(cat "$html" | sed -n '2p' | grep -o 'http://.*\.html')
    echo "$address $mtime" >> $ref_file;
done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.