Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have a lot of html files in a folder. I used them in a project. I want to build a References.txt file in which the URL from which each html file was saved will be listed with the time stamp on which it was saved. There will be a space between each URL and its corresponding timestamp.

How can I do that from Ubuntu 12.04 using a terminal command or shell script?

One information which can help you is that the URL from which a html file was saved is present in the second line of the html file and that is also visible by opening the file in a browser and then going to source view. But I don't know how to extract it from there using a command.

First two lines from one of those files:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<!-- saved from url=(0084)http://www.forexrealm.com/technical-analysis/technical-indicators/typical-price.html -->
share|improve this question
    
Please show the first 2 lines of one of these html file –  glenn jackman May 31 '13 at 16:40
    
See the edited post. These html files are not any special kind of html files. They are just pages saved from internet. –  Manojit Ghosh May 31 '13 at 17:59
add comment

1 Answer

Run the following command in a directory holding html files.

for f in *.html; do echo $(cat "$f" | sed -n '2p' | grep -o 'http://.*\.html') $(stat -c %y "$f");  done > References.tx

Script:

#!/usr/bin/bash

ref_file='References.txt';
> "$ref_file";

for html in *.html
do
    mtime=$(stat -c %y "$html")
    address=$(cat "$html" | sed -n '2p' | grep -o 'http://.*\.html')
    echo "$address $mtime" >> $ref_file;
done
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.