4

I am currently writing a script to archive a couple log files and want them combined in a single archive which is named according to the first and last lines date and time in one of the log files (i.e. access.log).

But for the best of me I can't wrap my head around how to get this information from the lines and assemble it to a file name.

the lines in question are from an apache.log file which i simply could get with head and tail:

Example:

$ head -n1 /home/server/log/access.log.1 
84.1.11.243 - - [21/Jan/2017:14:53:49 +0000] "GET /index.php/2016/05/26/tutorial-how-to-install-ubuntu-and-other-debian-based-distributions-via-debootstrap/ HTTP/1.1" 200 18413 "https://www.google.hu/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"

$ tail -n1 /home/server/log/access.log.1 
71.3.17.120 - - [20/Dec/2017:16:17:50 +0000] "POST / HTTP/1.1" 200 27639 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; LCTE; rv:11.0) like Gecko"

The expected file name result should be including the date and time stamps from those messages.

Example line I would like to use, but can change that according to the answers results:

tar -caf "backup-logfiles-$start-til-$end.tar.gz" access.log error.log ftp.log

Any solution is welcome to extract that values into $start and $end.

4

Here's a horribly convoluted shell one-liner (using the date format you mentioned in chat):

$ name=$(printf 'backup-logfiles-%s-til-%s' $(date -d "$(head -n1 logfile | grep -oP '\[\K\S+' | sed 's|/| |g; s/:/ /')" +%Y-%m-%d-%H:%M:%S) $(date -d "$(tail -n1 logfile | grep -oP '\[\K\S+' | sed 's|/| |g; s/:/ /')" +%Y-%m-%d-%H:%M:%S))
$ echo $name
logfiles-2017-01-21-14:53:49-til-2017-12-20-16:17:50

To get the start and end variables separately, do:

$ start=$(head -n1 logfile | grep -oP '\[\K\S+' | sed 's|/|-|g; s/:/ /')
$ end=$(tail -n1 logfile | grep -oP '\[\K\S+' | sed 's|/|-|g; s/:/ /')
$ echo "backup-logfiles-$start-til-$end.tar.gz"
backup-logfiles-21-Jan-2017 14:53:49-til-20-Dec-2017 16:17:50.tar.gz

Or, if you want a numerical date:

$ start=$(date -d "$(head -n1 logfile | grep -oP '\[\K\S+' | sed 's|/|-|g; s/:/ /')" +%Y-%m-%d-%H:%M:%S)
$ end=$(date -d "$(tail -n1 logfile | grep -oP '\[\K\S+' | sed 's|/|-|g; s/:/ /')" +%Y-%m-%d-%H:%M:%S)
$ echo "backup-logfiles-$start-til-$end.tar.gz"
backup-logfiles-2017-01-21-14:53:49-til-2017-12-20-16:17:50.tar.gz
4

Using only sed, just for fun winning golf ;)

name=$(sed -rn 's|/|-|g;s/.* \[([^ ]+) .*/\1/;1p;$p' file | sed 'N;s/\n/-til-/')
$ echo $name
21-Jan-2017:14:53:49-til-20-Dec-2017:16:17:50

But if this file is something you want to pass to tar, the colons could cause problems:

An  archive  name  that has a colon in it specifies a file or device on a remote
machine.  The part before the colon is taken as the machine name or IP address,
and the part after it as the file or device pathname, e.g.:

    --file=remotehost:/dev/sr0

You can get around this by passing an option:

--force-local
      Archive file is local even if it has a colon.

But here's a command that replaces the colons with more hyphens:

name=$(sed -rn 's|[/:]|-|g;s/.* \[([^ ]+) .*/\1/;1p;$p' file | sed 'N;s/\n/-til-/')

Instead of a character class we could use alternation and save a byte :)

name=$(sed -rn 's#/|:#-#g;s/.* \[([^ ]+) .*/\1/;1p;$p' file | sed 'N;s/\n/-til-/')

Notes

  • -r use ERE
  • -n don't print anything until we ask for it
  • s|/|-|g replace all the / characters with - (because we can't have a filename with /)
  • s|[/:]|-|g replace / and : characters with hyphens everywhere.
  • s#/|:#-#g replace / or : with - everywhere
  • ; separate sed commands
  • s/.* \[([^ ]+) .*/\1/ capture the date and time from between the square brackets (from the first [ to the first space).
  • 1p;$p print just the first line and the last line
  • | pipe it to another sed (ugh!)
  • N read both lines into the pattern space...
  • s/\n/-til-/ ... so we can replace the newline with -til-

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.