5

I need a one-line command to compile and print all of the Expendable Launch Vehicle names listed in a log file.

The ELV names are all listed in capital letters under the /elv directory.

The output should appear in the format of one name per line, with no duplicates:

ALICE
BOB
CHARLIE

I tried

grep "GET" NASA_access_log_Aug95.txt | grep "ELV" | wc -l

but it only showed me the number of ELV not printed ELV names

Below is a sample of my log file NASA_access_log_Aug95.txt:

cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:31 -0400] "GET /elv/TITAN/mars1s.jpg HTTP/1.0" 200 1156
www-a2.proxy.aol.com - - [03/Aug/1995:20:43:31 -0400] "GET /elv/DELTA/dsolids.jpg HTTP/1.0" 200 24558
cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:32 -0400] "GET /elv/TITAN/mars3s.jpg HTTP/1.0" 200 1744
castor.gel.usherb.ca - - [03/Aug/1995:20:43:33 -0400] "GET /shuttle/missions/51-l/movies/ HTTP/1.0" 200 372
cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:33 -0400] "GET /elv/ATLAS_CENTAUR/atc69s.jpg HTTP/1.0" 200 1659
cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:35 -0400] "GET /elv/TITAN/mars2s.jpg HTTP/1.0" 200 1549
palona1.cns.hp.com - - [03/Aug/1995:20:43:36 -0400] "GET /shuttle/missions/sts-69/count69.gif HTTP/1.0" 200 46053
www-c1.proxy.aol.com - - [03/Aug/1995:20:43:38 -0400] "GET /shuttle/missions/sts-71/images/KSC-95EC-0882.gif HTTP/1.0" 200 51289
cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:40 -0400] "GET /elv/ATLAS_CENTAUR/acsuns.jpg HTTP/1.0" 200 2263
cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:41 -0400] "GET /elv/ATLAS_CENTAUR/goess.jpg HTTP/1.0" 200 1306
cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:45 -0400] "GET /elv/DELTA/dsolidss.jpg HTTP/1.0" 200 1629 
6

I would just use grep and sort -u:

$ grep -Po '/elv/\K[^/]+' NASA_access_log_Aug95.txt | sort -u
ATLAS_CENTAUR
DELTA
TITAN

The -P enables Perl Compatible Regular Expressions which let us use \K which means "ignore anything matched up to this point". The -o means "only show the matching portion of the line". Then, the regular expression means "look for /elv/, ignore everything matched until the /elv/, and then look for one or more non-/ characters ([^/]+).

6

You need just:

awk -F'/' '/elv/ && !seen[$5]++ {print $5}' infile

This prints the fifth field per slash / as the field separator, if it's not previously set in the array named seen as well as the line should contain elv. See also How does awk '!a[$0]++' work? and this answer on Stack Overflow.

For the given sample, the output would be:

TITAN
DELTA
ATLAS_CENTAUR
  • /GET \/elv\// or $4 == "elv" might be better search commands in case elv also appears in the host name or other path parts. – David Foerster Oct 5 '18 at 8:04
5

You can do it like this:

grep 'elv' NASA_access_log_Aug95.txt | awk '{print $7}' | sed 's/[a-z0-9./]//g' | sort -u

Given your example snippet from the log file this will output:

ATLAS_CENTAUR
DELTA
TITAN

Explanation of the piped commands in order they occur:

  • grep 'elv' NASA_access_log_Aug95.txt

    Will output you all lines containing elv

    cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:31 -0400] "GET /elv/TITAN/mars1s.jpg HTTP/1.0" 200 1156
    www-a2.proxy.aol.com - - [03/Aug/1995:20:43:31 -0400] "GET /elv/DELTA/dsolids.jpg HTTP/1.0" 200 24558
    cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:32 -0400] "GET /elv/TITAN/mars3s.jpg HTTP/1.0" 200 1744
    cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:33 -0400] "GET /elv/ATLAS_CENTAUR/atc69s.jpg HTTP/1.0" 200 1659
    cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:35 -0400] "GET /elv/TITAN/mars2s.jpg HTTP/1.0" 200 1549
    cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:40 -0400] "GET /elv/ATLAS_CENTAUR/acsuns.jpg HTTP/1.0" 200 2263
    cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:41 -0400] "GET /elv/ATLAS_CENTAUR/goess.jpg HTTP/1.0" 200 1306
    cc-rd6-mg1-dip4-9.massey.ac.nz - - [03/Aug/1995:20:43:45 -0400] "GET /elv/DELTA/dsolidss.jpg HTTP/1.0" 200 1629 
    
  • awk '{print $7}'

    Will give you the 7th column information (the one you want). Remember That this counts colums divided by spaces.

    /elv/TITAN/mars1s.jpg
    /elv/DELTA/dsolids.jpg
    /elv/TITAN/mars3s.jpg
    /elv/ATLAS_CENTAUR/atc69s.jpg
    /elv/TITAN/mars2s.jpg
    /elv/ATLAS_CENTAUR/acsuns.jpg
    /elv/ATLAS_CENTAUR/goess.jpg
    /elv/DELTA/dsolidss.jpg
    
  • sed 's/[a-z0-9./]//g'

    Will filter out all unwanted characters (i.e. lower case a-z, numbers 0-9, . and /)

    TITAN
    DELTA
    TITAN
    ATLAS_CENTAUR
    TITAN
    ATLAS_CENTAUR
    ATLAS_CENTAUR
    DELTA
    
  • sort -u

    Will prevent duplicates from appearing and sorts them alphabetically.

    ATLAS_CENTAUR
    DELTA
    TITAN
    
  • 1
    tr -d a-z0-9./ would suffice in place of sed. – minmaxavg Sep 30 '18 at 11:21
5

With Perl, regex matching the /-delimited elements after elv and pushing them into a hash:

$ perl -lne '$h{$1}++ if m:/elv/(.*?)/: }{ for $k (sort keys %h) {print $k}' NASA_access_log_Aug95.txt 
ATLAS_CENTAUR
DELTA
TITAN
5

You can also use sed with just a little help from sort

$ sed -rn '\|/elv/| s|.*/elv/([^/]+).*|\1|p' NASA_access_log_Aug95.txt | sort -u
ATLAS_CENTAUR
DELTA
TITAN

Explanation

  • -r Use extended regex (saves a couple of backslashes)
  • -n Don't print the lines we don't ask for
  • \|/elv/| find lines with /elv/ (the \| at the start means use | not / to delimit the address)
  • s|old|new| replace old with new
  • .*/elv/ any characters before and including /elv/
  • ([^/]+) save all the characters until the next /
  • .* any number of any characters
  • \1 reference to the characters we saved
  • p print the lines we worked on
  • sort -u sort the input and remove duplicates

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.