2

The idea is this: I have a document with data listed in the following way (shown below), and I want to delete any line that has a value of 70 or less in the 6th (last) column. On top of that, I want to delete any line after it with a matching 3rd column (a latitude coordinate).
Here is the catch, once the latitude changes (3rd column) I want to restart the hunt for the first sub 70 value and delete any lines afterwards in similar fashion until I hit a new latitude value.

The catch is not every 'grouping' of same latitudes has the same amount of lines, nor do they always end on 48.15 (1st column).

I have been attempting to utilize sed -z options, however I am having a hard time creating a proper string pattern that recognizes what I want to delete, and only deletes up to the next grouping before restarting the string.

6.15 21.31750 29.11549 -70.2565 28.7203 99.00000
8.15 21.31750 29.11549 -90.3238 35.0326 99.00000
10.15 21.31750 29.11549 -84.9625 43.2992 100.00000
12.15 21.31750 29.11549 -77.0993 44.3515 99.00000
14.15 21.31750 29.11549 -70.9164 49.5554 96.00000
16.15 21.31750 29.11549 -82.2717 38.6834 98.00000
18.15 21.31750 29.11549 -83.7156 35.6462 99.00000
20.15 21.31750 29.11549 -83.9505 35.1276 95.00000
22.15 21.31750 29.11549 -167.575 66.1472 57.00000
24.15 21.31750 29.11549 -94.5072 38.15 98.00000
26.15 21.31750 29.11549 -38.9739 69.8479 71.00000
28.15 21.31750 29.11549 -94.5072 38.15 98.00000
30.15 21.31750 29.11549 -28.6407 51.3899 24.00000
32.15 21.31750 29.11549 -43.7269 119.979 2.00000
34.15 21.31750 29.11549 -69.6645 150 0.00000
36.15 21.31750 29.11549 -103.964 150 1.00000
38.15 21.31750 29.11549 42.6041 142.656 0.00000
40.15 21.31750 29.11549 109.056 150 0.00000
42.15 21.31750 29.11549 -14.6037 150 45.00000
44.15 21.31750 29.11549 -118.694 53.7305 94.00000
46.15 21.31750 29.11549 -167.053 115.74 92.00000
48.15 21.31750 29.11549 -171.917 150 66.00000
6.15 20.38500 29.12283 -87.9018 36.2993 100.00000
8.15 20.38500 29.12283 -98.356 43.8404 100.00000
10.15 20.38500 29.12283 -88.9825 46.6824 100.00000
12.15 20.38500 29.12283 -78.2202 44.7168 97.00000
14.15 20.38500 29.12283 -78.1702 42.5794 97.00000
16.15 20.38500 29.12283 -76.6382 40.3678 98.00000
18.15 20.38500 29.12283 -79.449 49.3087 95.00000
20.15 20.38500 29.12283 -137.565 45.7575 66.00000
22.15 20.38500 29.12283 -112.652 37.5735 100.00000
24.15 20.38500 29.12283 -55.8986 43.9287 54.00000
26.15 20.38500 29.12283 -50.4227 48.2312 70.00000
28.15 20.38500 29.12283 -55.8986 43.9287 54.00000
30.15 20.38500 29.12283 -57.3999 98.6111 8.00000
32.15 20.38500 29.12283 -74.2068 150 6.00000
36.15 20.38500 29.12283 17.7038 117.808 5.00000
38.15 20.38500 29.12283 -5.36164 96.0492 0.00000
40.15 20.38500 29.12283 -98.5051 99.8733 42.00000
42.15 20.38500 29.12283 -149.328 41.7056 96.00000
44.15 20.38500 29.12283 -172.026 126.696 92.00000
46.15 20.38500 29.12283 -174.664 150 76.00000
48.15 20.38500 29.12283 -176.269 139.467 31.00000

So I would want to delete everything from 22.15 to 48.15 in the first grouping, and from 20.15 to 48.15 in the second.

bash format is ideal, as I am using GMT5 (only accepts bash).

Any help will be appreciated.

3
  • I will edit that, it was a misprint. My first question to the site so still getting used to the formatting.
    – T. Byrne
    Apr 5, 2016 at 20:33
  • The file is much longer, and the values are not static. This is essentially a temp folder used in a loop to store the data for usage and then update it when the next file in the loop (or case/switch statement) comes along
    – T. Byrne
    Apr 5, 2016 at 20:35
  • Don't forget to tick the icon to the left of the answer to accept it as the solution, thanks.
    – mchid
    Apr 5, 2016 at 20:51

1 Answer 1

2

Using Perl:

perl -lane 'if($F[5] < 70 || $F[2] == $x) {$x = $F[2]; next} undef $x; print' file
perl -lane '
    if($F[5] < 70 || $F[2] == $x) {
        $x = $F[2];
        next
    }
    undef $x;
    print
' file
  • -l[octnum]: enables automatic line-ending processing. It has two separate effects. First, it automatically chomps $/ (the input record separator) when used with -n or -p. Second, it assigns $\ (the output record separator) to have the value of octnum so that any print statements will have that separator added back on. If octnum is omitted, sets $\ to the current value of $/.
  • -a: turns on autosplit mode when used with a -n or -p. An implicit split command to the @F array is done as the first thing inside the implicit while loop produced by the -n or -p.
  • -n: causes Perl to assume the following loop around your program, which makes it iterate over filename arguments somewhat like sed -n or awk:

    LINE:
      while (<>) {
          ...             # your program goes here
      }
    
  • -e: may be used to enter one line of program;

  • if($F[5] < 70 || $F[2] == $x) {$x = $F[2]; next} print: if the sixth field contains a number lower than 70 or the second field contains a number equal to $x, assigns the second fields to $x and skips to the next record; otherwise unsets $x and prints the record.
% cat file
6.15 21.31750 29.11549 -70.2565 28.7203 99.00000
8.15 21.31750 29.11549 -90.3238 35.0326 99.00000
10.15 21.31750 29.11549 -84.9625 43.2992 100.00000
12.15 21.31750 29.11549 -77.0993 44.3515 99.00000
14.15 21.31750 29.11549 -70.9164 49.5554 96.00000
16.15 21.31750 29.11549 -82.2717 38.6834 98.00000
18.15 21.31750 29.11549 -83.7156 35.6462 99.00000
20.15 21.31750 29.11549 -83.9505 35.1276 95.00000
22.15 21.31750 29.11549 -167.575 66.1472 57.00000
24.15 21.31750 29.11549 -94.5072 38.15 98.00000
26.15 21.31750 29.11549 -38.9739 69.8479 71.00000
28.15 21.31750 29.11549 -94.5072 38.15 98.00000
30.15 21.31750 29.11549 -28.6407 51.3899 24.00000
32.15 21.31750 29.11549 -43.7269 119.979 2.00000
34.15 21.31750 29.11549 -69.6645 150 0.00000
36.15 21.31750 29.11549 -103.964 150 1.00000
38.15 21.31750 29.11549 42.6041 142.656 0.00000
40.15 21.31750 29.11549 109.056 150 0.00000
42.15 21.31750 29.11549 -14.6037 150 45.00000
44.15 21.31750 29.11549 -118.694 53.7305 94.00000
46.15 21.31750 29.11549 -167.053 115.74 92.00000
48.15 21.31750 29.11549 -171.917 150 66.00000
6.15 20.38500 29.12283 -87.9018 36.2993 100.00000
8.15 20.38500 29.12283 -98.356 43.8404 100.00000
10.15 20.38500 29.12283 -88.9825 46.6824 100.00000
12.15 20.38500 29.12283 -78.2202 44.7168 97.00000
14.15 20.38500 29.12283 -78.1702 42.5794 97.00000
16.15 20.38500 29.12283 -76.6382 40.3678 98.00000
18.15 20.38500 29.12283 -79.449 49.3087 95.00000
20.15 20.38500 29.12283 -137.565 45.7575 66.00000
22.15 20.38500 29.12283 -112.652 37.5735 100.00000
24.15 20.38500 29.12283 -55.8986 43.9287 54.00000
26.15 20.38500 29.12283 -50.4227 48.2312 70.00000
28.15 20.38500 29.12283 -55.8986 43.9287 54.00000
30.15 20.38500 29.12283 -57.3999 98.6111 8.00000
32.15 20.38500 29.12283 -74.2068 150 6.00000
36.15 20.38500 29.12283 17.7038 117.808 5.00000
38.15 20.38500 29.12283 -5.36164 96.0492 0.00000
40.15 20.38500 29.12283 -98.5051 99.8733 42.00000
42.15 20.38500 29.12283 -149.328 41.7056 96.00000
44.15 20.38500 29.12283 -172.026 126.696 92.00000
46.15 20.38500 29.12283 -174.664 150 76.00000
48.15 20.38500 29.12283 -176.269 139.467 31.00000
user@user-X550CL ~/tmp % perl -lane 'if($F[5] < 70 || $F[2] == $x) {$x = $F[2]; next} undef $x; print' file
6.15 21.31750 29.11549 -70.2565 28.7203 99.00000
8.15 21.31750 29.11549 -90.3238 35.0326 99.00000
10.15 21.31750 29.11549 -84.9625 43.2992 100.00000
12.15 21.31750 29.11549 -77.0993 44.3515 99.00000
14.15 21.31750 29.11549 -70.9164 49.5554 96.00000
16.15 21.31750 29.11549 -82.2717 38.6834 98.00000
18.15 21.31750 29.11549 -83.7156 35.6462 99.00000
20.15 21.31750 29.11549 -83.9505 35.1276 95.00000
6.15 20.38500 29.12283 -87.9018 36.2993 100.00000
8.15 20.38500 29.12283 -98.356 43.8404 100.00000
10.15 20.38500 29.12283 -88.9825 46.6824 100.00000
12.15 20.38500 29.12283 -78.2202 44.7168 97.00000
14.15 20.38500 29.12283 -78.1702 42.5794 97.00000
16.15 20.38500 29.12283 -76.6382 40.3678 98.00000
18.15 20.38500 29.12283 -79.449 49.3087 95.00000
8
  • I will attempt to use this. However I am inputting to GMT 5, so I will need to do some tweaking from perl to bash
    – T. Byrne
    Apr 5, 2016 at 20:41
  • I will edit post to mention bash/GMT5
    – T. Byrne
    Apr 5, 2016 at 20:41
  • 1
    @T.Byrne This perl command works in bash. Maybe you are thinking about the % which is the same as bash is $ in zsh.
    – mchid
    Apr 5, 2016 at 20:43
  • @T.Byrne Exactly, all this command requires is to be run in Bash or a compatible shell. What's the concern exactly?
    – kos
    Apr 5, 2016 at 20:47
  • @mchid ok thanks! I have been teaching myself bash/csh for my job, and haven't had much formal training. Good to know that perl commands can run in bash.
    – T. Byrne
    Apr 5, 2016 at 20:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .