# Looping through csv, removing patterns

I have a question similar to my earlier question, but a bit different

say you have 10 columns on each line, and 100000 lines, I want to check if each row for the following patterns:

If a row looks like this, i want to remove it

`````` lambda a, b, c, d: a==a and a+1==b and b+1==c and c+1==d and d+1==e
``````

While this solution works great, all solutions worked, all followed the same logic. The problem is that say i have 10 rows, That solution removes lines that have 5 consecutive numbers starting from a to e,

If the sequence starts from b to f, it will stay

I want a solution that is controlled by a loop so i won't have to keep adding and removing variables

I mean I want to remove all the sequences greater than 3, in the row,

I mean that if there are 3 consecutive numbers in the row, `a==a and a+1==b and b+1==c` leave them, if there are more than 3 consecutive numbers `a==a and a+1==b and b+1==c and c+1==d and so on` then remove them, wherever they may occur.

I mean it may be

``````a==a and a+1==b and b+1==c and c+1==d

a==a and a+1==a and b+1==c and c+1==d and d+1==e and e+1==f

and so on
``````

So I'd rather to do it in a loop or similar solution, so I don't have to write all possibilities

Like last time, I'd prefer awk or perl for learning purposes, I was told by a friend that haskell can do that easily but I didn't find the solution in haskell, if that's true then I'm curious to know how.

Please note that I don't mind recursive solutions. I will edit my question to give input and output example in few hours, when I get back home

EDIT

example input

``````1,2,3,4,5,6,7,8,9
1,5,8,11,14,16,17,18,19
1,2,3,5,7,9,15,29,34
11,22,25,30,40,55,80,83,90
``````

expected output

``````1,2,3,5,7,9,15,29,34
11,22,25,30,40,55,80,83,90
``````

Reason: the third row, contains only 3 consecutive numbers, fourth row, contains no consecutive numbers. first two rows should be removed.

• It's pretty unclear - can you give an example with a couple of lines input, and the output you like to get? Also, for what reason are you proposing to implement it with a loop? Is the loop in itself really important? – Volker Siegel Oct 1 '14 at 13:15
• With "10 rows on each line", do you mean "10 columns on each line"? – Volker Siegel Oct 1 '14 at 13:17
• @VolkerSiegel the loop isnt important, recursive would work too, I will edit my question to give example when Im back home in few hours, cant do much on the phone and will let u know. Yes I meant 10 columns, I will try to fix that from my phone now – Lynob Oct 1 '14 at 13:43
• @VolkerSiegel I edited my question to include examples, anything else just let me know – Lynob Oct 2 '14 at 19:54

tested:

``````awk -F, '{
for (i=1; i<=NF-3; i++)
if (\$i+1==\$(i+1) && \$i+2==\$(i+2) && \$i+3==\$(i+3))
next
print
}' file
``````
• ok thank you sir, i'll test your solution tomorrow, to see how it works, and i have edited my question to include examples,if my question was unclear – Lynob Oct 2 '14 at 19:56
• tested against your data and fixed. – glenn jackman Oct 2 '14 at 20:09
• Ok thank you I'll test it tomorrow and let u know what happens – Lynob Oct 2 '14 at 20:30
• works perfectly - what is the NF-3 doing exactly? – Lynob Oct 4 '14 at 0:05
• NF is an awk variable: it's the number of fields in the current record. So NF-3 is the 4th last field. – glenn jackman Oct 4 '14 at 1:50

Perl: (reedited, after the examples were added to the question)

from commandline:

``````perl -F, -ane'my(\$a,\$b)=\$F[0];\$b=\$b>(\$l=\$a++==\$_?\$l+1:0)?\$b:\$l for@F;print if \$b>5' \
file.txt
``````

if you want to change the file, use the `-i` flag for the command line version, or use the redirect operator of your shell (`>`, do not use it on the same file, because it will be truncated before it is read!).

• ok thank you sir, i'll test your solution tomorrow, to see how it works, and i have edited my question to include examples,if my question was unclear – Lynob Oct 2 '14 at 19:56
• Oh, I misunderstood your question, I thought only the beginning was interesting. – Patrick J. S. Oct 2 '14 at 19:59
• No, in that case, the answers to my previous question would work fine, what was bad about those solutions is that they won't remove all, sequences, I need to keep changing the script and run it again – Lynob Oct 2 '14 at 20:07
• edited, should work now. Maybe I'll write an explanation tomorrow, since the script changed completely. – Patrick J. S. Oct 2 '14 at 21:50