Consider this file:
#!/usr/bin/env bash
cat > example_file.txt <<EOL
TITLE something
some data
some data
some data
TITLE something else
some other data
TITLE some more
some other data
some other data
some other data
TITLE extra info
some more data
some more data
EOL
I need to add a new column that:
- counts the number of lines,
- goes back to 1 after an occurrence of
/^TITLE/
, - starts from the bottom of the file and works its way upward,
Basically, the result should look like:
TITLE something,4
some data,3
some data,2
some data,1
TITLE something else,2
some other data,1
TITLE some more,4
some other data,3
some other data,2
some other data,1
TITLE extra info,3
some more data,2
some more data,1
P.S. you can assume that:
- the file always starts with a line matching
/^TITLE/
- the file always ends with a line not matching
/^TITLE/
- there are no two consecutive lines matching
/^TITLE/
Edit:
Results so far
on a 100MB file:
@Yarom
time tac trial.txt | awk 'BEGIN{x=0} {x++;{if ($1 !~/^pattern/) printf "%s,%s\n",$0,x;else if ($1 ~/^pattern/) {printf "%s,%s\n",$0,x;x=0}}}' | tac > trial2.txt
real 0m0,896s
@bac0n
time awk '{ a[i++]=$0 } END { while (i--) { a[i]=a[i] "," ++j; if (a[i] ~ /^pattern/) { j=0 } }; for (i=0; i<NR; i++) { print a[i] } }' trial.txt > trial2.txt
real 0m0,830s
@oliv:
time awk -v RS='^pattern' -v FS='\n' '
{
for(i=NF-1;i>0;i--)
printf "%s,%d\n",$i,i;
printf RT
}' trial.txt > trial2.txt
real 0m2,343s
@steeldriver
time awk -vRS='\n(^pattern|$)' -F'\n' -vOFS=, '
NR>1 {$1 = "^pattern" $1}
{for(i=1;i<=NF;i++) print $i, NF-i+1}
' trial.txt > trial2.txt
real 0m1,889s
using mawk instead of awk, I get:
mawk: program limit exceeded: maximum number of fields size=32767
/^TITLE/
lines start with DATA.TITLE
and notTITLE
, hoping it's better now :)