12

Currently, I have multiple text files with contents looking like this (with many lines):

565 0 10 12 23 18 17 25
564 1 7 12 13 16 18 40 29 15

I wish to change each line to have the following format:

0 565:10:1 565:12:1 565:23:1 565:18:1 565:17:1 565:25:1
1 564:7:1 564:12:1 564:13:1 564:16:1 564:18:1 564:40:1 564:29:1 564:15:1

Are there any way of doing the above using sed? Or do I need to resort to Python?

Improve this question
0

5 Answers 5

Reset to default
22

You could do it with sed, yes, but other tools are simpler. For example:

$ awk '{
        printf "%s ", $2; 
        for(i=3;i<=NF;i++){
            printf "%s:%s:1 ",$1,$(i) 
        }
        print ""
       }' file 
0 565:10:1 565:12:1 565:23:1 565:18:1 565:17:1 565:25:1 
1 564:7:1 564:12:1 564:13:1 564:16:1 564:18:1 564:40:1 564:29:1 564:15:1

Explanation

awk will split each line of input on whitespace (by default), saving each fields as $1, $2, $N. So:

  • printf "%s ", $2; will print the 2nd field and a trailing space.
  • for(i=3;i<=NF;i++){ printf "%s:%s:1 ",$1,$(i) } : will iterate over fields 3 to the last field (NF is the number of fields) and for each of them it will print the 1st field, a :, then the current field and a :1.
  • print "" : this just prints a final newline.

Or Perl:

$ perl -ane 'print "$F[1] "; print "$F[0]:$_:1 " for @F[2..$#F]; print "\n"' file 
0 565:10:1 565:12:1 565:23:1 565:18:1 565:17:1 565:25:1 
1 564:7:1 564:12:1 564:13:1 564:16:1 564:18:1 564:40:1 564:29:1 564:15:1

Explanation

The -a makes perl behave like awk and split its input on whitespace. Here, the fields are stored in the array @F, meaning that the 1st field will be $F[0], the 2nd $F[1] etc. So:

  • print "$F[1] " : print the 2nd field.
  • print "$F[0]:$_:1 " for @F[2..$#F]; : iterate over fields 3 to the last field ($#F is the number of elements in the array @F, so @F[2..$#F] takes an array slice starting at the 3rd element until the end of the array) and print the 1st field, a :, then the current field and a :1.
  • print "\n" : this just prints a final newline.
Improve this answer
12

Here is a horrible sed way!

$ sed -r 's/^([0-9]+) ([0-9]+) ([0-9]+)/\2 \1:\3:1/; :a s/([0-9]+)(:[0-9]+:1) ([0-9]+)( |$)/\1\2 \1:\3:1 /; t a; s/ $//' file
0 565:10:1 565:12:1 565:23:1 565:18:1 565:17:1 565:25:1 
1 564:7:1 564:12:1 564:13:1 564:16:1 564:18:1 564:40:1 564:29:1 564:15:1

More readably:

sed -r '
s/^([0-9]+) ([0-9]+) ([0-9]+)/\2 \1:\3:1/
:a 
s/([0-9]+)(:[0-9]+:1) ([0-9]+)( |$)/\1\2 \1:\3:1 /
t a
s/ $//'

Notes

  • -r use ERE
  • s/old/new/ replace old with new
  • ^([0-9]+) save some numbers at the start of the line
  • \1 backreference to first saved pattern
  • :a label this section of the script a
  • ( |$) either a space or the end of the line
  • t test whether the last replacement was successful - if it was, then do the next command
  • a find the label :a and do it again
  • s/ $// remove the trailing space

So after adding the structure to the first part, we repeatedly find the last instance of the structure and apply it to the next number...

But I agree other tools make it easier...

Improve this answer
2
  • I was waiting for your sed solution :D
    – Ravexina ♦
    Jun 5, 2017 at 8:57
  • :D it took me a while @Ravexina - I reckon muru can make a cleaner one
    – Zanna ♦
    Jun 5, 2017 at 8:59
5

With awk:

awk '{printf "%s ",$2; for (i=3; i<=NF; i++) printf $1":"$i":1 "; printf "\n"}' file

or with bash:

while read -r -a a; do                  # read line to array a
  printf "%s " ${a[1]}                  # print column #1
  for ((i=2;i<${#a[@]};i++)); do        # loop from column #2 to number of columns
    printf "%s " "${a[0]}:${a[$i]}:1"   # print content/values
  done
  echo                                  # print line break
done < file                             # read file from stdin

Output:

0 565:10:1 565:12:1 565:23:1 565:18:1 565:17:1 565:25:1 
1 564:7:1 564:12:1 564:13:1 564:16:1 564:18:1 564:40:1 564:29:1 564:15:1
Improve this answer
5

Well, you can do it in sed, but python works also.

$ ./reformatfile.py  input.txt                                                                        
0 565:10:1 565:12:1 565:23:1 565:18:1 565:17:1 565:25:1
1 564:7:1 564:12:1 564:13:1 564:16:1 564:18:1 564:40:1 564:29:1 564:15:1

Contents of the reformatfile.py are as so:

#!/usr/bin/env python3
import sys

with open(sys.argv[1]) as fd:
    for line in fd:
        words = line.strip().split()
        pref = words[0]
        print(words[1],end=" ")
        new_words = [ ":".join([pref,i,"1"]) for i in words[2:] ]
        print(" ".join(new_words))

How does this work? There's really nothing particularly special going on. We open first command-line argument as file for reading and proceed with breaking down each line into "words" or individual items. First words becomes pref variable, and we print on stdout second ( words[1] ) item ending with space. Next we construct new set of "words" via list comprehensions and .join() function on a temporary list of pref, each word, and string "1". Final step is to print those out

Improve this answer
4

With awk:

awk '{printf("%s ", $2); for(i=3; i<NF; i++) printf("%s:%s:1 ", $1, $i);\
          printf("%s:%s:1\n", $1, $NF)}' file.txt

It's all about formatting space-separated fields in desired format:

  • printf("%s ", $2) prints the second field with a trailing space

  • for(i=3; i<NF; i++) printf("%s:%s:1 ", $1, $i) iterates over the 3rd to second last fields and prints the fields in desired format (first field, then a colon, then the current field, then a colon, finally 1) with a trailing space

  • printf("%s:%s:1\n", $1, $NF) prints the last field with newline

Example:

% cat file.txt
565 0 10 12 23 18 17 25
564 1 7 12 13 16 18 40 29 15

% awk '{printf("%s ", $2); for(i=3; i<NF; i++) printf("%s:%s:1 ", $1, $i); printf("%s:%s:1\n", $1, $NF)}' file.txt
0 565:10:1 565:12:1 565:23:1 565:18:1 565:17:1 565:25:1
1 564:7:1 564:12:1 564:13:1 564:16:1 564:18:1 564:40:1 564:29:1 564:15:1
Improve this answer

You must log in to answer this question.