I have a list which is an output from a media program actually and i want to remove all the junk so i can use that list to my Zenity script but involves alot of sed commands in one string which is the way i prefer im open any suggestion that involves only sed.

I will explain exactly what i'm trying to succeed:

The actual list is List.txt: enter image description here The * is just hex 1b

*line here is not needed*
*[1m0  *[22m : *[35mThis has SPACES inside, Must be DOTS!*[39m : *[34m123.123ABCD*[39m
*[1m1  *[22m : *[35mThis-DONT-have(SPACES)-inside*[39m : *[34m900.578CD*[39m
*[1m2  *[22m : *[35mThis SPACE!!! inside, # Must be DOTS!*[39m : *[34m9K99123AD*[39m
*[1m3  *[22m : *[35mSPACES *Must* be DOTS!*[39m : *[34mAB*[39m
*[1m4  *[22m : *[35mMust[be] [DOTS] !*[39m : *[34m12.AZZZ*[39m
*[1m5  *[22m : *[35mSPACES ~DOTS!*[39m : *[34m654KK45*[39m

This is my sed equation:

cat ~/List.txt | sed -e 1d -e 's/\x1b\[1m//g' -e 's/\x1b\[22m : \x1b\[35m/|---|/g' -e 's/\x1b\[39m : /|---| /g' -e 's/.|---|*//' -e 's/|---|*//' -e 's/\x1b*\[34m/(/g' -e 's/\x1b\[39m/)/g' -e 's/^/FALSE /' -e '0,/FALSE /s//TRUE /' > sedList.txt

This is the sedList.txt which has spaces between the words:

TRUE 0 This has SPACES inside, Must be DOTS! (123.123ABCD)
FALSE 1 This-DONT-have(SPACES)-inside (900.578CD)
FALSE 2 This SPACE!!! inside, # Must be DOTS! (9K99123AD)
FALSE 4 Must[be] [DOTS] ! (12.AZZZ)

I want to replace those spaces with dots in the specific location (not everything) so the sedList.txt looks exacly like this:

TRUE 0 This.has.SPACES.inside,.Must.be.DOTS! (123.123ABCD)
FALSE 1 This-DONT-have(SPACES)-inside (900.578CD)
FALSE 2 This.SPACE!!!.inside,.#.Must.be.DOTS! (9K99123AD)
FALSE 4 Must[be].[DOTS].! (12.AZZZ)

So what is missing from my sed equation should i add something?


I guess you were almost there, I just replaced all spaces by dots first.

sed -e 1d \
        -e 's/\s/./g' \
        -e 's/x1b\[1m0/TRUE 0/g' \
        -e 's/x1b\[1m/FALSE /g' \
        -e 's/\.\./ /g' \
        -e 's/\!x1b\[39m/\!/g' \
        -e 's/x1b\[34m/\(/g' \
        -e 's/x1b\[39m/\)/g' \
        -e 's/x1b\[22m\.\:\.x1b\[35m//g' \
        -e 's/\.\./\./g' \
        -e 's/\.\:\./ /g' \
~/List.txt > SEDList.txt
  • It get's all mash up with dots and hexes please see my examples above see SEDList.txt how should look – GoldHaloWings Oct 23 '17 at 21:06
  • @GoldHaloWings I just tested this and it works perfectly. You might need to replace the x1bs in this answer with \x1b. – wjandrea Oct 24 '17 at 1:53
  • @wjandrea What the heck? When i tested with zenity with an actual output list (from the media program) it didn't work and then i test it with the List.txt it worked i don't get this, so is working but is not wotking with the actual output i guess? – GoldHaloWings Oct 24 '17 at 11:26
  • @J. Starnes SED string working only with List.txt not with actual output your answer is correct thank you. – GoldHaloWings Oct 24 '17 at 11:28
  • @J. Starnes im sure it got to do with the fact that x1b must be /\x1b pls take a look above my bad sed equation plus a posted a picture an example of the text to get the idea – GoldHaloWings Oct 24 '17 at 12:04

Since you ask for sed:

sed -re 1d \
  -e 's/\x1b\[[0-9]+m//g' \
  -e 'y/ /./' \
  -e 's/^([0-9]+)\.*:\./FALSE \1 /'\
  -e 's/\.:\.(.*)/ (\1)/'\
  -e '1s/FALSE/TRUE/'

This one produces the desired output:

sed '1d;s/\x1b\[[0-9]*m//g;s/  *: /\n/g;y/ /./;s/^/FALSE /;2s/FALSE/TRUE/;s/$/)/;s/\n/ /;s// (/' yourfile

My hints for you:

  • First of all: Don't cat and pipe to sed. Rather give the filename as argument to sed.
  • Next, handle all Escape-sequences at once with a regular expression like \x1b\[[0-9]*m to simplify your script
  • Avoid "magical" patterns like your |---|. With GNU sed you can use newline instead. This is unique and easy to handle.

Commented version:

sed '1d;                # drop the first line
     s/\x1b\[[0-9]*m//g;# remove all esc sequences
     s/  *: /\n/g;      # replace the colons with spaces by newlines
     y/ /./;            # replace the spaces with dots
     s/^/FALSE /;       # add the `FALSE` to the beginning
     2s/FALSE/TRUE/;    # except for the second line
     s// (/;            # first separator becomes space
     s/$/)/;            # second becomes space with (
     s/\n/ /;           # finally the trailing )' yourfile
  • Can SED strings be syntaxed like this; echo "$TheList" | sed -e ... -e ... -e In more detail how the list is syntaxed in my script: VAR=$(zenity ... ... ... $(echo "$TheList" | sed -e ... -e ... -e)) – GoldHaloWings Oct 24 '17 at 10:40
  • That's not a question of sed, but of escaping inside $(...) – Philippos Oct 24 '17 at 12:29

While the following is not as fast as a single sed call, it adds flexibility.

sed \
    -e 1d  \
    -re 's/\x1b\[[0-9]+m//g' \
    -e 's/:/ /g' List.txt |
awk '{
    if ($1=="0")
        {printf("TRUE "$1" "); for (i=2; i<=(NF-1); i++) printf("%s%s",".",$i); printf(" ("$NF")\n")}
        {printf("FALSE "$1" "); for (i=2; i<=(NF-1); i++) printf("%s%s",".",$i); printf(" ("$NF")\n")}
    }' |
sed 's/ \./ /g' > SEDList.txt

As a single line sutible for use in a script.

"$( sed -e 1d -re 's/\x1b\[[0-9]+m//g' -e 's/:/ /g' "$TheList" | awk '{ if ($1=="0") {printf("TRUE "$1" "); for (i=2; i<=(NF-1); i++) printf("%s%s",".",$i); printf(" ("$NF")\n")} else {printf("FALSE "$1" "); for (i=2; i<=(NF-1); i++) printf("%s%s",".",$i); printf(" ("$NF")\n")} }' | sed 's/ \./ /g' )"

I see this question as composed of 2 problems: (1) remove the ansi-seq and (2) adjust and clean the format. The first part obfuscates the second.

Here a give just a partial answer: I suggest the use of ansifilter for an initial cleaning of ansi-sequences:

ansifilter ~/List.txt | ...

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.