17

Is there a way to delete lines starting with certain strings.

I have this youtube-dl code

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E

and its result is like this

[youtube:playlist] PL1C815DB73EC2678E: Downloading webpage
[download] Downloading playlist: Less than 1 minute
[youtube:playlist] playlist Less than 1 minute: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
[youtube] KNLwsqzFfNg: Downloading webpage
[youtube] KNLwsqzFfNg: Extracting video information
[youtube] KNLwsqzFfNg: Downloading DASH manifest
[download] Destination: _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a

[download]   0.4% of 231.51KiB at  6.10KiB/s ETA 00:30
[download]   1.1% of 231.51KiB at 27.07KiB/s ETA 00:10
[download]   4.0% of 231.51KiB at 19.24KiB/s ETA 00:04
[download]   6.5% of 231.51KiB at 75.06KiB/s ETA 00:03
[download]  13.4% of 231.51KiB at 98.22KiB/s ETA 00:03
[download]  28.7% of 231.51KiB at 81.40KiB/s ETA 00:02
[download]  61.7% of 231.51KiB at 91.56KiB/s ETA 00:01
[download]  86.2% of 231.51KiB at 82.96KiB/s ETA 00:00
[download] 100.0% of 231.51KiB at 73.21KiB/s ETA 00:00
[download] 100% of 231.51KiB in 00:02
[ffmpeg] Correcting container in "_1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a"
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
[avconv] Destination: _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.mp3
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
Deleting original file _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a (pass -k to keep)
[download] Downloading video 2 of 4
[youtube] wTvXkMpJflk: Downloading webpage
[youtube] wTvXkMpJflk: Extracting video information
[youtube] wTvXkMpJflk: Downloading DASH manifest

I want to delete all lines starting with [youtube], [ffmpeg] and [avconv] and get like this

[youtube:playlist] PL1C815DB73EC2678E: Downloading webpage
[download] Downloading playlist: Less than 1 minute
[youtube:playlist] playlist Less than 1 minute: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
[download] Destination: _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a

[download]   0.4% of 231.51KiB at  6.10KiB/s ETA 00:30
[download]   1.1% of 231.51KiB at 27.07KiB/s ETA 00:10
[download]   4.0% of 231.51KiB at 19.24KiB/s ETA 00:04
[download]   6.5% of 231.51KiB at 75.06KiB/s ETA 00:03
[download]  13.4% of 231.51KiB at 98.22KiB/s ETA 00:03
[download]  28.7% of 231.51KiB at 81.40KiB/s ETA 00:02
[download]  61.7% of 231.51KiB at 91.56KiB/s ETA 00:01
[download]  86.2% of 231.51KiB at 82.96KiB/s ETA 00:00
[download] 100.0% of 231.51KiB at 73.21KiB/s ETA 00:00
[download] 100% of 231.51KiB in 00:02
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
Deleting original file _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a (pass -k to keep)
[download] Downloading video 2 of 4
etc..
etc..
.
.

i tried this method but its showing error and it only suppose to delete [youtube]

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | sed '^/[youtube]/ d' 
1
  • Your pattern is wrong. It should be sed '/^\[youtube\]/d'.
    – Pilot6
    Jun 14, 2015 at 13:53

5 Answers 5

17
sed -i '/\[youtube\]/d' /path/to/file

will delete lines, containing "[youtube]".

As one command you can combine patterns like

sed -i '/\[youtube\]\|\[ffmpeg\]\|\[avconv\]/d' /path/to/file

Or right from your command

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 \
 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | 
    sed '/\[youtube\]\|\[ffmpeg\]\|\[avconv\]/d' > output.txt

This will write the result to a file output.txt.

If you want to delete lines not just containing [youtube], but starting with [youtube], then add ^ to the pattern, like sed '/^\[youtube\]/d'.

But in your case it does not matter.

13
  • I tried this sed -i '/\[youtube\]\|\[ffmpeg\]\|\[avconv\]/d' youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E and not working. may be because its not a file and its a download link
    – potholiday
    Jun 14, 2015 at 13:41
  • Try with a file. It will work. Or you can do it right from the link using | pipe.
    – Pilot6
    Jun 14, 2015 at 13:42
  • @potholiday I updated the answer
    – Pilot6
    Jun 14, 2015 at 13:55
  • worked... the edited one worked
    – potholiday
    Jun 14, 2015 at 13:58
  • and for me > output.txt is not necessary
    – potholiday
    Jun 14, 2015 at 14:04
6

I suggest using grep -vE like so:

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | grep -vE '^\[(youtube|ffmpeg|avconv)\]'

From man grep:

-v, --invert-match
              Invert the sense of matching, to select non-matching lines.  (-v
              is specified by POSIX.)



-E, --extended-regexp
              Interpret  PATTERN  as  an extended regular expression (ERE, see
              below).  (-E is specified by POSIX.)

The -E flag is used to avoid escaping square brackets with slashes. Without -E flag you have to escape the square brackets with a backslash, like so grep -vE '\[youtube\]\|\[ffmpeg\]\|\[avconv\]' Edit:

Since you've requested awk,here's one with awk:

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | awk '{if ($0~/^\[youtube\]/||/^\[ffmpeg\]/||/^\[avconv\]/||/^WARNING/) next;print}'

16
  • The [ and ] should be escaped. With the -E option (extended regex option) you can avoid escaping (, ) and | in the command: grep -vE '\[(youtube|ffmpeg|avconv)\]'. As for the "starting requirement", ^ needs to be prepended.
    – Lekensteyn
    Jun 14, 2015 at 13:40
  • @Lekensteyn good point. I'll add that Jun 14, 2015 at 13:40
  • @Serg not working. when executed it shows > and a blinking cursor. may be some syntax problem
    – potholiday
    Jun 14, 2015 at 13:46
  • @potholiday it was missing the single quote at the end. Fixed everything and added awk version. please review Jun 14, 2015 at 13:47
  • grep script doesn’t show any difference from the original code, no change at all. awk i tried like this youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | awk '{if(/[youtube]/||/[ffmpeg]/) $0="";print}' it deletes all except warning lines
    – potholiday
    Jun 14, 2015 at 13:53
2

Use grep -v as following:

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | grep -v '^[youtube]' | grep -v '^[ffmpeg]' | grep -v '^[avconv]'
1

Using Perl:

< inputfile perl -pe 's/^\[(youtube|ffmpeg|avconv)\].*$//' > outputfile

To parse the output of youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E directly, pipe its output to the command without redirecting the content of inputfile:

`youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | perl -pe 's/^\[(youtube|ffmpeg|avconv)\].*$//' > outputfile` 
  • < inputfile: redirects the content of inputfile to perl's stdin
  • > outputfile: redirects the content of perl's stdout to outputfile
  • -p: places a while (<>) { [...] } loop around the script and prints each processed line
  • -e: reads the script from the arguments

Perl script breakdown:

  • s: asserts to perform a substitution
  • /: starts the pattern
  • ^: matches the start of the line
  • \[: matches a [ character
  • (: starts grouping the allowed strings
  • youtube: matches a youtube string
  • |: separates the second allowed string
  • ffmpeg: matches a ffmpeg string
  • |: separates the third allowed string
  • avconv: matches a avconv string
  • ): stops grouping the allowed strings
  • \]: matches a ] character
  • .*: matches any number of any character
  • $: matches the end of the line
  • /: stops the pattern / starts the replacement string
  • /: stops the replacement string
2
  • 1
    you might want to edit the command to include pipe from the OP's youtube-dl command. Others might not understand. Otherwise, I've tested it with an input file with text from OP's post,seems to work, hence +1 Jun 14, 2015 at 14:37
  • Thanks @Serg, I saw you updated yours as well so +1 on yours either
    – kos
    Jun 14, 2015 at 14:46
0

Here is a similar question with easy answer

https://askubuntu.com/questions/1493280/remove-lines-from-a-txt-file-that-start-with-a-special-string?noredirect=1#comment2616486_1493280 

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .