5

Is there a way to delete lines starting with certain strings.

I have this youtube-dl code

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E

and its result is like this

[youtube:playlist] PL1C815DB73EC2678E: Downloading webpage
[download] Downloading playlist: Less than 1 minute
[youtube:playlist] playlist Less than 1 minute: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
[youtube] KNLwsqzFfNg: Downloading webpage
[youtube] KNLwsqzFfNg: Extracting video information
[youtube] KNLwsqzFfNg: Downloading DASH manifest
[download] Destination: _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a

[download]   0.4% of 231.51KiB at  6.10KiB/s ETA 00:30
[download]   1.1% of 231.51KiB at 27.07KiB/s ETA 00:10
[download]   4.0% of 231.51KiB at 19.24KiB/s ETA 00:04
[download]   6.5% of 231.51KiB at 75.06KiB/s ETA 00:03
[download]  13.4% of 231.51KiB at 98.22KiB/s ETA 00:03
[download]  28.7% of 231.51KiB at 81.40KiB/s ETA 00:02
[download]  61.7% of 231.51KiB at 91.56KiB/s ETA 00:01
[download]  86.2% of 231.51KiB at 82.96KiB/s ETA 00:00
[download] 100.0% of 231.51KiB at 73.21KiB/s ETA 00:00
[download] 100% of 231.51KiB in 00:02
[ffmpeg] Correcting container in "_1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a"
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
[avconv] Destination: _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.mp3
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
Deleting original file _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a (pass -k to keep)
[download] Downloading video 2 of 4
[youtube] wTvXkMpJflk: Downloading webpage
[youtube] wTvXkMpJflk: Extracting video information
[youtube] wTvXkMpJflk: Downloading DASH manifest

I want to delete all lines starting with [youtube], [ffmpeg] and [avconv] and get like this

[youtube:playlist] PL1C815DB73EC2678E: Downloading webpage
[download] Downloading playlist: Less than 1 minute
[youtube:playlist] playlist Less than 1 minute: Collected 4 video ids (downloading 4 of them)
[download] Downloading video 1 of 4
[download] Destination: _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a

[download]   0.4% of 231.51KiB at  6.10KiB/s ETA 00:30
[download]   1.1% of 231.51KiB at 27.07KiB/s ETA 00:10
[download]   4.0% of 231.51KiB at 19.24KiB/s ETA 00:04
[download]   6.5% of 231.51KiB at 75.06KiB/s ETA 00:03
[download]  13.4% of 231.51KiB at 98.22KiB/s ETA 00:03
[download]  28.7% of 231.51KiB at 81.40KiB/s ETA 00:02
[download]  61.7% of 231.51KiB at 91.56KiB/s ETA 00:01
[download]  86.2% of 231.51KiB at 82.96KiB/s ETA 00:00
[download] 100.0% of 231.51KiB at 73.21KiB/s ETA 00:00
[download] 100% of 231.51KiB in 00:02
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
WARNING: Your copy of avconv is outdated, update avconv to version 10-0 or newer if you encounter any errors.
Deleting original file _1 min. - Amendes pour les particules du LHC-KNLwsqzFfNg.m4a (pass -k to keep)
[download] Downloading video 2 of 4
etc..
etc..
.
.

i tried this method but its showing error and it only suppose to delete [youtube]

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | sed '^/[youtube]/ d' 
  • Your pattern is wrong. It should be sed '/^\[youtube\]/d'. – Pilot6 Jun 14 '15 at 13:53
9
sed -i '/\[youtube\]/d' /path/to/file

will delete lines, containing "[youtube]".

As one command you can combine patterns like

sed -i '/\[youtube\]\|\[ffmpeg\]\|\[avconv\]/d' /path/to/file

Or right from your command

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 \
 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | 
    sed '/\[youtube\]\|\[ffmpeg\]\|\[avconv\]/d' > output.txt

This will write the result to a file output.txt.

If you want to delete lines not just containing [youtube], but starting with [youtube], then add ^ to the pattern, like sed '/^\[youtube\]/d'.

But in your case it does not matter.

  • I tried this sed -i '/\[youtube\]\|\[ffmpeg\]\|\[avconv\]/d' youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E and not working. may be because its not a file and its a download link – potholiday Jun 14 '15 at 13:41
  • Try with a file. It will work. Or you can do it right from the link using | pipe. – Pilot6 Jun 14 '15 at 13:42
  • @potholiday I updated the answer – Pilot6 Jun 14 '15 at 13:55
  • worked... the edited one worked – potholiday Jun 14 '15 at 13:58
  • and for me > output.txt is not necessary – potholiday Jun 14 '15 at 14:04
4

I suggest using grep -vE like so:

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | grep -vE '^\[(youtube|ffmpeg|avconv)\]'

From man grep:

-v, --invert-match
              Invert the sense of matching, to select non-matching lines.  (-v
              is specified by POSIX.)



-E, --extended-regexp
              Interpret  PATTERN  as  an extended regular expression (ERE, see
              below).  (-E is specified by POSIX.)

The -E flag is used to avoid escaping square brackets with slashes. Without -E flag you have to escape the square brackets with a backslash, like so grep -vE '\[youtube\]\|\[ffmpeg\]\|\[avconv\]' Edit:

Since you've requested awk,here's one with awk:

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | awk '{if ($0~/^\[youtube\]/||/^\[ffmpeg\]/||/^\[avconv\]/||/^WARNING/) next;print}'

  • The [ and ] should be escaped. With the -E option (extended regex option) you can avoid escaping (, ) and | in the command: grep -vE '\[(youtube|ffmpeg|avconv)\]'. As for the "starting requirement", ^ needs to be prepended. – Lekensteyn Jun 14 '15 at 13:40
  • @Lekensteyn good point. I'll add that – Sergiy Kolodyazhnyy Jun 14 '15 at 13:40
  • @Serg not working. when executed it shows > and a blinking cursor. may be some syntax problem – potholiday Jun 14 '15 at 13:46
  • @potholiday it was missing the single quote at the end. Fixed everything and added awk version. please review – Sergiy Kolodyazhnyy Jun 14 '15 at 13:47
  • grep script doesn’t show any difference from the original code, no change at all. awk i tried like this youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | awk '{if(/[youtube]/||/[ffmpeg]/) $0="";print}' it deletes all except warning lines – potholiday Jun 14 '15 at 13:53
1

Using Perl:

< inputfile perl -pe 's/^\[(youtube|ffmpeg|avconv)\].*$//' > outputfile

To parse the output of youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E directly, pipe its output to the command without redirecting the content of inputfile:

`youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | perl -pe 's/^\[(youtube|ffmpeg|avconv)\].*$//' > outputfile` 
  • < inputfile: redirects the content of inputfile to perl's stdin
  • > outputfile: redirects the content of perl's stdout to outputfile
  • -p: places a while (<>) { [...] } loop around the script and prints each processed line
  • -e: reads the script from the arguments

Perl script breakdown:

  • s: asserts to perform a substitution
  • /: starts the pattern
  • ^: matches the start of the line
  • \[: matches a [ character
  • (: starts grouping the allowed strings
  • youtube: matches a youtube string
  • |: separates the second allowed string
  • ffmpeg: matches a ffmpeg string
  • |: separates the third allowed string
  • avconv: matches a avconv string
  • ): stops grouping the allowed strings
  • \]: matches a ] character
  • .*: matches any number of any character
  • $: matches the end of the line
  • /: stops the pattern / starts the replacement string
  • /: stops the replacement string
  • 1
    you might want to edit the command to include pipe from the OP's youtube-dl command. Others might not understand. Otherwise, I've tested it with an input file with text from OP's post,seems to work, hence +1 – Sergiy Kolodyazhnyy Jun 14 '15 at 14:37
  • Thanks @Serg, I saw you updated yours as well so +1 on yours either – kos Jun 14 '15 at 14:46
1

Use grep -v as following:

youtube-dl --extract-audio --audio-quality 0 --newline --audio-format mp3 https://www.youtube.com/playlist?list=PL1C815DB73EC2678E | grep -v '^[youtube]' | grep -v '^[ffmpeg]' | grep -v '^[avconv]'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.