How to select several words, and remove all lines that contain those words?

Question

See screenshot. I want to get rid of the clutter, and leave only things like ID, and TEXT, but remove the other things, such as type, date, date_unixtime, edited, from, etc.

I tried the following command, to try remove just all lines with just one word (I don't know how to combine the needed words in the command) but it produced a 0 byte file (see pic2). Note: it didn't throw any errors.

<1n6Envrionment.json awk '! "date_unixtime"' >1n6Envrionment2.json

Answer 1

A better tool to manipulate json files is jq. jq understands json so it will ensure that its output is a valid json file.

You can install jq by running

sudo apt install -y jq

then

jq 'del(.date_unixtime)' 1n6Envrionment2.json

would delete the date_unixtime field for instance, similarly

jq 'del(.text_entities)' 1n6Envrionment2.json

would delete text_entities and all its children. jq has an extensive matching capability for its filters so you can either run it multiple times or, depending on exactly what you want to remove you might be able to construct a single filter that removes everything.

Answer 2

Grep can do this. To exclude lines with the word date:

grep -v "date" 1n6Envrionment2.json

To exclude more words, separate with an escaped |, like this:

grep -v "date\|date_unixtime\|edited" 1n6Envrionment2.json

You can redirect back to the same file (> 1n6Envrionment2.json) if you want to write the changes immediately.

Answer 3

This solution is similar to @Artur Meinild's

Example Task

Remove all lines including thisname or othername

Example File filename

There is one line
then another
but then there is thisname
and then there is not
then othername
then both thisname and othername

Example Code

grep -vE "thisname|othername" filename

This uses regular expressions and is therefore more powerful as far as I know than the solution given by Meinild.

If you only want to match complete words, use:

grep -vE "[ ]{0,1}thisname[ ]{0,1}|[ ]{0,1}othername[ ]{0,1}" filename

which means that a word is only allowed to be surrounded by nothing or a space.

Example Output

There is one line
then another
and then there is not