-1

I have searched and seen many question similar to mine but none of the answer satisfy me.

My scenario is: i am writing a script which will going to run on the server. i have a curl command which return a json body and i want to pick all the values with specific key and i can not use any external tool for parsing json like jq and i dont even have python there just simple bash shell.

so my question is how i do it with using just default shell?

there are going to multiple values for that same key and i want to make the array of these value.

an example json:-

{
     "people":[
                 {
                     "id":"4568734",
                     "name":"suneel"
                 },
                 {
                     "id":"3678976",
                     "name":"adi"
                  }
             ]
   }

if say 'name' then i want array("suneel" "adi")

Note: No use of external tool and with just bash commands.

4
  • I guess this is not Ubuntu, as you would have python. Also, grep is an external tool... as well as awk or sed. Please clarify.
    – pLumo
    Aug 21 '19 at 11:09
  • @pLumo I just learned about jq and installed it an hour ago. (Before reading this question). To say jq it can't be used makes the question above sound like a homework project. Python is not installed on all systems but GNU utilities and Bash usually are. If this is a homework project I think that's how the prof made the rules of engagement. Aug 21 '19 at 11:51
  • @pLumo you can assume any machine with awk , sed and grep and do not have python..and cannot install any other package..this is the condition for my working. i already tried to talk with bosses for jq.but no change in the condition. Aug 21 '19 at 14:15
  • i already know about 'jq' its a great tool and it took me hardly 10 min to make this work with jq.....but the condition is no jq... Aug 21 '19 at 14:16
2

A bash function like below can be used:

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

I've saved this function as jsonVal and then sourced this file using source jsonVal. You can very well use it within your script.

It expects two arguments. First argument is the property name. If you need all values, skip second argument. If specific value is needed, you can add the second argument as shown below.

[root@localhost Desktop]# cat data.json | jsonValue id
4568734
3678976
[root@localhost Desktop]# cat data.json | jsonValue id 1
4568734
[root@localhost Desktop]# cat data.json | jsonValue id 2
3678976
[root@localhost Desktop]# cat data.json | jsonValue name
suneel
adi
[root@localhost Desktop]# cat data.json | jsonValue name 1
suneel
[root@localhost Desktop]# cat data.json | jsonValue name 2
adi
[root@localhost Desktop]#

Hope this helps.

1
  • 1
    If a value contains : , } this seems to break :(
    – Phill
    May 18 '20 at 4:56
1

Using tools other than proper json parsers will always be prone to errors or security issues.

Your best option: If you don't have the tools you need to do your work, ask your IT/Server admin to install them.


Anyways, the following will work at least for your example:

Using grep -P:

$ curl ... | grep -Po '"name":"\K[^"]*'
suneel
adi

With normal grep:

$ curl ... | grep -o '"name":"[^"]*' | cut -d'"' -f4
suneel
adi

If you have "name" somewhere outside of "people" which you do not want, this will obviously fail.

0

This is a little overkill but I already have an app that reads gmail meta data in order to Trash daily backup files compressed and attached to gmail messages. The file format is similar to yours:

SAMPLE RECORD (REFORMATTED):
============================

{u'internalDate': u'1541947153000', 
u'historyId': u'1517343', 
u'payload': 
    {u'mimeType': u'multipart/mixed',
     u'headers': [
        {u'name': u'Return-Path', u'value': u'<me@gmail.com>'}, 
        {u'name': u'Received', u'value': u'from alien (node-ISP. [IPv6.Address])        by smtp.gmail.com with ESMTPSA id x184-v6sm1211487pfx.42.2018.11.11.06.39.15        for <me@gmail.com>        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);        Sun, 11 Nov 2018 06:39:35 -0800 (PST)'}, 
        {u'name': u'Message-ID', u'value': u'<5be83f27.1c69fb81.4f2bc.2906@mx.google.com>'}, 
        {u'name': u'From', u'value': u'root <me@gmail.com>'}, 
        {u'name': u'X-Google-Original-From', u'value': u'"root" <root@gmail.com>'}, 
        {u'name': u'Received', u'value': u'by alien (sSMTP sendmail emulation); Sun, 11 Nov 2018 07:39:13 -0700'}, 
        {u'name': u'Date', u'value': u'Sun, 11 Nov 2018 07:39:13 -0700'}, 
        {u'name': u'to', u'value': u'me@gmail.com'}, 
        {u'name': u'Subject', u'value': u'Daily-alien-Ubuntu-16.04-Backup-2018-11-11-Sunday.tar.gz.64'}
        ]
    }, 
u'snippet': u'', 
u'sizeEstimate': 18340047, 
u'threadId': u'1670336bc9ac099d', 
u'labelIds': [u'IMPORTANT', u'SENT', u'Label_12'], 
u'id': u'1670336bc9ac099d'
}

Here is the code snippet from the large bash script:

GrepLine () {

#https://askubuntu.com/questions/952467/extracting-a-specific-string-after-a-given-string-from-html-file-using-a-bash-sc

    End1="'"
    Srch2="{u'name': u'Date', u'value': u'"
    Srch3="{u'name': u'Subject', u'value': u'" # sometimes lower-case "subject"
    End3="'}"
    Srch4="u'sizeEstimate': "
    End4=","
    Srch5="u'labelIds': \[u'"
    End5="]"
    Srch6="u'id': u'" 

    # grep allows fast search (compiled C) and supports case insensitivity
    HeaderDate=$(grep -oiPm1 "$Srch2\K[^$End1]+" <<< "$plLine")
    HeaderSubject=$(grep -oiPm1 "$Srch3\K[^$End3]+" <<< "$plLine")
    [[ $HeaderSubject == "" ]] && HeaderSubject="GREP ERROR: Subject blank"

    Size=$(grep -oiPm1 "$Srch4\K[^$End4]+" <<< "$plLine")

    LabelIds=$(grep -oiPm1 "$Srch5\K[^$End5]+" <<< "$plLine")
    LabelIds="${LabelIds//\', u\'/$LabelSep}"
    TrimLen="${#LabelIds}"
    let TrimLen--
    LabelIds="${LabelIds:0:TrimLen}" # Remove trailing '
    # Convert "Label_12" to "Backup", etc.
    ReplaceLabels

    MessageId=$(grep -oiPm1 "$Srch6\K[^$End1]+" <<< "$plLine")

    # Some email messages have 50 extra spaces or more.
    HeaderSubject="${HeaderSubject%% }"

    # Convert HTML to ASCII, eg: Can&#39;t wear the same one
    # Replace external command: Line=$(sed 's/&amp;/\&/g; s/&lt;/\</g; 
    # s/&gt;/\>/g; s/&quot;/\"/g; s/&#39;/\'"'"'/g; s/&ldquo;/\"/g; 
    # s/&rdquo;/\"/g;' <<< "$Line") -- With faster builtin commands.
    HeaderSubject="${HeaderSubject//&nbsp;/ }"
    HeaderSubject="${HeaderSubject//&amp;/&}"
    HeaderSubject="${HeaderSubject//&lt;/<}"
    HeaderSubject="${HeaderSubject//&gt;/>}"
    HeaderSubject="${HeaderSubject//&quot;/'"'}"
    HeaderSubject="${HeaderSubject//&#39;/"'"}"
    HeaderSubject="${HeaderSubject//&ldquo;/'"'}" # TODO: ASCII/ISO for opening quote
    HeaderSubject="${HeaderSubject//&rdquo;/'"'}" # TODO: ASCII/ISO for closing quote

    # Truncate subject to 80 characters or whatever variable is set to
    HeaderSubject="${HeaderSubject:0:$SubjectTruncate}"
    HeaderSubject="${HeaderSubject//|/?}"       # Strip out Yad array separators
    HeaderSubject="${HeaderSubject//\--/==}"    # Yad doesn't like --
#    HeaderSubject="${HeaderSubject//\'/\"}"   # Yad doesn't like
    [[ $HeaderSubject == "" ]] && HeaderSubject="PARSING ERROR: Subject blank"

    # Convert date to YAD format
    HeaderDate=$(date -d "$HeaderDate" +'%Y-%m-%d')

} # GrepLine

Not the answer you're looking for? Browse other questions tagged or ask your own question.