7

I was assigned a database project where the dataset I downloaded, was 50GB. After extracting the dataset I have 31 folders. Each of this 31 folders contain 23 folders. And each of this 23 folders contain 59 (00-59).json data. I need to feed this data to MongoDB. I was writing this command for it:

 mongoimport --db twitter --collection twitterCol --file /media/shamsad/1E8A00A88A007E91/archiveteam-twitter-stream-2013-08/08/01/00/00.json

But it is tiresome for writing this command for 31*23*59 .json data.
How can I write a script to go through directories, locate jsons and call mongoimport?

11

No script needed, just find:

find /media/shamsad/1E8A00A88A007E91/ -name '*.json' \
    -exec  mongoimport --db twitter --collection twitterCol --file {} \;
12
  • could you kindly explain what does "name '*.json data' \" this part do?
    – sphoenix
    Apr 27 '17 at 15:39
  • @sphoenix -name '*.json data' : all files with json data extension. `\` : is used to write a command in two line; so bash will know the following line is a part of same command and not another command;
    – Ravexina
    Apr 27 '17 at 15:45
  • @sphoenix what ravexina said. Let me know if it's still unclear.
    – terdon
    Apr 27 '17 at 15:47
  • I wrote your command in a db_script.sh and ran it. It says "find: paths must precede expression: Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec|time] [path...] [expression] ./db_script.sh: line 2: -exec: command not found "
    – sphoenix
    Apr 27 '17 at 15:54
  • @terdon You had *.json data where it should just be *.json. I fixed it.
    – wjandrea
    Apr 27 '17 at 15:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.