How to cut some text from several files, replace them with names and some more?

Question

This may be a big favor as I want you to help me write a complete script, but I appreciate anyone helping me in it and assure you that your answers help science improve :)

If you don't have the time, please just name the commands that I need to learn their manuals (but I still will need to know how to relate each command to the other)

I am working on some proteins, and each of them has a unique name (like 2DMS). These proteins have a text file describing their properties. I need to cut through this text for each protein, cut from the first line starting with "ATOM" to the end of the line starting from "TER", and type a " ' " in the end. Then rename the output file to the protein ID (2DMS). Here you can see a sample: (Normally the file name is just the protein ID, in this case= 2DMS.)

http://www.rcsb.org/pdb/download/downloadFile.do?fileFormat=pdb&compression=NO&structureId=2DMS

you can open it as a text file.

Then the script needs to modify the following and rename it to CURL_2DMS (2DMS is replaced with each protein ID):

curl LINK/idock/jobs -d '&description=2DMS&email=myemailaddress@gmail.com&center_x=150&center_y=109&center_z=22&size_x=18&size_y=17&size_z=15&mwt_lb=390&mwt_ub=420&lgp_lb=1&lgp_ub=3&nrb_lb=4&nrb_ub=6&hbd_lb=2&hbd_ub=4&hba_lb=4&hba_ub=6&chg_lb=0&chg_ub=0&ads_lb=0&ads_ub=10&pds_lb=-40&pds_ub=0&psa_lb=60&psa_ub=80&receptor=ATOM      1  N   PRO A  19     148.930 114.148   5.178  1.00138.31           N  
ATOM      2  CA  PRO A  19     149.869 115.263   5.360  1.00138.99           C  
ATOM      3  C   PRO A  19     151.021 115.224   4.348  1.00144.31           C  
ATOM      4  O   PRO A  19     150.914 114.542   3.320  1.00144.78           O  
ATOM      5  CB  PRO A  19     150.357 115.076   6.800  1.00138.04           C  
ATOM      6  CG  PRO A  19     149.222 114.376   7.496  1.00140.40           C  
ATOM      7  CD  PRO A  19     148.354 113.704   6.460  1.00137.06           C  
TER    2743      GLN A 313                                                      
'

As you can see, the curl has these 5 strings which need to be modified for each protein:

description= “protein ID”
center_x= “column 1”
center_y= “column 2”
center_z= “column 3”
receptor= “the protein_ID.txt which was produced in the first step”

The columns 1,2,3 exist in a file named coordinates.csv

Then I need the script to run each and every CURL_proten_ID file ( Let’s guess each job in the website takes 3 hours to be completed, I need the script to run each curl in a 3-hours period). Then the script should go to [this link][1] and search for the protein_ID (like 2DMS) and copy the string of _id (in this example: 567048e420d3a8813691c534). Then it should download a CSV file with this link with wget (or alternatives) and replace the _id:

websitelink/567048e420d3a8813691c534 (_id) /log.csv.gz

example:

{"0":244,"1":1656,"2":1072,"3":2039,"4":2642,"5":2750,"6":83,"7":286,"8":371,"9":487,"_id":"567048e420d3a8813691c534","description":"2DMS","ligands":11630,"scheduled":10,"submitted":"2015-12-15T17:07:48.729Z","done":"2015-12-15T17:20:53.143Z"},

Then I need the script to rename the downloaded file to 2DMS( ID)_log.csv.gz

Then I am grateful for your time which you put in this project :)

**Here is the algorithm for the job:

1. Open the protein file (PDB)

2. Delete all the lines upper than the first line containing “ATOM”

3. Delete all the lines lower than the last line containting “TER”

4. Rename the file to PDB.txt

5. Make a new CURL.txt file

6. Rename it to PDB_curl.txt

7. Copy the following into PDB_curl.txt :

   curl LINK/idock/jobs -d '&description=PDB&email=myemailaddress@gmail.com&center_x=xcenter&center_y=ycenter&center_z=zcenter&size_x=18&size_y=17&size_z=15&mwt_lb=390&mwt_ub=420&lgp_lb=1&lgp_ub=3&nrb_lb=4&nrb_ub=6&hbd_lb=2&hbd_ub=4&hba_lb=4&hba_ub=6&chg_lb=0&chg_ub=0&ads_lb=0&ads_ub=10&pds_lb=-40&pds_ub=0&psa_lb=60&psa_ub=80&receptor=RECEPTOR_DISCRIPTION'
   

8. Change the “PDB” to the protein ID (PDB)

9. Copy PDB.txt and replace the “RECEPTOR_DISCRIPTION”

10. Open coordinates.csv

11. for each PDB, copy coulmn1 and replace “xcenter” with it

12. for each PDB, copy coulmn2 and replace “ycenter” with it

13. for each PDB, copy coulmn3 and replace “zcenter” with it

14. close and save PDB_CURL.txt

15. Run PDB_CURL

16. Wait for 3 hours

17. Search through this link: {link/idock/jobs} for PDB

18. Copy the next string after “_id” in that line

19. Replace the “_id” in this link with the copied text : link/idock/jobs/_id /log.csv.gz

20. Download the link via wget

21. Rename the downloaded file to PDB_log.csv.gz

22. Repeat the upper commands until every PDB_log.csv.gz is downloaded

23. If there is an error, skip that job completely and copy that PDB to “errors.log”**

Answer 1

#!/bin/bash
tail -n $(($(wc -l pdb2dms.ent|cut -d' ' -f 1)-1)) pdb2dms.ent|head -n $(($(wc -l pdb2dms.ent|cut -d' ' -f 1)-1)) pdb2dms.ent | grep -v "ATOM" | grep -v "TER" #2 and 3
mv pdb2dms.ent PDB.txt # 4
touch curl.txt #5
mv curl.txt PDB_curl.txt #6

for i in coordinates.csv
do
  curl LINK/idock/jobs -d "&description=PDB&email=myemailaddress@gmail.com&center_x=`cat $i|cut -d',' -f 1`&center_y=`cat $i|cut -d',' -f 2`&center_z=`cat $i|cut -d',' -f 3`&size_x=18&size_y=17&size_z=15&mwt_lb=390&mwt_ub=420&lgp_lb=1&lgp_ub=3&nrb_lb=4&nrb_ub=6&hbd_lb=2&hbd_ub=4&hba_lb=4&hba_ub=6&chg_lb=0&chg_ub=0&ads_lb=0&ads_ub=10&pds_lb=-40&pds_ub=0&psa_lb=60&psa_ub=80&receptor=`cat ./*_curl.txt`" > PDB_curl.txt
done #11,12,13,15
sleep 3h #16
wget -O PDB_log.csv.gz $(wget -O - link/idock/jobs|grep 'PDB'|sed \_id\`wget -O - link/idock/jobs/_id /log.csv.gz|echo`\) #17,18,19,20,21

Not sure about #22 and #23
Please check that the commands do what they are supposed to since I had some trouble understanding the objective. man <command> can be used, or a google search.