3

I have nearly 4 lakhs (400,000) of files in a folder named model and from this folder I have selected a few thousand files and their names are saved in id.txt. Now I want to take only the files listed in id.txt from the folder model and copy them to another folder named selected_ids.

The two folders are present inside the same directory test which contains

  1. model
  2. selected_ids
  3. id.txt

id.txt contains filenames one by one eg:

ENSP00000290866.4_2.pdb.xz
385719215_2.pdb.xz
GENSCAN00000006392_3.pdb.xz

Model folder contains nearly 3 lakh files

ENSP00000290866.4_2.pdb.xz
385719215_2.pdb.xz
GENSCAN00000006392_3.pdb

*note: in folder some files have .pdb.xz extension and some only .pdb

I know how to copy a single file from one folder to another like this

cp model/ENSP00000290866.4_2.pdb.xz selected_ids/

But I have thousands and thousands of files to copy in short time.

Can anyone give some efficient command to copy the files?

4

the command you are looking is as follow,

cd /path/to/model
$ xargs -d '\n' -a /path/to/id.txt cp -t /path/to/selected_ids

this will read the files name saved in id.txt and copy to selected_ids directory.

| improve this answer | |
  • 1
    @Nitha please stop having these long discussions in the comments. It is very hard for anyone else to follow. All these details need to be added to your question. We need to know what you tied and how it failed and, most importantly, we need to know what the actual file names are. So please edit your question and show us an actual example with a few ids and a few files. Including pdb and pdb.xz extensions. Finally, it sounds like you might be interested in our new site: Bioinformatics. – terdon Jul 21 '17 at 11:43
0

Use while IFS= read -r variable; do... done < input.txt structure to read file line by line and execute desired code in ... part. This is a standard bash way to read file line-by-line and act upon it. With the directory as you described, it should be done as so:

$ tree                                                                                                                                               
.
├── 385719215_2.pdb.xz
├── ENSP00000290866.4_2.pdb.xz
├── GENSCAN00000006392_3.pdb.xz
├── id.txt
├── model
└── selected_ids

2 directories, 4 files
$ while IFS= read -r file; do cp "$file" selected_ids/"$file" ;done < id.txt                                                                         
$ tree
.
├── 385719215_2.pdb.xz
├── ENSP00000290866.4_2.pdb.xz
├── GENSCAN00000006392_3.pdb.xz
├── id.txt
├── model
└── selected_ids
    ├── 385719215_2.pdb.xz
    ├── ENSP00000290866.4_2.pdb.xz
    └── GENSCAN00000006392_3.pdb.xz
| improve this answer | |
  • Ubuntu 17.04: The program 'tree' is currently not installed. You can install it by typing: sudo apt install tree – madneon Jul 28 '17 at 21:26
  • @madneon Yeah, it's not a default program, but tree isn't important as far as the question goes, it's used here only for giving visual idea of how directory is structured. The important part is the while loop – Sergiy Kolodyazhnyy Jul 28 '17 at 21:36
0

Assuming there is no white-spaces in input file you can simply use for loop:

for f in $(<id.txt); do cp -v "model/$f" selected_ids/;done

If you having trouble with "\r" at the end of file names (most likely file was saved in Windows-family OS), you also need to trim variable $f:

for f in $(<id.txt); do f=`echo $f|tr -d "\n\r"`; cp -v "model/$f" selected_ids/; done

Note that cp with option -v will display the current file being copied, so its easy to verify this batch copying process.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.