processing lots of images which have their names stored in a csv file from terminal

Question

How can I copy all the images in the second column of a CSV file and put them to a new folder named images? It has about 300k images so I wonder if there is a fast way to this (like I have 12 processors):

[jalal@goku cleaned_affenet_manually_annotated]$ head -5 modified_training_for_FER2013.csv 
,subDirectory_filePath,expression
0,689/737db2483489148d783ef278f43f486c0a97e140fc4b6b61b84363ca.jpg,3
1,392/c4db2f9b7e4b422d14b6e038f0cdc3ecee239b55326e9181ee4520f9.jpg,6
2,468/21772b68dc8c2a11678c8739eca33adb6ccc658600e4da2224080603.jpg,6
3,944/06e9ae8d3b240eb68fa60534783eacafce2def60a86042f9b7d59544.jpg,3
[jalal@goku cleaned_affenet_manually_annotated]$ wc -l modified_training_for_FER2013.csv 
283903 modified_training_for_FER2013.csv

Answer 1

echo ",subDirectory_filePath,expression
0,689/737db2483489148d783ef278f43f486c0a97e140fc4b6b61b84363ca.jpg,3
1,392/c4db2f9b7e4b422d14b6e038f0cdc3ecee239b55326e9181ee4520f9.jpg,6
2,468/21772b68dc8c2a11678c8739eca33adb6ccc658600e4da2224080603.jpg,6
3,944/06e9ae8d3b240eb68fa60534783eacafce2def60a86042f9b7d59544.jpg,3
" |  sed -r '1d;s/,(.*\.jpg),.*/\1/' | while read f; do echo "copy $f to dir/$f"; done 

• 1d; deletes the first line (headline).
• s/from/to/ substitutes the filename with prefix postfix to a pure filename.
• The while-Loop echos a copy command but could do the copying as well instead.

Maybe the directory names do not exist (689, 392, ...). If you want to flatten the file hierarchy, that is possible to, but might lead to overwritings. cp --parents generates the parent dir(s) if they don't exist, afaik.

Working on the path directly would look like

sed -r '...' modified_training_for_FER2013.csv | while ...

Using 12 processors won't help much, afaik, since the bottleneck should be File-IO, harddrive usage.

Answer 2

This should help out:

sed -r '1d;$d; s/^.*\/(.*\.jpg),./\1/g' mona.csv | xargs -i cp {} /path/to/images

In case the files are some place else but have their names in the csv file then to move them do:

sed -r '1d;$d; s/^.*\/(.*\.jpg),./\1/g' mona.csv | xargs -i cp /path/to/files/{} /path/to/images

Information:

1. 1d;1$: deletes first and last lines