Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have a set of data files (*.all) located in different folders

Documents/rawdata/Geob1005/1.all, 2.all, 3.all,...,n.all
Documents/rawdata/Geob1006/A.all, B.all, C.all,...,Z.all          and so on...

I have created a file called datalist.mb-1 in each directory which lists all the *.all files located in the same directory using:

/bin/ls -1 *.all | awk '{print $1" 58"}' >datalist.mb-1

so I get this:

Documents/rawdata/Geob1005/datalist.mb-1
Documents/rawdata/Geob1006/datalist.mb-1
and so on...

Now I need to create a "master" datalist in Documents/rawdata/ that points out to all the other datalists inside the directories of rawdata/

I need the outcome of the "master" datalist to look like this:

Geob1005/datalist.mb-1  -­1 
Geob1006/datalist.mb-1  -1  

Any idea of how to do this task?

Thanks in advance.

share|improve this question
    
Nice skills. Does it have to be in bash, or will python do? Oh, can you put your file paths and code in coding formatting (use these ``)? –  don.joey Aug 8 '13 at 18:16
    
what is the -1 at the end of Geob1005/datalist.mb-1 -­1 for, or the 58? –  Symin Aug 8 '13 at 18:27
    
@Private The 58 is because I need to add a column to the datalist.mb-1 files which describes the format of the data (which in this case is 58) –  napogeof Aug 9 '13 at 11:06
    
@Symin The -1 at the end of 'Geob1005/datalist.mb-1 -1' is to also describe the format of the datalist located in that path. I am trying to follow a HowTo MB_System tutorial for bathymetry data processing. ldeo.columbia.edu/res/pi/MB-System –  napogeof Aug 9 '13 at 11:13

3 Answers 3

Untested

You could use bash to make a list of all the files in a single .txt file (just use a find command to find the 'datalist' files and then pipe that the file).

In python you can use that file list (for example, datalist_master.txt) as follows:

fi = file.open('datalist_master.txt', 'r')
li = fi.readlines()
new_li = []    

for element in li:
    # get rid of unnecessary path info
    element = element.lstrip('Documents/rawdata/')

    # add last two digits 
    element = element + ' ' + element[-2:]
    new_li.append(element)

fi.close()
print new_li #or save it somewhere

This is most certainly also possible in bash (but I do not know how).

share|improve this answer

A really simple way to do this would be to use the locate command like so:

locate Documents/rawdata/Geob*/datalist.mb-1 > master.txt

That is if full paths are okay.

However, if you need the format which you described above with only the "geo" subdirectory and file you could use a simple little python script like this one:

#!/bin/python
f = open("/path/to/master.txt","r+")
paths = [x.split("/",3)[-1] + " -1" for x in f.readlines()]
f.seek(0)
f.write("".join(paths))
f.truncate()
f.close()

Slightly truncated version of napogeof's answer:

locate ~/Documents/rawdata/*/datalist.mb-1 | awk -F/ '{print $4 "/" $5 " -1"}' > master.txt

share|improve this answer
    
I finally managed to do it using Locate, actually I didn't need the full path but using awk was enough to just show the path from the "geo" subdirectory. this is how I did it: Guys I finally managed to do the task using the Locate command locate ~/Documents/rawdata/*/datalist.mb-1 | awk 'BEGIN {FS="/"} {print $4 "/" $5 " -1"}' > master.txt –  napogeof Aug 9 '13 at 18:20
1  
You should post that as an answer. I'm sure plenty of people would find it useful. –  Bryan Aug 9 '13 at 18:53
up vote 2 down vote accepted

I finally managed to do it using Locate, actually I didn't need the full path but using awk was enough to just show just the path from the "geo" subdirectory.

This is how I did it:

locate ~/Documents/rawdata/*/datalist.mb-1 | awk 'BEGIN {FS="/"} {print $4 "/" $5 " -1"}' > master.txt

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.