Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I do a lot of file manipulation on my system and in one particular batch job I end up with around 16 Gb file. I need to prepare this data into smaller chunks for another process.

I split it into 10k lines per file and numeric index, padded to 5 digits

split -a 5 -d -l 10000 large_input_file /out_path/out.

This way I end up with files named
out.00000
out.00001
...

The problem is that this way indexing always starts with 0.
Is there a way to set it to arbitrary starting index? man reveals nothing ...

share|improve this question
    
Why do you want to start counting the chunk filenames from a non-zero index? Knowing the use case would help to find a solution to your problem. –  loevborg Jan 1 '11 at 16:52
add comment

2 Answers

up vote 1 down vote accepted

It's not a big deal to renumber the files afterwards.

split -a 5 -d -l 10000 large_input_file /out_path/tmp.
for x in /outpath/tmp.*; do
  mv -i -- "$x" "${x%/*}/out.$(printf %05d $((10#${x##*.}+1)))";
done

Note a couple of pitfalls:

  • The naive approach of renaming out.00000 to out.00001 and so on doesn't work, because globbing expands to a lexicographically ordered list, so the first renaming would overwrite the second file. In zsh, you can use the (On) glob qualifier. In other shells, the easiest way is to change the stem of the name as well.
  • A leading 0 in shell arithmetic causes the number to be interpreted in octal. 10#${…} forces a decimal interpretation.
  • printf %05d is a way to maintain the padding to 5 digits.
share|improve this answer
    
nice, and by replacing +1 with +$start_id I get arbitrary first index ... –  nEJC Jan 2 '11 at 15:49
    
btw. what is the meaning of -- in mv command? Can't seam to find it ... probably "no more switches after this point", right? –  nEJC Jan 2 '11 at 15:52
    
@nEJC: Yes, that's exactly it. In this particular case, it's not necessary, because the arguments can't begin with -, but it's a good habit to get into (and it's a reflex with me). –  Gilles Jan 2 '11 at 16:15
add comment

I think this option doesn't exist in split. I've this bug (feauture request) for coreutils -- http://savannah.gnu.org/bugs/?22231

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.