I have lots of EDI files (which is pretty much a single line of gibberish text) that are about 300mb/pcs.

I'm looking to pattern matching this:

`*UP*605284466346 {lots of other stuff inbetween with set length} QTY*17`

What I know is that UP is at the head of every section followed by the sku at the tail end is QTY with the quantity #.

What I'm trying to find is SKU: 605284466346 with a QTY: 17

This was my valiant attempt:

find * | grep -lir 'UP*605284466346.*QTY\*97' * | grep -v svn


awk is much better for that:


# Usage findsku <SKU> <QTY> <DIR>
# Example: findsku 605284307892 14 ~/mydir
# Result: found ./teste.edi: SKU=605284307892, QTY=14


echo "looking for SKU #${sku} with QTY=${qty} in ${dir}:"

while read -r file; do
    awk -F'*' -v sku="$sku" -v qty="$qty" 'BEGIN{RS="*UP*"} $2==sku &&
    $NF==qty {print "found " FILENAME ": SKU=" sku ", QTY=" qty}' "$file"
done < <(find "$dir" -name "*.edi" -type f)

I think it is pretty much self-explanatory, and you can adapt for any sku and qty values you want. But feel free to ask any question you want about it!

It will recurse in the given DIR, and you can easily adapt it to scan other files besides *.edi

  • If this works for ehime, I'll upvote this when I've got more votes. – belacqua May 4 '12 at 18:08
  • It didn't work, I'm not sure if its looking through files or not? Above was a test case so 605284466346 should have 17 but I have zero returns from searching using *.edi? – ehime May 4 '12 at 18:09
  • ok, but like i said, its a single 300mb line i'll try and cat out a chunk though – ehime May 4 '12 at 18:20
  • pastebin.com/HY4mc4Dq @MestreLion no there are multiple *UP* segments, but the entire file is a continuous line, look at the pastebin and you should get a better idea – ehime May 4 '12 at 18:26
  • let us continue this discussion in chat – ehime May 4 '12 at 18:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.