I have used this command to successfully convert a .txt file to a .pcap file. However these were all for .txt files I obtained from the www,but when i attempted to use it for .txt files I had personally created on another program, it did create filename.pcap, but it is empty.

Does anyone have a better alternative?

Also in as much as I appreciate all the assistance here in code that achieves the result, if anyone can refer me to the wikipedia articles regarding the subject that is relevant as to why this occurs, thats really going to help me contextually understand how things work, where as just the code as helpful as I feel as it this is, will leave me none the wiser as far as how linux works. So I'm sure this will be considered a duplicate question, but I actually would really like to understand specifically this scenario, rather than be referred to another apt package.

Many thanks

sed 's/^[0-9:]*//' filename.txt | sed 's/^  //g' | sed 's/    .*$//g' | xxd -r -p > filename.pcap

Updated Edit:

The only consistent difference between the collection of txt files for which the above worked, and those I created, was that mine exclusively consist of numbers, one per line, 10 digits in length, where as the others contain everything else on the keyboard.

Don't know if that's any help, and yeah also I took a look at a pcap file in the text editor simply by changing it's extension to txt, and it appears they are encrypted with a cipher that utilizes many more characters beyond what is on the standard keyboard, so... yes it's pretty standard, not sure what I was expecting there.

  • IMHO it's going to be hard to answer this satisfactorily without examples of files that it does work for and files that it doesn't - basically the sed command deletes (well, substitutes with nothing) various text strings: whether that results in something that will convert to a valid PCAP file rather depends on the original contents of filename.txt – steeldriver Feb 14 '19 at 16:07
  • ... also FYI the g modifiers make little sense when applied to anchored expressions such as these (a pattern can only have one start and one end) – steeldriver Feb 14 '19 at 16:10
  • @steeldriver thanks well most of the ones I obtained from the www were considerably larger than mine, and opening them in the text editor was just too much for my little laptop! But I have found a few that are able to be opened now, and looking at the first of them., the first thing that is different to mine is a "header" that is as follows: !@#$% !@#$%^ !@#$%^& !@#$%^&* * – Adam Feb 14 '19 at 16:20
  • fyi, white space there is meant to represent a new line, but it won't let me do that on SE comments or at least the regular way isn't working – Adam Feb 14 '19 at 16:24
  • 1
    You can edit your question to add (properly formatted) text - comments aren't a good place for that, as you've discovered – steeldriver Feb 14 '19 at 16:38

You mention that

The only consistent difference between the collection of txt files for which the above worked, and those I created, was that mine exclusively consist of numbers, one per line, 10 digits in length

Assuming by "numbers" you mean sequences of decimal digits, then the first sed expression


meaning match zero or more decimal digits or colon characters anchored to the start of the line, and replace them with nothing (which is presumably intended to remove the default byte offset from a regular xxd output) will remove everything, leaving only a sequence of empty lines.

  • ok so is there a "control character" of sorts that I could use to de-anchor each digit sequence from it's respective line? – Adam Feb 14 '19 at 18:52
  • also if this is the case I would have thought the method Charles proposed would eliminate the issue since we are not piping anything from sed to xxd input – Adam Feb 14 '19 at 18:56
  • @Adam until you tell us what is in your filename.txt file and how it relates to pcap, this is an XY problem. FYI piping isn't the issue: you will get the same result whether you pipe the results between programs or save them in intermediate files. – steeldriver Feb 14 '19 at 19:12
  • ok true yep I misread the part about default byte offset my bad dyslexia – Adam Feb 14 '19 at 19:32
  • But I did tell you what is in my text file, numbers, one per line, each having exactly 10 digits – Adam Feb 14 '19 at 19:33

I don't know sed, or xxd but I would try to follow the 4 commands individually:

  1. sed 's/^[0-9:]*//' filename.txt > step1.txt
  2. sed 's/^ //g' step1.txt > step2.txt
  3. sed 's/ .*$//g' step2.txt > step3.txt
  4. xxd -r -p step3.txt > filename.pcap

Following each step, I'd find a handy sed tutorial, and check to see if the output of the command matches what I think it should be, thus learning a little about sed and xxd in the process.

You may have noticed that I removed the "|" character from the command line you originally used. "|" is a pipe, indicating that the output from the command is to be used as the input for the next command. Instead of piping the data about, the command outputs are stuffed into intermediate files, which are then used as the input to the next command.

  • ok thankyou very much so how would I go about learning what part of my computer handles the piping of data like as with this case, and get an idea of how this process works? Like in looking at the method you have shown me I can see now that it's not a necessity, but I'd like to learn more about it for any future scenarios that may arise where it is – Adam Feb 14 '19 at 16:11
  • 1
    The pipe itself is a really low-level bit of OS stuff - a Bash tutorial is a great place to start learning about Linux, and I believe that it's included in most basic Linux tutorials. – Charles Green Feb 14 '19 at 16:16
  • 1
    And since Ubuntu and Linux are open source, you can always find the source code for the commands that are executed. Some of them are entirely incomprehensible to me, as they are written in languages that I don't understand (yet) – Charles Green Feb 14 '19 at 16:18
  • Sorry @Charles Green the same outcome occurs when I follow the steps you gave, but none the less I appreciate the help, its turning out to be quite an intriguing problem who would have thought file conversion would be interesting! – Adam Feb 14 '19 at 17:01
  • So, knowing the input to the first step, what is the output which is in step1.txt? – Charles Green Feb 14 '19 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.