5

I have a file of 1 GB and would like to convert it into a certain format. The file contains the following URLs:

google.com
trafficinviter.com
example.com

I would like to convert the complete content in this format:

"http://google.com","http://trafficinviter.com","http://example.com"

Using Ubuntu. This is exactly the format of PHP array. So I would like to know how I can convert to this format and save to another file.

  • 1
    @Zanna Hello, the do is the number urls one after another... actually the file is huge. hence I have just mentioned a few sample for asking. – Jaffer Wilson Jan 4 '17 at 11:06
  • 1
    @Zanna I have edited the question and hope that it will help to understand. – Jaffer Wilson Jan 4 '17 at 11:07
9

I'd do this in Perl:

$ perl -lne 'printf "\"http://$_\","' file | sed 's/,$/\n/'
"http://google.com","http://trafficinviter.com","http://example.com"

The -l removes newline characters from the end of each line (also adds a \n to each print call which is why I'm using printf instead). The -ne means "run the script given by -e on each line of the input file. The script itself will print "http://, the current line ($_), and ",. Then, the sed removes the final trailing comma.


Alternatively, you could do:

$ sed 's#^#"http://#;s#$#"#' file | tr "\n" "," | sed 's/,$/\n/'
"http://google.com","http://trafficinviter.com","http://example.com"

Here, the 1st sed will replace the beginning of each line (^) with "http://, and the end of each line ($) with a "; the tr replaces each newline character with a comma and the 2nd sed removes the last trailing comma.

| improve this answer | |
  • I want to write the output another file. But I don't know why I am not able to. The output file is 0 byte – Jaffer Wilson Jan 4 '17 at 11:32
  • @JafferWilson OK, but how are you doing it? Actually, it might be simpler if you just come into chat a second and we can see what's wrong. – terdon Jan 4 '17 at 11:33
  • it is simple. just using your solution and putting > and output file name. I thing this should give answer. – Jaffer Wilson Jan 4 '17 at 11:36
  • @JafferWilson yes, it should, that's why I asked you to come to chat. There's nothing special here that would stop the file from being created. So come into chat where we can discuss it and you can show me the exact commands you used. – terdon Jan 4 '17 at 11:37
  • The first sed command could be simpler with a record: sed 'sx\(.*\)x"http://\1"x' test.txt | tr '\n' ',' | sed 's/,$//' – rexkogitans Jan 4 '17 at 20:33
7

You could do it like this:

tr '\n' ',' < file | sed 's#[^,]\+#"http://&"#g;s#,$#\n#'

Explanation

  • tr '\n' ',' <file replace newlines with commas in file
  • | pipe output to
  • s#[^,]\+#"http://&"#g replace a series of characters excluding comma with the same characters prepended by http:// and surrounded by "quotes"
  • s#,$#\n# remove the last comma, replace with a newline

Note: this will fail if the urls have commas, so if that's possible in your case, use terdon's answer for reliable results

| improve this answer | |
  • Swapping tr and sed would make the command easier to read and also immune to URLs containing commas: sed 's#.*#"http://&"#' file | tr '\n' ',' – Byte Commander Jan 4 '17 at 17:30
  • @ByteCommander I started out that way but I didn't want to pipe again to remove the last comma. Also, that solution is the same as terdon's, so I'm stuck with my post as it is I think :) – Zanna Jan 4 '17 at 17:55
  • Oh right, the evil trailing comma... – Byte Commander Jan 4 '17 at 18:02
5

Awk solution

awk '{ printf("\"%s\",", $0) }' input.txt

PHP allows terminating commas in array literals, if I recall correctly.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.