5

I have a file of 1 GB and would like to convert it into a certain format. The file contains the following URLs:

google.com
trafficinviter.com
example.com

I would like to convert the complete content in this format:

"http://google.com","http://trafficinviter.com","http://example.com"

Using Ubuntu. This is exactly the format of PHP array. So I would like to know how I can convert to this format and save to another file.

2
  • 1
    @Zanna Hello, the do is the number urls one after another... actually the file is huge. hence I have just mentioned a few sample for asking. Jan 4, 2017 at 11:06
  • 1
    @Zanna I have edited the question and hope that it will help to understand. Jan 4, 2017 at 11:07

3 Answers 3

9

I'd do this in Perl:

$ perl -lne 'printf "\"http://$_\","' file | sed 's/,$/\n/'
"http://google.com","http://trafficinviter.com","http://example.com"

The -l removes newline characters from the end of each line (also adds a \n to each print call which is why I'm using printf instead). The -ne means "run the script given by -e on each line of the input file. The script itself will print "http://, the current line ($_), and ",. Then, the sed removes the final trailing comma.


Alternatively, you could do:

$ sed 's#^#"http://#;s#$#"#' file | tr "\n" "," | sed 's/,$/\n/'
"http://google.com","http://trafficinviter.com","http://example.com"

Here, the 1st sed will replace the beginning of each line (^) with "http://, and the end of each line ($) with a "; the tr replaces each newline character with a comma and the 2nd sed removes the last trailing comma.

5
  • I want to write the output another file. But I don't know why I am not able to. The output file is 0 byte Jan 4, 2017 at 11:32
  • @JafferWilson OK, but how are you doing it? Actually, it might be simpler if you just come into chat a second and we can see what's wrong.
    – terdon
    Jan 4, 2017 at 11:33
  • it is simple. just using your solution and putting > and output file name. I thing this should give answer. Jan 4, 2017 at 11:36
  • @JafferWilson yes, it should, that's why I asked you to come to chat. There's nothing special here that would stop the file from being created. So come into chat where we can discuss it and you can show me the exact commands you used.
    – terdon
    Jan 4, 2017 at 11:37
  • The first sed command could be simpler with a record: sed 'sx\(.*\)x"http://\1"x' test.txt | tr '\n' ',' | sed 's/,$//' Jan 4, 2017 at 20:33
7

You could do it like this:

tr '\n' ',' < file | sed 's#[^,]\+#"http://&"#g;s#,$#\n#'

Explanation

  • tr '\n' ',' <file replace newlines with commas in file
  • | pipe output to
  • s#[^,]\+#"http://&"#g replace a series of characters excluding comma with the same characters prepended by http:// and surrounded by "quotes"
  • s#,$#\n# remove the last comma, replace with a newline

Note: this will fail if the urls have commas, so if that's possible in your case, use terdon's answer for reliable results

3
  • Swapping tr and sed would make the command easier to read and also immune to URLs containing commas: sed 's#.*#"http://&"#' file | tr '\n' ','
    – Byte Commander
    Jan 4, 2017 at 17:30
  • @ByteCommander I started out that way but I didn't want to pipe again to remove the last comma. Also, that solution is the same as terdon's, so I'm stuck with my post as it is I think :)
    – Zanna
    Jan 4, 2017 at 17:55
  • Oh right, the evil trailing comma...
    – Byte Commander
    Jan 4, 2017 at 18:02
5

Awk solution

awk '{ printf("\"%s\",", $0) }' input.txt

PHP allows terminating commas in array literals, if I recall correctly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.