Tag Info

Hot answers tagged

8

With only awk command: awk -F, '{getline f1 <"file2" ;print f1,$3,$4}' OFS=, file1 Get a line from file1 and store it into local variable f1, then print the line that stored in f1 and finally print the third($3) and forth($3) fields from file1 which delimited with comma , altogether, and change the OFS(output field separator [space by default]) to ...


7

There's an app for that! The newusers tool which is part of the passwd package is designed to batch create users: DESCRIPTION The newusers command reads a file (or the standard input by default) and uses this information to update a set of existing users or to create new users. Each line is in the same format as the standard password file (see ...


6

Here's a beauty (I think): join -t, <(csvcut -c 1,3,4 file1.csv) <(csvcut -c 1,2 file2.csv) Broken down in steps: Step 1. Install csvkit: sudo pip install csvkit sudo apt-get install python-dev python-pip python-setuptools build-essential Step 2. Use the join command with a comma as separator join -t, Step 3. Feed it the actual columns you ...


6

You can use tr: tr -s '[\0-\200]' ' ' < file replace all characters from ASCII-code=(0)10 to ASCII-code=(128)10 with single space. \nnn is character with octal value nnn. (128)decimal=(200)octal [\0-\200] like alpha class [A-Z] it's a character class that includes all ASCII control characters. or use : tr -s '[ -~]' ' ' < file This command is ...


6

This should work for you: sed 's/"//g' files.csv | while IFS=, read orig new; do mv "$orig" "$new"; done Explanation: sed 's/"//g' files.csv : remove the quotes IFS=, : split the input on , while read orig new; do ... done : This will read each input line, split it on the value of $IFS (here a comma) and save the 1st field as $orig and the rest as ...


6

I think you need a heavier programming language for this. Python is my language of choice so here's a simple script with a simple example of a test: import sys tests = [ lambda a, b, c, d, e: a+1==b and b+1==c and c+1==d and d+1==e, ] with open(sys.argv[1]) as f: for line in f: if any(t(*map(int, line.split(','))) for t in tests): ...


6

You can use the following perl command to create the CSV output, open a terminal and type: perl -n0e '@a= $_ =~ /"date":(".*?").*?"id":(".*?").*?"to":"(.*?)".*?".*?"subject":(".*?").*?"fromfull":"(.*?)"/gs; while (my @next_n = splice @a, 0, 5) { print join(q{,}, @next_n)."\n"}' inputfile.txt It will work even if you have multiple headers in your input ...


5

If we interpret your requirement to mean that the value of the third field (column) should be one more than that of the second field (column), then with awk you can do things like awk -F, ' $3==$2+1 {print "row "NR": "$0" was removed from "FILENAME > "file.log"; next}1 ' file.csv > newfile.csv which will create your file.log as specified and write ...


5

Here is another beautiful one. I think it is the easiest of all suggestions, thus far. csvtool pastecol 2 2 file1.csv file2.csv If you have not installed csvtool already in the past, you have to sudo apt-get install csvtool. From the docs: pastecol <column-spec1> <column-spec2> input.csv update.csv Replace the content of the columns ...


5

On the community website on converting there is a link to a command line tool called csv2xml. Since it is unmaintained you might want to choose another option. There is also mention of a java tool called csv2xml (warning: website is in German) and a command line tool called ff-extractor. The link also has references to Python, Perl, PHP, XSLT but that ...


5

In order to be able to use comm, you have to sort the lines first. comm -23 <(sort file1.csv) <(sort file2.csv) > file3.csv


5

With a script you can. This needs a file with user and password commaseparated: while IFS="," read -r user passwd ; do echo "useradd -m -p $(mkpasswd "$passwd") $user" done < /home/$USER/Downloads/users.txt mkpasswd can be installed with sudo apt-get install whois Change the , to a ; if you need that. Save this as users.sh and set ...


4

Here is an awk one-liner: sudo awk -F',' '{ command="useradd -p $( mkpasswd "$2" ) "$1;print command; system(command) }' userlist.txt Or for readability, here's the format: sudo awk -F',' '{ command="useradd -p $(mkpasswd "$2 ")" $1; print command; system(command) }' userlist.txt Explanation: awk's ...


4

I don't know about efficient, but sed with a regular expression. Make a backup of the file in case you make a typo, but perhaps something like this: sed -i 's/"\([0-9]\+\)_\([0-9]\+\)"/\1\2/' bigcsvfile.csv Edit: removed the g, as kos pointed out, no g is needed.


4

One way: echo "else if(a,b,c,d,e)" | perl -pe 's/,([a-z])(?=[^)])/+x==$1 and $1/g; s/,([a-z])/+x==$1/'


4

Since you are working with JSON files, why not parse it as such? Install nodejs-legacy and create a NodeJS script such as: #!/usr/bin/env node // parseline.js process lines one by one 'use strict'; var readline = require('readline'); var rl = readline.createInterface({ input: process.stdin, output: process.stdout, terminal: false }); rl.on('line', ...


3

I had a script that I adjusted (good idea the (N+1)q part!) thanks to @chronitis comment and the SO answer: #! /bin/bash # N=10 M=20 P=2 Q=3 sed -n "$N,${M}p; $((M+1))q" $1 | cut -d, -f$P-$Q Save the file as for example cut_csv, make it executable and use as cut_csv file It can be made fancier by accepting the N,M,P,Q parameters as input etc, but I ...


3

Just tied this way, with one of the people that need to use this, and that way is the correct way, even when it show it as (.csv) that doesn't matter, when opening up in Ubuntu gedit - it is working like the Notepad and everything is correct. So even if you have the latest version of LibreOffice, that is still working, and that link is for Open Office, and ...


3

Since you asked about perl specifically, it appears to be possible to do it using perl's \p{ARABIC} unicode property specifier, although it seems to be necessary to decode the UTF-8 encoding before applying the regex test. At its simplest, you could just delete non-arabic characters and re-encode: $ perl -MEncode -lpe '$_ = decode("UTF-8",$_); ...


3

Here is a Pythonic solution using the Counter class of collections module which will count the number of occurrences of each element in an iterable: #!/usr/bin/env python2 import collections with open('file.txt') as f: names = [] for line in f: names.append(line.strip().split()[0] + ' ' + line.strip().split()[2]) result_dict = ...


3

I have a shorter awk version (@Serg =)): First install whois, we need the command mkpasswd sudo apt-get install whois Than you can use this oneliner: awk -F',' '{print "Create user: "$1; system("sudo useradd -p $(mkpasswd "$2") "$1)}' users or without any output: awk -F',' '{system("sudo useradd -p $(mkpasswd "$2") "$1)}' users Content of users: ...


3

For a better CSV support use perl and Text::CSV #!/usr/bin/perl use strict; use warnings; use Text::CSV; my $file = $ARGV[0]; open my $fh, "<", $file or die "$file: $!"; my $csv = Text::CSV->new ({ binary => 1, auto_diag => 1, # uncomment the line below to avoid leading and trailing whitespaces allow_whitespace => 1, ...


2

My AWK version has a problem with , in the password. Here is my perl version: #!/usr/bin/perl use strict; use warnings; use Text::CSV; my $file = $ARGV[0]; open my $fh, "<", $file or die "$file: $!"; my $csv = Text::CSV->new ({ binary => 1, # Allow special character. Always set this auto_diag => 1, # Report irregularities ...


2

Here is a python solution: #!/usr/bin/env python2 import subprocess, crypt with open('/path/to/file.csv') as f: for line in f: i = line.find(',') name = line[:i] password = line.rstrip()[i+1:] encpt_passwd = crypt.crypt(password, '2b') command = 'useradd -m -p {0} {1}'.format(encpt_passwd, name).split() ...


2

awk -F"," '{ gsub (" ", "", $7); print "."$7"("$9"), ."$12"("$14"), ."$17"("$19")" }' For n pattern awk -F"," '{ gsub (" ", "", $7); for ( i = 1 ; i <=(NF-2) ; i++) if ( ((i - 7) % 5 ) == 0 && (i - 7) >= 0 ) printf "."$i"("$(i+2)"), "; print "" }'


2

You can create a hex-dump of your file with xxd which is part of the vim-common package. xxd file.csv | less Then check the line endings: 0a => \n 0d => \r 0d0a => \r\n


2

The following perl oneliner will extract the data for you: perl -e '/(.*)\t.*\t(.*)/ and $a{"$1 $2"}++ for (<>); print "$_ $a{$_}\n" foreach (keys%a);' file.tsv Output: joe ibm 2 joe google 2 rachel google 1


2

Paste this into a file and save it as posture.py for line in open('temp.txt','r'): a,b,c=line.split(':') state=b.replace('Posture side','').replace(' ','') side=c.rstrip().replace(' ','') print('%s,%s' %(state,side)) And then ./posture.exe > temp.txt python posture.py > posture_result_acc5.csv How it works. The output from your ...


2

awk -F',' '$N == "string to search"' filename.csv Replace N with column number and filename.csv with filename to search


2

When you know the format of the csv file and the structure you need in the xml file, it's fairly straightforward to make a script that can handle the conversion. Take the file simple.csv: Jack,35,United States Jill,22,United Kingdom You can create the following xml file: <?xml version="1.0"?> <Customers> <Customer> ...



Only top voted, non community-wiki answers of a minimum length are eligible