Tag Info

Hot answers tagged

7

With only awk command: awk -F, '{getline f1 <"file2" ;print f1,$3,$4}' OFS=, file1 Get a line from file1 and store it into local variable f1, then print the line that stored in f1 and finally print the third($3) and forth($3) fields from file1 which delimited with comma , altogether, and change the OFS(output field separator [space by default]) to ...


6

This should work for you: sed 's/"//g' files.csv | while IFS=, read orig new; do mv "$orig" "$new"; done Explanation: sed 's/"//g' files.csv : remove the quotes IFS=, : split the input on , while read orig new; do ... done : This will read each input line, split it on the value of $IFS (here a comma) and save the 1st field as $orig and the rest as ...


6

I think you need a heavier programming language for this. Python is my language of choice so here's a simple script with a simple example of a test: import sys tests = [ lambda a, b, c, d, e: a+1==b and b+1==c and c+1==d and d+1==e, ] with open(sys.argv[1]) as f: for line in f: if any(t(*map(int, line.split(','))) for t in tests): ...


5

If we interpret your requirement to mean that the value of the third field (column) should be one more than that of the second field (column), then with awk you can do things like awk -F, ' $3==$2+1 {print "row "NR": "$0" was removed from "FILENAME > "file.log"; next}1 ' file.csv > newfile.csv which will create your file.log as specified and write ...


5

You can use the following perl command to create the CSV output, open a terminal and type: perl -n0e '@a= $_ =~ /"date":(".*?").*?"id":(".*?").*?"to":"(.*?)".*?".*?"subject":(".*?").*?"fromfull":"(.*?)"/gs; while (my @next_n = splice @a, 0, 5) { print join(q{,}, @next_n)."\n"}' inputfile.txt It will work even if you have multiple headers in your input ...


5

Here's a beauty (I think): join -t, <(csvcut -c 1,3,4 file1.csv) <(csvcut -c 1,2 file2.csv) Broken down in steps: Step 1. Install csvkit: sudo pip install csvkit sudo apt-get install python-dev python-pip python-setuptools build-essential Step 2. Use the join command with a comma as separator join -t, Step 3. Feed it the actual columns you ...


5

In order to be able to use comm, you have to sort the lines first. comm -23 <(sort file1.csv) <(sort file2.csv) > file3.csv


5

Here is another beautiful one. I think it is the easiest of all suggestions, thus far. csvtool pastecol 2 2 file1.csv file2.csv If you have not installed csvtool already in the past, you have to sudo apt-get install csvtool. From the docs: pastecol <column-spec1> <column-spec2> input.csv update.csv Replace the content of the columns ...


4

One way: echo "else if(a,b,c,d,e)" | perl -pe 's/,([a-z])(?=[^)])/+x==$1 and $1/g; s/,([a-z])/+x==$1/'


4

Since you are working with JSON files, why not parse it as such? Install nodejs-legacy and create a NodeJS script such as: #!/usr/bin/env node // parseline.js process lines one by one 'use strict'; var readline = require('readline'); var rl = readline.createInterface({ input: process.stdin, output: process.stdout, terminal: false }); rl.on('line', ...


3

I had a script that I adjusted (good idea the (N+1)q part!) thanks to @chronitis comment and the SO answer: #! /bin/bash # N=10 M=20 P=2 Q=3 sed -n "$N,${M}p; $((M+1))q" $1 | cut -d, -f$P-$Q Save the file as for example cut_csv, make it executable and use as cut_csv file It can be made fancier by accepting the N,M,P,Q parameters as input etc, but I ...


3

Just tied this way, with one of the people that need to use this, and that way is the correct way, even when it show it as (.csv) that doesn't matter, when opening up in Ubuntu gedit - it is working like the Notepad and everything is correct. So even if you have the latest version of LibreOffice, that is still working, and that link is for Open Office, and ...


2

This creates a CSV file with file name, time stamp and size for all files in /some/folder/ and its subfolders: find /some/folder -printf '"%P";"%Tc";"%s";\n' See the documentation for -printf in the manpage for find if you want to use other fields. Note that it doesn't work for file names containing " characters.


2

perl: $ perl -i.bak -F, -ane ' if ($F[0]+1 == $F[1] and $F[1]+1 == $F[2]) {warn "row $.: $_"} else {print} ' file.csv 2>file.log $ cat file.log row 7: 2,3,4,25,11 $ cat file.csv 4,6,18,23,26 5,12,19,29,31 2,5,13,16,30 9,10,24,27,32 4,5,10,19,22 4,6,8,10,25


2

Skip the perl part and try this: awk -F',' '{x = $1"+x=="$2; \ for (i=2; i< NF; i++) { \ x = x " and " $i "+x=="$(i+1) \ }; \ print "else if" x \ }' Effect: $ echo '(a,b,c,d,e)' | awk -F',' '{x = $1"+x=="$2; \ quote> for (i=2; i< NF; i++) { \ quote> x = x " and " $i "+x=="$(i+1) \ quote> }; \ quote> ...


2

Pure textual python solution. It needs at least 2 variables, no maximum. #!/usr/bin/env python3 sourcefile = "/path/to/sourcefile" def newline(oldline): subject = oldline.replace(" ", "").split("(")[-1].replace(")", "").split(",") out = [subject[i]+"+x=="+subject[i+1] for i in range(len(subject)-1)] print("else if("+" and ".join(out)+")") ...


2

You can convert this JSON to CSV in a single line with jq. jq '.data.headers | [.sender, .to, .subject, ."x-received-time", .received, .from, .date, .id, .to, .subject, .fromfull] + [(.time | tostring)] | join(", ")' Breakdown: .data.headers - Emit headers as an object If data contained an array of headers it would be .data[].headers […string keys ...


2

Try this command: grep -v -f file2.csv file1.csv > file3.csv According to grep manual: -f FILE, --file=FILE Obtain patterns from FILE, one per line. The empty file contains zero patterns, and therefore matches nothing. (-f is specified by POSIX.) -v, --invert-match Invert the sense of matching, ...


2

A python option: #!/usr/bin/env python3 import sys def readfile(file): with open(file) as src: return [line.strip() for line in src.readlines()] lines_1 = readfile(sys.argv[1]); lines_2 = readfile(sys.argv[2]) for line in lines_1: if not line in lines_2: print(line) Output: 1,4,5,6 1,11,13,17 Paste the script into an empty ...


2

Without sort and Why didn't work with diff? $ diff file1.csv file2.csv | grep -Po "^< \K.*" output if lines exist in file1 but not in file2: 1,4,5,6 1,11,13,17 And output if lines exist in file2 but not in file1, with just changing left angle(<) to right angle(>): $ diff file1.csv file2.csv | grep -Po "^> \K.*" 2,4,9,10 13,14,17,18


2

awk -F',' '$N == "string to search"' filename.csv Replace N with column number and filename.csv with filename to search


1

To move a chosen number of columns from one file to another: #!/usr/bin/env python3 cols = 1; file_1 = "/path/to/file_1"; file_2 = "/path/to/file_2" def readfile(file): with open(file) as src: return [item.strip().split(",") for item in src.readlines()] file_1 = readfile(file_1); file_2 = readfile(file_2) for i in range(len(file_1)): ...


1

Here is an awk implementation: awk -F ":" '{gsub("\"","",$1);key=$1;sub(key " ","");gsub("\\","",$0);value[key]=$0; if ("fromfull"== key) print value["from"] ";" value["to"] ";" value["fromfull"] ";" value["id"] ";" value["subject"] ";" value["date"] ;}' jsonFile > csvFile This script read line until found "fromfull" line, than print csv line, so it ...


1

Here's a gawk script I just whipped up for you! #!/usr/bin/gawk -f BEGIN { FS="\"" output="" nodata=1 } /^"data"/{ if( ! nodata ) { gsub("|$","",output) print output nodata=0 } output="" } /^"[^d][^a][^t][^a]/{ if ( $2 == "to" || $2 == "fromfull" || $2 == "id" || $2 == "subject" || $2 == "date" ) output=output$4"|" } END{ ...


1

Part of the answer depend which kind of interface you want and the kind of output you require (quality, details, etc). If you are OK with a command line interface, the "classical" package for doing that is gnuplot, as stated in the comments. This is what I normally use for the first shot. I however think that PyXplot will normally give much nicer plots ...


1

Your file names should be escaped with quotes - MySQL considers them strings, so your MySQL block should be: LOAD DATA LOCAL INFILE '$RDDATEI' INTO TABLE bv_tmp_all FIELDS TERMINATED BY ';'; SELECT * FROM bv_tmp_all GROUP BY BAN INTO OUTFILE '$bv_out_distinct' FIELDS TERMINATED BY ';'; LOAD DATA LOCAL INFILE '$bv_out_distinct' INTO TABLE ...


1

You can combine awk and sed with a pipe sed -n '10,50p' file.csv | awk -F ';' '{print $3 $4}' 10 and 50 are rows. -F ';' is the field separator.(Semicolon in my example) $3 and $4 are the fields to show.


1

The following script using head and tail is able to print a part of a .csv file filtered through row and column's number. #!/bin/bash m="$2" n="$3" s="$4" t="$5" head -n "$n" "$1" | tail -n +"$m" | cut -d, -f "$s"-"$t" Save the above script as csv_view.sh and make it executable. chmod +x csv_view.sh where, m=row number where to begin n=row number ...


1

Thank you to @bodhi.zazen for the answer: #!/bin/bash # This file will gather who is information while IFS=, read url do whois $url &> "$url.txt" done < Urls.csv



Only top voted, non community-wiki answers of a minimum length are eligible