Questions tagged [csv]
A comma-separated values file (CSV) stores tabular data in plain-text form.
127
questions
2
votes
1
answer
418
views
How to batch rename files (images) recursively based on CSV file
before writing this post, I obviously searched the internet and found several solutions that could perhaps work. But I'm a complete newbie to command line programming or writing.
What I have :
...
- 23
-4
votes
1
answer
179
views
count frequency values of all csv
I have a large csv file (without header or index), for example
A T C G
G T A C
CT T A G
G G G G
I want to count all values in all csv (not for special column or row), the output will be:
A 3
T 3
C 2
...
4
votes
4
answers
2k
views
Convert text file to CSV on shell
I would like to convert a text file into CSV format.
Here is an excerpt from the file:
{"Outdated":false,"Watt":233,"Timestamp":1669647142,"A_Plus":6523.896,&...
- 51
0
votes
1
answer
50
views
How to grab 2 data points from the beginning and end of a CSV file and subtract them using command line?
I was able to script the following line to output just the column I needed. I can't seem to grab the first number, 56485.65, and the last, 56498.2, to subtract the two and output the results, though.
$...
- 1
0
votes
2
answers
2k
views
How to convert terminal output to CSV format?
I'm currently running two scripts (One using iostat and one using ssacli) to collect some hardware performance data. I run my scripts directing their output to a .txt file.
While this works for all ...
- 3
1
vote
1
answer
279
views
Why is a simple sed command not working on the raw csv file I downloaded from an EXO instrument?
I have hundreds of csv files downloaded from an EXO instrument that I want to reformat. The first part of every file contains general information on the instruments followed by a table with header and ...
- 11
0
votes
0
answers
693
views
Searching text from a list in another file
Is there a grep-like program for Ubuntu that can search a text file, or batch of text files in a folder, for any occurances of words in another text file or a list like a csv file or an ods file or ...
0
votes
0
answers
406
views
Some PHP codes aren't running in ubuntu 20.04
I'm very frustrated because of PHP codes that are not executing in my Ubuntu 20.04 but working smoothly in windows 10. There are many codes that are not running I have attached one.
`<...
0
votes
2
answers
78
views
Convert output file with the following columns to CSV removing the pipes
This is what I have in my output file:
v6_ntoa | v6_ntoa | id | client_syn_packets
---------------+--------------+-------------+--------------------
105.245.90.61 | 82.17.112.84 |...
- 1
1
vote
1
answer
273
views
Read from a text file and get the corresponding row from a csv file
I have a text file named train_ids.txt, and a csv file named dataset.csv.
The text file contains ids by this way:
dish_1.png
dish_5.png
The input csv file has a lot of colums and rows, but the first ...
- 135
0
votes
3
answers
2k
views
CSV tools for terminal?
I'm using Ubuntu 18.04 and above, both desktop and server, and looking for terminal tools to query / write to large CSV file efficiently, and also easy to interact (few simple commands or commands ...
- 1,378
-1
votes
2
answers
2k
views
How to filter rows in a csv file with bash based in two conditions?
I'm doing a project in which I have to parse a csv file (indian liver patient dataset) and I'm trying to change the position of one column. The second to last must be the last column. I'm following ...
- 3
0
votes
1
answer
533
views
How to insert and update data from CSV using sqlcmd on ubuntu 20.04?
I can convert data database to .csv using sqlcmd syntax on my ubuntu
sqlcmd -S ipdb -U sa -P "passwordsa" -d dbname -Q "select * from tabelname" -o "direktori/file.csv" -...
- 111
0
votes
1
answer
63
views
Extract one column of strace -c
I am trying to get the system call names for common commands and don't have list of commands.
Do you have any idea how can I get list of system calls for some commands.
Finally I tried to do such ...
- 11
0
votes
2
answers
132
views
Replace sequence of a charachter with another one
I have an output from a network monitoring command and it looks like this:
391KB 7.48MB 7.86MB
Which have a lot of spaces first. Now I want to replace all spaces with ,.
...
- 11
10
votes
1
answer
1k
views
Convert a .txt file in a .csv with a row every 3 lines
Let's say I have a txt file like this:
Mario
mario@me.com
+399038259953
Luigi
luigi@live.com
+395902385093
Yoshi
yoshi@yahoo.com
+81293565291
[and so on for 300 lines...]
How do I make a .ods (or ....
- 113
0
votes
3
answers
958
views
sorting a csv file with 3 columns
I am trying to sort a csv file containing 3 values. The first one is the one that matters. It contains many times the number 1, 2, 3, 4 etc etc. But each number doesnt appear the same number of times ...
- 21
0
votes
2
answers
1k
views
Individual attachments in mail merge from thunderbird
I am trying to send multiple emails with unique fields from thunderbird with mail merge addon in ubuntu. How to add individual attachments using this method?
I have tried adding a separate field ...
- 1,015
1
vote
0
answers
736
views
Adding multiple CSV files into one .XLSX file having different sheets using python
I have many CSV files which I want to add in a single excel file with each sheet having single CSV file and also I want name of CSV file at the top. Please help me with python code preferably.
1
vote
1
answer
891
views
Split CSV file based on column values and number of lines
I must work with data from a huge CSV file in a format like this one. The file is very huge (~200MB) and my PC is having difficulties with it, so I would like to split the file in smaller ones easier ...
- 1,336
1
vote
1
answer
364
views
awk adding up two rows if the rows have a matching name
I am trying to take a csv file that looks like
customerID,firstname,lastname,charge
5efc522,Romo,goodrick,37.89
4a7f057,jeny,vel,38.39
5efc522,Romo,goodrick,39.00
and make it into
firstname,lastname,...
- 17
0
votes
1
answer
2k
views
Append specific output from ubuntu terminal in a csv file
Above is the output of my command. I want to save the highlighted output into a csv file in append mode. They should be in the same row but in different columns. My final csv file should be like this
...
- 1
0
votes
1
answer
4k
views
Create a CSV file from a TXT file
How do I create a CSV file from a TXT file as per the following example? Thanks.
TXT file:
This is line number one
This is line number two with extra words
This is line number three with even more ...
- 2,051
6
votes
1
answer
3k
views
Trying to create a CSV file from a text file in Ubuntu
I have a text file that contains passwords and their respective hashes that is formatted as such:
'plaintext password'
'SHA256 hash'
'SHA3-512 hash'
This format continues for several more passwords. ...
- 169
0
votes
1
answer
54
views
How do I use a shell script to keep only the last populated value for each month in a CSV?
I have a csv file which gets updated by a cron daily and looks something like this:
Date,Value
01/11/2019,123
02/11/2019,456
03/11/2019,789
...
31/01/2020,123
01/02/2020,456
02/02/2020,789
03/02/2020,...
2
votes
4
answers
2k
views
Selecting rows from csv for a particular day in bash script
I have a CSV file called test.csv containing these values :
A,table,20191229 16:41:58
B,table2,20191222 16:41:58
C,table3,20191223 16:40:58
D,table4,20191228 16:41:58
E,table5,20191227 16:41:...
- 23
1
vote
1
answer
94
views
How to add new columns to CSVs with the same text in all lines
This a sample of my input file:
22/12/1999,conductivity,42,uS/cm
06/01/2000,conductivity,22,uS/cm
26/01/2000,conductivity,26,uS/cm
09/02/2000,conductivity,32,uS/cm
23/02/2000,conductivity,33,uS/cm
I ...
5
votes
1
answer
5k
views
How to find the delimiter used in a txt file and convert it to comma separated csv?
I have a txt file that has either tab or space delimiter. I need to convert it to csv. How do you suggest to do so using command line?
$ perl -F"//" -nlae 'for (@F) {$a{$_}++};for (keys %a){print $_ ...
- 4,249
4
votes
0
answers
2k
views
Remove last character of a text file
How may I remove the last character of a text file in the shell without an unwanted final nextline?
Can you believe I tried more than 3 hours...
Actually, only this works :
printf '%s\n' '$' 's/.$//...
- 67
2
votes
1
answer
528
views
How to copy files containing a specific phrase to another directory?
I have a huge directory with log-files and want to move files WITH a specific "phrase" in it.
i.e.
log1.csv ------ containing word "xzy"
log2.csv ------ containing word "abc"
log3.csv ------ ...
- 21
0
votes
0
answers
191
views
Replace string with another string in bash script?
I have a .csv sheet with 3 columns and 'n' number of rows.
Column1: It defines the name of the Rule.
Column2: This is the schema line present in all the test files.
Column3: This is the line which ...
- 353
0
votes
1
answer
88
views
How to rename the exisitng cell names based on number of their occurnaces in a particular column?
I have a column in my excel as below:
REF
---------
BV1013_1
BV1013_1
BV1013_2
BV1013_2
BV1026
BV1040
BV1040
BV1041
BV1041
BV1042
BV1042
Desired OUTPUT: Some of the cell values/names are same at ...
- 353
5
votes
4
answers
2k
views
How to make a Bash script to change the format of a date in a CSV file
I've got a series of CSV files whose second field is dates:
R.OUSE BOURTON MILL,2000-03-22T14:50:00,Ammoniacal Nitrogen as N,0.03,mg/l
How can I change the date format into DD/MM/YYYY for all files? ...
1
vote
0
answers
611
views
JSON TO CSV WITH JQ PLAY ERROR
I am new to this. Please forgive me. I am trying to make a CSV file from JSON data. I got far but my data is complex.
I am using this line:
jq --raw-output '(map(keys) | add | unique) as $cols | map(. ...
- 11
0
votes
2
answers
120
views
Compare 2 cols; when 1st col > 2nd col, remove 2nd col
How can I compare 2 cols in csv file, divided by ,, and cols contains quotes " when , is present in generated col, and when 2nd col > 1st col, then remove 2nd (last) col.
I did something similar, ...
- 830
0
votes
1
answer
157
views
CSV flies lost when transferred
I changed my OS from Linux to Windows 10 a week back. I saved a few CSV files and some code. After clearing the hard drive and making the switch, I realized that the CSV files were saved in this ...
1
vote
1
answer
2k
views
How to name the columns in BASH?
I am writing output of 6 UNIX variables into a CSV file. Yes,now I have a CSV file with all the columns that I want.But, I want to use those variable names as column Name/ column Header.
How can we ...
- 353
-1
votes
1
answer
7k
views
How to compare two rows in a csv file? [closed]
I've a data.csv file which looks as follows.
-1.95e+01 -2.30e-01 -2.56e-01 4.44e+01
-1.95e+01 -2.30e-01 -7.68e-01 4.48e+01
-2.00e+01 -2.30e-01 -2.56e-01 3.41e+01
-2.00e+01 -2.30e-...
- 591
1
vote
2
answers
4k
views
AWK: Sum columns, write them into a "final" row
My CSV looks like this
A 5 3
B 3 1
...
I need to get the sum of all columns and add those to a new line in the CSV so it becomes
A 5 3
B 3 1
SUM 8 4
I was able to print the sum ...
- 153
0
votes
1
answer
455
views
System has 32 GB ram but maximum usuage is up to 6 GB
I am using Ubuntu 16.04 (64 bit) and my system has 32 GB RAM. When I run a python (64 bit) program on it, I get a "MemoryError".
the output to free -m is:
total used free ...
2
votes
3
answers
569
views
CSV to TSV without replacing commas within quotes
All the CSV to TSV tutorials are suggesting a simple:
tr ',' '\t'
though some CSVs look like this:
1,310,"IntAct,PINA"
in which case I would like to keep "IntAct,PINA":
1 310 "IntAct,PINA"
How ...
- 319
0
votes
2
answers
54
views
combine the rows to file3 only if the date is present in both files in sorted order of dates and month
My problem is I need to consider the rows to file3 only if the date is present in both files
Consider file1.csv
dtime,c,y,s
2015-02-01,395,396,g
2015-02-02,395,397,g
2015-02-03,395,397,g
This file2....
- 1
5
votes
1
answer
688
views
Changing second column in csv conditionally
I have a csv file that looks like:
data/train/4/36280.png,four
data/train/2/10317.png,two
data/train/2/57890.png,two
data/train/1/53448.png,one
data/train/8/58233.png,eight
data/train/4/23599.png,...
- 53
3
votes
2
answers
5k
views
MySQL importing CSV files really slow
I am trying to import CSV files into a table, but it has been very slow. I have about 1000 files with a file size of 40 MB each. Whenever I try to import it, I can see with for example MySQL workbench ...
- 151
4
votes
1
answer
4k
views
Libre Office Calc keeps removing quotes
I am opening a .csv file using LibreOffice Calc.
However, no matter the format I choose to save the file with after modifying it (either ODF or CSV) for some columns that contained quoted values, the ...
- 1,981
7
votes
3
answers
20k
views
From XML to CSV with command line
if I have a XML file on local disk, which command is need to convert file in CSV?
And from XLSX or XLS to CVS?
Thanks.
- 71
4
votes
4
answers
3k
views
Move specific column from csv file in front (select column by name)
Consider this data:
#!/usr/bin/env bash
cat > example_file.txt <<EOL
group, value, price
1, 3.21, 3.21
1, 3.42, 4.11
1, 3.5, 1.22
2, 4.1, 9.2
2, 4.2, 2.11
EOL
I want to move the 'value' ...
- 13.7k
0
votes
0
answers
792
views
csvkit - command line csv viewer and Ubuntu 18.04
Has anyone tried using csvkit in Ubuntu 18.04. I use the command csvlook to view csv files from command line. It used to work in other Ubuntu versions but in 18.04 I get error like the following when ...
- 153
0
votes
1
answer
5k
views
Convert output of shell script to .csv
I want to convert the output of the below patchhistory.sh shell script to csv, so that I can get the information in column format like this:
servername patch history ...
2
votes
1
answer
892
views
script for csvtool for replacing columns
I want to replace certian columns in a csv file into another and i found this in the csvtoll man page. Which replaces column 1 in the first file with the column 1 in the second file.
csvtool ...
- 35