Skip to main content

Questions tagged [csv]

A comma-separated values file (CSV) stores tabular data in plain-text form.

Filter by
Sorted by
Tagged with
0 votes
1 answer
37 views

Join details from two lines based on a match of the first two or three fields

I'm seeking help to extract / merge data from a single csv file. The file contains the arrival and departure times of ships over a few months. The details of the arrival and departure are on separate ...
James's user avatar
  • 11
2 votes
1 answer
824 views

How to batch rename files (images) recursively based on CSV file

before writing this post, I obviously searched the internet and found several solutions that could perhaps work. But I'm a complete newbie to command line programming or writing. What I have : ...
djinnroh's user avatar
-4 votes
2 answers
349 views

count frequency values of all csv

I have a large csv file (without header or index), for example A T C G G T A C CT T A G G G G G I want to count all values in all csv (not for special column or row), the output will be: A 3 T 3 C 2 ...
ARWA ABDULKADER BASHANFAR's user avatar
4 votes
4 answers
4k views

Convert text file to CSV on shell

I would like to convert a text file into CSV format. Here is an excerpt from the file: {"Outdated":false,"Watt":233,"Timestamp":1669647142,"A_Plus":6523.896,&...
jotrocken's user avatar
0 votes
1 answer
62 views

How to grab 2 data points from the beginning and end of a CSV file and subtract them using command line?

I was able to script the following line to output just the column I needed. I can't seem to grab the first number, 56485.65, and the last, 56498.2, to subtract the two and output the results, though. $...
ml41782's user avatar
0 votes
2 answers
7k views

How to convert terminal output to CSV format?

I'm currently running two scripts (One using iostat and one using ssacli) to collect some hardware performance data. I run my scripts directing their output to a .txt file. While this works for all ...
cVos's user avatar
  • 3
1 vote
2 answers
416 views

Why is a simple sed command not working on the raw csv file I downloaded from an EXO instrument?

I have hundreds of csv files downloaded from an EXO instrument that I want to reformat. The first part of every file contains general information on the instruments followed by a table with header and ...
SophiaFeuer's user avatar
0 votes
0 answers
972 views

Searching text from a list in another file

Is there a grep-like program for Ubuntu that can search a text file, or batch of text files in a folder, for any occurances of words in another text file or a list like a csv file or an ods file or ...
gerardnfarrell's user avatar
0 votes
0 answers
455 views

Some PHP codes aren't running in ubuntu 20.04

I'm very frustrated because of PHP codes that are not executing in my Ubuntu 20.04 but working smoothly in windows 10. There are many codes that are not running I have attached one. `<...
Naeem Navjivan's user avatar
0 votes
2 answers
103 views

Convert output file with the following columns to CSV removing the pipes

This is what I have in my output file: v6_ntoa | v6_ntoa | id | client_syn_packets ---------------+--------------+-------------+-------------------- 105.245.90.61 | 82.17.112.84 |...
Sal121's user avatar
  • 1
1 vote
1 answer
413 views

Read from a text file and get the corresponding row from a csv file

I have a text file named train_ids.txt, and a csv file named dataset.csv. The text file contains ids by this way: dish_1.png dish_5.png The input csv file has a lot of colums and rows, but the first ...
Abanoub Asaad's user avatar
1 vote
3 answers
6k views

CSV tools for terminal?

I'm using Ubuntu 18.04 and above, both desktop and server, and looking for terminal tools to query / write to large CSV file efficiently, and also easy to interact (few simple commands or commands ...
Pizza's user avatar
  • 1,446
-1 votes
2 answers
2k views

How to filter rows in a csv file with bash based in two conditions?

I'm doing a project in which I have to parse a csv file (indian liver patient dataset) and I'm trying to change the position of one column. The second to last must be the last column. I'm following ...
tucomax's user avatar
0 votes
1 answer
1k views

How to insert and update data from CSV using sqlcmd on ubuntu 20.04?

I can convert data database to .csv using sqlcmd syntax on my ubuntu sqlcmd -S ipdb -U sa -P "passwordsa" -d dbname -Q "select * from tabelname" -o "direktori/file.csv" -...
Preman Terminal's user avatar
0 votes
1 answer
81 views

Extract one column of strace -c

I am trying to get the system call names for common commands and don't have list of commands. Do you have any idea how can I get list of system calls for some commands. Finally I tried to do such ...
ali-sezar's user avatar
0 votes
2 answers
222 views

Replace sequence of a charachter with another one

I have an output from a network monitoring command and it looks like this: 391KB 7.48MB 7.86MB Which have a lot of spaces first. Now I want to replace all spaces with ,. ...
ali-sezar's user avatar
10 votes
1 answer
2k views

Convert a .txt file in a .csv with a row every 3 lines

Let's say I have a txt file like this: Mario [email protected] +399038259953 Luigi [email protected] +395902385093 Yoshi [email protected] +81293565291 [and so on for 300 lines...] How do I make a .ods (or ....
Dario Costa's user avatar
0 votes
3 answers
1k views

sorting a csv file with 3 columns

I am trying to sort a csv file containing 3 values. The first one is the one that matters. It contains many times the number 1, 2, 3, 4 etc etc. But each number doesnt appear the same number of times ...
Jim Hat's user avatar
  • 21
0 votes
2 answers
2k views

Individual attachments in mail merge from thunderbird

I am trying to send multiple emails with unique fields from thunderbird with mail merge addon in ubuntu. How to add individual attachments using this method? I have tried adding a separate field ...
Praveen's user avatar
  • 1,015
1 vote
0 answers
764 views

Adding multiple CSV files into one .XLSX file having different sheets using python

I have many CSV files which I want to add in a single excel file with each sheet having single CSV file and also I want name of CSV file at the top. Please help me with python code preferably.
Hrishikesh Morankar's user avatar
1 vote
1 answer
1k views

Split CSV file based on column values and number of lines

I must work with data from a huge CSV file in a format like this one. The file is very huge (~200MB) and my PC is having difficulties with it, so I would like to split the file in smaller ones easier ...
Ntakwetet's user avatar
  • 1,566
1 vote
1 answer
496 views

awk adding up two rows if the rows have a matching name

I am trying to take a csv file that looks like customerID,firstname,lastname,charge 5efc522,Romo,goodrick,37.89 4a7f057,jeny,vel,38.39 5efc522,Romo,goodrick,39.00 and make it into firstname,lastname,...
Ian McCauley's user avatar
0 votes
1 answer
2k views

Append specific output from ubuntu terminal in a csv file

Above is the output of my command. I want to save the highlighted output into a csv file in append mode. They should be in the same row but in different columns. My final csv file should be like this ...
Fathr's user avatar
  • 1
0 votes
1 answer
5k views

Create a CSV file from a TXT file

How do I create a CSV file from a TXT file as per the following example? Thanks. TXT file: This is line number one This is line number two with extra words This is line number three with even more ...
Jags's user avatar
  • 2,186
6 votes
1 answer
4k views

Trying to create a CSV file from a text file in Ubuntu

I have a text file that contains passwords and their respective hashes that is formatted as such: 'plaintext password' 'SHA256 hash' 'SHA3-512 hash' This format continues for several more passwords. ...
Ultracrepidarian's user avatar
0 votes
1 answer
55 views

How do I use a shell script to keep only the last populated value for each month in a CSV?

I have a csv file which gets updated by a cron daily and looks something like this: Date,Value 01/11/2019,123 02/11/2019,456 03/11/2019,789 ... 31/01/2020,123 01/02/2020,456 02/02/2020,789 03/02/2020,...
user1041276's user avatar
2 votes
4 answers
3k views

Selecting rows from csv for a particular day in bash script

I have a CSV file called test.csv containing these values : A,table,20191229 16:41:58 B,table2,20191222 16:41:58 C,table3,20191223 16:40:58 D,table4,20191228 16:41:58 E,table5,20191227 16:41:...
saaga's user avatar
  • 23
1 vote
1 answer
97 views

How to add new columns to CSVs with the same text in all lines

This a sample of my input file: 22/12/1999,conductivity,42,uS/cm 06/01/2000,conductivity,22,uS/cm 26/01/2000,conductivity,26,uS/cm 09/02/2000,conductivity,32,uS/cm 23/02/2000,conductivity,33,uS/cm I ...
Nicola Gambaro's user avatar
6 votes
1 answer
6k views

How to find the delimiter used in a txt file and convert it to comma separated csv?

I have a txt file that has either tab or space delimiter. I need to convert it to csv. How do you suggest to do so using command line? $ perl -F"//" -nlae 'for (@F) {$a{$_}++};for (keys %a){print $_ ...
Mona Jalal's user avatar
  • 4,605
4 votes
0 answers
2k views

Remove last character of a text file

How may I remove the last character of a text file in the shell without an unwanted final nextline? Can you believe I tried more than 3 hours... Actually, only this works : printf '%s\n' '$' 's/.$//...
Quentin's user avatar
  • 67
2 votes
1 answer
671 views

How to copy files containing a specific phrase to another directory?

I have a huge directory with log-files and want to move files WITH a specific "phrase" in it. i.e. log1.csv ------ containing word "xzy" log2.csv ------ containing word "abc" log3.csv ------ ...
Elfer Tipp's user avatar
0 votes
0 answers
212 views

Replace string with another string in bash script?

I have a .csv sheet with 3 columns and 'n' number of rows. Column1: It defines the name of the Rule. Column2: This is the schema line present in all the test files. Column3: This is the line which ...
Manoj Kumar's user avatar
0 votes
1 answer
111 views

How to rename the exisitng cell names based on number of their occurnaces in a particular column?

I have a column in my excel as below: REF --------- BV1013_1 BV1013_1 BV1013_2 BV1013_2 BV1026 BV1040 BV1040 BV1041 BV1041 BV1042 BV1042 Desired OUTPUT: Some of the cell values/names are same at ...
Manoj Kumar's user avatar
5 votes
4 answers
3k views

How to make a Bash script to change the format of a date in a CSV file

I've got a series of CSV files whose second field is dates: R.OUSE BOURTON MILL,2000-03-22T14:50:00,Ammoniacal Nitrogen as N,0.03,mg/l How can I change the date format into DD/MM/YYYY for all files? ...
Nicola Gambaro's user avatar
1 vote
0 answers
731 views

JSON TO CSV WITH JQ PLAY ERROR

I am new to this. Please forgive me. I am trying to make a CSV file from JSON data. I got far but my data is complex. I am using this line: jq --raw-output '(map(keys) | add | unique) as $cols | map(. ...
Pas Musique's user avatar
0 votes
2 answers
124 views

Compare 2 cols; when 1st col > 2nd col, remove 2nd col

How can I compare 2 cols in csv file, divided by ,, and cols contains quotes " when , is present in generated col, and when 2nd col > 1st col, then remove 2nd (last) col. I did something similar, ...
genderbee's user avatar
  • 850
0 votes
1 answer
191 views

CSV flies lost when transferred

I changed my OS from Linux to Windows 10 a week back. I saved a few CSV files and some code. After clearing the hard drive and making the switch, I realized that the CSV files were saved in this ...
Redstomite's user avatar
1 vote
1 answer
3k views

How to name the columns in BASH?

I am writing output of 6 UNIX variables into a CSV file. Yes,now I have a CSV file with all the columns that I want.But, I want to use those variable names as column Name/ column Header. How can we ...
Manoj Kumar's user avatar
-1 votes
1 answer
8k views

How to compare two rows in a csv file? [closed]

I've a data.csv file which looks as follows. -1.95e+01 -2.30e-01 -2.56e-01 4.44e+01 -1.95e+01 -2.30e-01 -7.68e-01 4.48e+01 -2.00e+01 -2.30e-01 -2.56e-01 3.41e+01 -2.00e+01 -2.30e-...
Jenny's user avatar
  • 591
2 votes
2 answers
5k views

AWK: Sum columns, write them into a "final" row

My CSV looks like this A 5 3 B 3 1 ... I need to get the sum of all columns and add those to a new line in the CSV so it becomes A 5 3 B 3 1 SUM 8 4 I was able to print the sum ...
user2161301's user avatar
0 votes
1 answer
577 views

System has 32 GB ram but maximum usuage is up to 6 GB

I am using Ubuntu 16.04 (64 bit) and my system has 32 GB RAM. When I run a python (64 bit) program on it, I get a "MemoryError". the output to free -m is: total used free ...
user3292475's user avatar
2 votes
3 answers
631 views

CSV to TSV without replacing commas within quotes

All the CSV to TSV tutorials are suggesting a simple: tr ',' '\t' though some CSVs look like this: 1,310,"IntAct,PINA" in which case I would like to keep "IntAct,PINA": 1 310 "IntAct,PINA" How ...
tr3quart1sta's user avatar
0 votes
2 answers
58 views

combine the rows to file3 only if the date is present in both files in sorted order of dates and month

My problem is I need to consider the rows to file3 only if the date is present in both files Consider file1.csv dtime,c,y,s 2015-02-01,395,396,g 2015-02-02,395,397,g 2015-02-03,395,397,g This file2....
sree's user avatar
  • 1
5 votes
1 answer
861 views

Changing second column in csv conditionally

I have a csv file that looks like: data/train/4/36280.png,four data/train/2/10317.png,two data/train/2/57890.png,two data/train/1/53448.png,one data/train/8/58233.png,eight data/train/4/23599.png,...
iKnowItAll's user avatar
3 votes
2 answers
6k views

MySQL importing CSV files really slow

I am trying to import CSV files into a table, but it has been very slow. I have about 1000 files with a file size of 40 MB each. Whenever I try to import it, I can see with for example MySQL workbench ...
Mark D's user avatar
  • 151
4 votes
1 answer
4k views

Libre Office Calc keeps removing quotes

I am opening a .csv file using LibreOffice Calc. However, no matter the format I choose to save the file with after modifying it (either ODF or CSV) for some columns that contained quoted values, the ...
pkaramol's user avatar
  • 2,191
7 votes
3 answers
26k views

From XML to CSV with command line

if I have a XML file on local disk, which command is need to convert file in CSV? And from XLSX or XLS to CVS? Thanks.
Noyzware's user avatar
4 votes
4 answers
3k views

Move specific column from csv file in front (select column by name)

Consider this data: #!/usr/bin/env bash cat > example_file.txt <<EOL group, value, price 1, 3.21, 3.21 1, 3.42, 4.11 1, 3.5, 1.22 2, 4.1, 9.2 2, 4.2, 2.11 EOL I want to move the 'value' ...
user2413's user avatar
  • 14.5k
0 votes
0 answers
835 views

csvkit - command line csv viewer and Ubuntu 18.04

Has anyone tried using csvkit in Ubuntu 18.04. I use the command csvlook to view csv files from command line. It used to work in other Ubuntu versions but in 18.04 I get error like the following when ...
Kinjal's user avatar
  • 163
0 votes
1 answer
5k views

Convert output of shell script to .csv

I want to convert the output of the below patchhistory.sh shell script to csv, so that I can get the information in column format like this: servername patch history ...
Nandlal Yadav's user avatar