Questions tagged [text-processing]

Questions about modifying, creating or parsing plain text files. This is a very broad tag, if your question is about using a specific tool, please tag it with the tool's name as well (e.g. sed, awk, perl, etc.)

Filter by
Sorted by
Tagged with
1
vote
1answer
56 views

How to add n columns below first column? [closed]

I have a tab-separated data: 1-1 2-1 3-1 ... 1-2 2-2 3-2 ... 1-3 2-3 3-3 ... 1-4 3-4 ... 3-5 My desired output should look like this: 1-1 1-2 1-3 1-4 2-1 2-2 2-3 3-1 3-2 3-3 3-4 3-5 ... ...
1
vote
2answers
48 views

How can I save the output of DIG in CSV format?

I am using DIG command in linux and getting results i want but how can i get output results written in CSV file and get them saved ? Current output is below which i want in csv file : inspectieSZW....
0
votes
1answer
24 views

How to search start and end with square bracket with multiple line

Text have [{hello=(1+2-0), hello2 = 1}] But are in two or three multiple lines when start with [ and end with ] are in different line How to extract like grep for this multiple line pattern?
0
votes
2answers
81 views

How to do a simple mathematical operation on all lines of a specific column?

I have a text file named nvt.gro. I want to add 0.5 (+0.5) on all the lines of the last column: my file is as below: GRM in vacuum 192700 1GRM C1 1 17.188 0.311 13.994 -0.5971 0.0204 -...
2
votes
4answers
142 views

How to align the text file after copy and paste?

I copied column 7,8 and 9 from file 1 into columns 7,8 and 9 in file 2 producing a new file 3. the produced file (file 3) is not aligned as the original files, How can I edit it to preserve the ...
2
votes
6answers
347 views

convert (sed or awk, or anything) file containing the following:

I am looking for a way to convert with a one line linux command (sed or awk, or anything) the following file (example1.txt); 1.2.3.4:21 172.16.1.2:80 192.168.5.4:443 192.168.10.1:7007 into a format ...
1
vote
1answer
27 views

awk gsub question

I've used awk in the past for large file manipulation and substitutions.Recently, I've used it to substitute, for ex letter A with a set of characters: $ awk '{gsub(/A/,"@@@")}1' in.txt >> out....
1
vote
3answers
126 views

Is it possible to delete a minus sign from a specific column (the last column) in a text file?

Below is part of my file named nvt.gro I want to delete the minus sign "-" in the last column only. I have tried to select the complete column and then try to modify it, but I didn't succeed. Can ...
0
votes
0answers
45 views

Comparing fuzzy filenames

Sometimes humans make an error or intentionally make text difficult to read. For example, 1 Queen Live At Hammersmith Odeon 1975 FULL GREAT SOUND.mp4 2 Queen Live at Hammersmith Odeon 1979 FULL CLEAN ...
0
votes
3answers
46 views

Problem with Copy pasting text from a document on ubuntu

Version: 19.10 The problem is while i copy some text from a file to libre office or google docs ( where i am creating my own document ), after pasting the text takes same formatting as of the source, ...
-1
votes
1answer
37 views

In a log like this: 21-03-2020 07:09 [LOG] ralex /home/ralei/secret ACCESS_WRITE_OK How can i see if the user is the same in both path and log?

So I have a fie of logs like this: 20-03-2020 10:01 [LOG] davis /home/davis/fis1 ACCESS_WRITE_OK 20-03-2020 11:11 [LOG] davis /home/davis/Pictures/cat.png ACCESS_READ_OK 20-03-2020 12:22 [LOG] root /...
1
vote
2answers
66 views

How to remove “ [ ” symbol from “ [19/Mar/2020:05:57:09 ” string using sed or awk?

How to remove [ from " [19/Mar/2020:05:57:09 " using sed or awk? I need to remove "[" from date and time from log file .
-1
votes
2answers
56 views

How to combine TSV file

I have four TSV files like this: file_1: abc \t 1 def \t 3 ghi \t 5 file_2: abc \t 10 def \t 4 ghi \t 7 file_3: abc \t 8 def \t 5 ghi \t 1 file_4: abc \t 4 def \t 2 ghi \t 1 I want to combine ...
1
vote
1answer
52 views

How to filter data with the highest number

I have thousands data like this: abc_1 abc_2 abc_3 abc_4 def_1 def_2 def_3 ghi_1 ghi_2 I want to filter it to get only data with the highest number which is abc_4, def_3, ghi_2. How I can get it?
2
votes
2answers
284 views

How to split string in TSV file

I have TSV file like this: abc_1 def_2 ghi_3 jkl_4 mno_5 I want to split that in to the: abc def ghi jkl mno and 1 2 3 4 5 How I can get that?
1
vote
1answer
76 views

Using AWK, how to use tabs instead of Space as field seperator?

I have the following fields: Name<tab>City<tab>Phone Number If I use to awk, it returns only first name ("Muhammad"), when name is "Muhammad Tahir". How can I use awk with grep so it ...
1
vote
3answers
95 views

Move 10 digit number to the beginning of file name

I have thousands of files that I need to rename. They all contain a 10 digit number that all begin with '42469'. I need to move this 10 digit in each of the files to the beginning of the filename ...
0
votes
0answers
41 views

bash: syntax error near unexpected token '('

I am trying to use git for the first time and have been given some code to extract a file name from a MatLab script. I keep encountering this error, and if I remove the bracket after imread it does ...
10
votes
1answer
1k views

How to convert graphical images to ASCII art?

I want a custom image in my terminal made of characters, is there a program I can use? Specifically, the gear within my school's robotics logo.
1
vote
1answer
59 views

how to pass argument “all” to column util-linux

I would like to apply some options of the column command of the util-linux to all my columns. I'm particularly interested in the --table-right, it works if a pass the number of the column. But I would ...
-1
votes
1answer
53 views

Sed, extract lines from text

I really would like to know how can I extrac some line from a text file using sed, but I'd like to extract the lines at interval: For example extract the line with number i=1:2:100 So that i can ...
-1
votes
1answer
33 views

Ways to terminal to replace all text from one document with another?

so grep gives us all these fancy ways of searching for text within documents, i want to know how you would take text from one document and put it in another document and/or replace the texts that was ...
0
votes
0answers
12 views

command or a script to find words that are in a file but absent from another? [duplicate]

I have two .txt files that are supposed to be exact replicas. Each file contains one word in each line. So basically, the number of lines is the number of words for each file. Here's an example of how ...
0
votes
0answers
169 views

How can I use aria2c to convert magnet link to .torrent file while changing .torrent file name?

I am using aria2c to convert a magnet link to a .torrent file with the following command: aria2c --bt-metadata-only=true --bt-save-metadata=true <MAGNET_LINK> However it returns a really messy ...
9
votes
1answer
401 views

How to pass list of names to be searched by grep using xargs?

I have a text file containing names (nameslist.txt) and I want to read them using cat and pipe the result with xargs to a grep command, so that grep checks the existence of each name that it receives ...
1
vote
1answer
62 views

Grep the contents of a file for contents from another file one at a time

I am using grep -o <string> <filename> | wc -l to count the number of times <string> occurs in the file. I now need to extend this such that the string is from a list of items in ...
1
vote
1answer
257 views

How do I get values from string and then convert these values to different time zone?

Data format as follows: 2019 12 18 18 15 05 TIMESTAMP 2019 12 18 23 15 05 TIMESTAMP These values in strings are respectively year, month, day, hour, minute, and second. I try to write awk ...
2
votes
1answer
65 views

How do I get words of certain length?

Supppose I have data like this: this is ttttaaaaa aaattt The expected output that I am looking for is: is ttttaaaaa I am using awk. This is the idea: awk 'length <= 4 && length >= ...
1
vote
1answer
59 views

extracting specific lines between first and second repated strings in text file

My data file looks like: #cP2019 6 30 0 0 0.00000000 192 ORBIT IGS14 BHN ESOC ## 2060 0.00000000 900.00000000 58664 0.0000000000000 + 54 ...
1
vote
1answer
138 views

How can I change the third occurrence of a word in a file using sed?

I need to change the 3rd occurrence of 'None' in a file text.txt using the sed command. I found this command sed 's/None/All/3' text.txt but this works for 3rd occurrence of 'None' in a line, not in ...
6
votes
2answers
647 views

How to grep words from a single string matching a pattern?

Will anyone guide me to grep only words containing pattern _ARA from the below single string. String: LINK:['IM219MIR_ARA1','IM18Q4_ARA1','SM18Q4_ARA1','IM18PLANNING_ARA1','IM118Q4DYNVA_ARA1','...
3
votes
2answers
81 views

Getting lines between special char pattern

Below is my input file: --- { "date":"2015-09-24", "title":"Getting Started with Git", "template":"post", "thumbnail":"content/thumbnails/test.jpeg", "slug":"getting-started-with-git", "...
1
vote
1answer
141 views

Remove lines starting with “string1” and NOT containing “string2” with sed

I would like to delete some lines in a file with more than 100K lines of data. I only want to delete line which started with MX and NOT containing the word sum. How can I do that with sed? ...
1
vote
0answers
91 views

Combining files based on chromosome and position next to each other - column vise [closed]

I have multiple files in format: chr position value. I want to combine them in format "chr", "position", "samp1", "samp2", "samp3", "samp4",........ For example: Samp1: chromosome position value ...
1
vote
0answers
63 views

Remove last character of a text file

How may I remove the last character of a text file in the shell without an unwanted final nextline? Can you believe I tried more than 3 hours... Actually, only this works : printf '%s\n' '$' 's/.$//...
3
votes
4answers
169 views

How do I remove trailing nonalphabetical characters from every line?

I'm trying to remove last characters except alphabets: support.help1.com,, support.help1.com. support.help1.com9 support.help1.com* support.help1.com@@ support.help1.com## support.help1.com%% support....
4
votes
4answers
725 views

How to make a Bash script to change the format of a date in a CSV file

I've got a series of CSV files whose second field is dates: R.OUSE BOURTON MILL,2000-03-22T14:50:00,Ammoniacal Nitrogen as N,0.03,mg/l How can I change the date format into DD/MM/YYYY for all files? ...
1
vote
3answers
103 views

How do I remove lines with sed that contain more than 1 word?

I need to remove lines from a file that contain > 1 word. Here is an example of the file's content: This line is not what I need printed-now. would print This not printed-now. I have tried: sed -...
6
votes
4answers
565 views

Count number of different name in a file

I want to count the number of different name in a text file of this presentation: 2008 girl Avah 2009 girl Avah 2008 girl Carleigh 2011 girl Kenley 2012 boy Joseph 2013 boy Joseph 2014 boy Isaac 2014 ...
0
votes
2answers
108 views

Compare 2 cols; when 1st col > 2nd col, remove 2nd col

How can I compare 2 cols in csv file, divided by ,, and cols contains quotes " when , is present in generated col, and when 2nd col > 1st col, then remove 2nd (last) col. I did something similar, ...
9
votes
3answers
572 views

How can I remove rest of file from string for all files?

How can I remove rest of file from string for all files? For example, input files 1 and 2, string is ddd. Input file 1. aaa bbb ccc ddfbbd dddaaa eee Output file 1. aaa bbb ccc ddfbbd Input ...
3
votes
1answer
144 views

sed replacing character in a file

I need to replace a single character in a/etc/request-key.conf The file format is; ############################################################################### # # Copyright (C) 2005 Red Hat, Inc....
2
votes
2answers
827 views

Bash, get values from json file, all in one line

Is possible get data for one specific product, closed in { }, from json, by EAN, all data are in one line? Example Input. ... { "article": {"code": "01333457004","name": "ALAZANIS VALLEY 2015","...
1
vote
2answers
207 views

sort is very slow for big files

I want to sort file (with removing duplicates) which contains wordlist with a size of almost 25GB. I am using sort command in Ubuntu terminal but it takes hours to output sorted file, the command is: ...
2
votes
2answers
45 views

How to merge two files' data line by line?

I have two files, and want to merge data line by line. The input files are file.txt and file2.txt. file1.txt howareu abc zooo file2.txt 123 @@ 099 I want to merge the two file, such that output ...
1
vote
1answer
59 views

Why does uniq seem to keep some consecutive identical lines?

Without uniq: amin@ubuntu:~/Desktop$ cut -f 1 info.log | tail -n +2 | head -n -1 | sort Abol Abol Ahmad Akbar Arash Hadi Hamed Mahmood Maryam Maryam Mohsen NIma Rasool Sadegh Sepide Sepide With ...
1
vote
1answer
81 views

How do I run similar commands with files in ten separate folders?

I have a folder y2015 in which I have ten subdirectories (d001, d002, d003, ..., d010). In each dXXX folder I have to run these commands which are shown below: gd2e.py -mkTreeS myKinTree rm /...
1
vote
1answer
42 views

Show only selected data in text file without removing any data from it

I have a file with dates (formatted YYYY M DD) and their max/min/avg temperature, alongside with the precipitation for each day. The dates are in the first three columns of strings (1900 9 19 for ...
2
votes
1answer
120 views

Create files if they don't already exist

I have many text files enumerated. I want to add missing text files and fill them with 0; e.g.,: -directory -1.txt -2.txt -6.txt I want to add 3.txt, 4.txt, and 5.txt to the ...
0
votes
5answers
81 views

Sorting each column and getting their corresponding key IDs in the first row for the bottom 2 values

This is my working file. (Column number and Row number are not known) key_ID col_1 col_2 col_3 AA 1 1 1 BB 56 2 7 CC 89 5 2 DD 7 77 7 I want to sort ...

1
2 3 4 5
18