Questions tagged [text-processing]

Questions about modifying, creating or parsing plain text files. This is a very broad tag, if your question is about using a specific tool, please tag it with the tool's name as well (e.g. sed, awk, perl, etc.)

Filter by
Sorted by
Tagged with
0 votes
1 answer
54 views

How do I find files of a certain extension(s) except in paths that contain certain strings and then print the result to a file?

Upon a little search, I came up with this command: > find . -type f \( -name "*.exe" \) | egrep -v > 'games_1|rei|games|tor|jogos|mods|text' > 1 What I wanted to do was find files ...
Nuno Fonseca's user avatar
7 votes
5 answers
1k views

Cut folder names out of string

How do I extract the folder named test from the following string? TRUE|/home/linux/test|(null)|
scotty030253's user avatar
0 votes
4 answers
465 views

How to use sed to remove newlines above and below a string?

I'd like to change all occurrences of \ntoken1\n and \ntoken2\n to token1 and token2 in a file). I thought this would do it: ${ N N s/\ntoken1\n/token1/ s/\ntoken2\n/token2/ } but it does not. Thanks ...
John-L_.'s user avatar
0 votes
2 answers
104 views

Removing spaces between words in a text file using a bash script

I am using a bash script that parses data from a bunch of different text files and generates a .dts file. The .dts file contains spaces between each column title and I need to remove them as it makes ...
Josh Altman's user avatar
2 votes
3 answers
494 views

What grep pattern allows to print only one-word lines?

I want to print the lines with only one no-spaces subsequences of characters while use grep on the text. title after grepping should to print title the sentence after grepping should not print ...
palmasd1's user avatar
0 votes
3 answers
161 views

sort in shell using multiple -t

I have a file that contains: 192.168.130.175 2014-09-04 10:25:01 /index.html 192.168.138.244 2014-09-04 11:23:00 /index.html 192.168.138.244 2014-09-04 10:29:37 /...
龚健翔's user avatar
4 votes
2 answers
786 views

Using grep to search for line that begins with a variable whose value is a dollar sign

I found a lot of very similar problems, but not exactly this. I have a text file with the following contents (no repeats, fixed 4 characters per line): A1234 G1234 $1234 M1234 I'm trying to use grep ...
Aylox's user avatar
  • 43
-1 votes
2 answers
207 views

Filter RE in grep

The command that I have is: grep -E '^ [0..9]' The pattern that I have 24 -38.54270131 24 -38.54085331 24 -38.53603119 24 -38.53427780 24 -38.53396521 24 -38....
Another.Chemist's user avatar
0 votes
3 answers
172 views

Pull mac address via bash script

A bit background. Im creating a script that will run on USB stick with Ubuntu. It needs to pull various info from the current laptop its connected to and save it to a file. Right now Im using "...
pa-sc's user avatar
  • 3
2 votes
4 answers
611 views

using bash, create files based on pattern match

we have a file with content as below. /* ----------------- pattern_1 ----------------- */ jb: pattern_1 Data: asdflj adfas Log: dir/log1 /* ----------------- pattern_2 ----------------- */ jb: ...
team 9's user avatar
  • 23
0 votes
1 answer
146 views

Search for a character string in Html files and output this and the <title>.*</title> tag of the file

I want to search Html files for a character string across all lines. If the string is found, then this string and the <title>.*</title> tag of the file shall be output. The search must be ...
elli's user avatar
  • 33
1 vote
1 answer
228 views

Deleting lines beginning with a string

I wrote a bash script that I use to backup certain directories to the cloud. I then direct that output to a log file. The log file consists of lines starting with either "upload" or "...
Kenneth Odle's user avatar
4 votes
2 answers
425 views

Modify text file with command

How can I modify a text file to do the following in 1 command? I have a text file called Errors.txt with the following 3 lines: Error in File /Test1/A.png Error in File /Test2/B.jpg Error in File /...
Kryptonyx's user avatar
-1 votes
1 answer
50 views

Multiple text exchange in all files in a folder

I want to search all files in a folder to this text $pw = 'my_password' If $pw = 'my_password' is included in the file, I want this through $pw = $new_pw' substitute also, I would like if $pw = '...
D-T's user avatar
  • 383
1 vote
1 answer
176 views

How to iterate eml file with mailutils?

How would a directory of .eml files be iterated? nicholas@mordor:~$ nicholas@mordor:~$ mail -f foomail/foo.eml "/home/nicholas/foomail/foo.eml": 0 messages nicholas@mordor:~$ Whereas, I'...
Nicholas Saunders's user avatar
6 votes
5 answers
2k views

Replace the same text in more than 200 files in the same folder

I have more than 200 files in one folder on Ubuntu 22.04. I would like to search these for a specific text. If the text is found, it should be replaced by another text. How can I achieve this?
D-T's user avatar
  • 383
0 votes
1 answer
205 views

How to import an mbox file into the Pine or Alpine mail client?

I'm a light Alpine e-mail client user. If I navigate into ~/mail and run mail -f baz I'll get an expected list of old alpine/pine messages. Alpine doesn't use mbox. I added an mbox backup to that ...
Nicholas Saunders's user avatar
1 vote
3 answers
413 views

Exclude email addresses in a separate file

I have a file that contains, among other characters, many hundreds of email addresses. The email address is always between the characters "=>" and "<=". So =>email@example....
D-T's user avatar
  • 383
-1 votes
1 answer
65 views

Remove lines from a TXT file that start with a special string [duplicate]

Is it possible to remove all lines starting with // from a txt file?
D-T's user avatar
  • 383
-1 votes
1 answer
173 views

Bash script to compare lines in 2 formatted files

I have two formatted files with lines with fields separeted by a delimitter ( in this case, '*' ) File1: qwerty*asd*1111111*poiu*222222*mnb nbcvxm*hjf*3333333*jhjd*444444*bvc dhfkjs*bmb*5555555*vcxz*...
JotaCe's user avatar
  • 1
5 votes
6 answers
1k views

Delete all lines after a certain number of lines

How can I truncate after 1000 lines in a csv file? The goal is to keep only the first 1000 lines in the file and delete all others.
D-T's user avatar
  • 383
0 votes
1 answer
194 views

Need to create csv file from of content of 1 file

I need help in writing a command or shell script that generates a csv file as per below details - My file has content like - + Creates a new NotificationEventMapping POST https://sbc.com:1234/openapi-...
guptpallav's user avatar
1 vote
2 answers
233 views

Process tesseract output: remove line breaks, concatenate individual pages

I have a pdf that I cannot directly process into .txt to go through piper TTS because the output from the .pdf is missing letters and otherwise generally unintelligible (https://pastebin.com/J3rASD2F):...
iconoclasthero's user avatar
1 vote
1 answer
125 views

Sorting a text file based on numerical values present in each line

I have a text file containing paths of various wav files. But when they are created they are sorted based on alphabetical order. Hence the paths are like this /home/wav_path/...
Turing101's user avatar
  • 143
-1 votes
2 answers
255 views

How can I make output of sed command as variable?

How can I make the below command as variable ? I mean I need to assign this command to variable to use it within for loop. for i in $(sed 's|/home/bssuser/Scripts/Amany/log/bssuser/CDR/...
Amany Abd Elmnem's user avatar
0 votes
1 answer
34 views

Join details from two lines based on a match of the first two or three fields

I'm seeking help to extract / merge data from a single csv file. The file contains the arrival and departure times of ships over a few months. The details of the arrival and departure are on separate ...
James's user avatar
  • 11
0 votes
3 answers
173 views

Create columns in text file

I have used a text-from-image generator to pull info from a image of a glossary that look like this: word1 :meaning1 word2 :meaning2 word3 :meaning3 word4 :meaning4 Which resulted ...
Kris's user avatar
  • 13
0 votes
1 answer
58 views

How to find a string in files, said string including line breaks, ", @

I know grep allows me to search for a string in files, however I did not manage to find the right method to search for the following string: "snapshot3" : { "@custom_name" : ...
Vinz M's user avatar
  • 169
0 votes
0 answers
113 views

How to get some lines before and after in log files using tail and grep? [duplicate]

How to get some lines before and after in log files using tail and grep ?
indrajit narvekar's user avatar
0 votes
0 answers
25 views

Re-arranging a table of values with VIM

I'm often in the situation of having to re-arrange a table of values, from this format: a = 1 b = 2 c = 3 d = 4 e = 5 f = 6 to this format a = 1 b = 2 c = 3 . . . in VIM. The fastest way to make it ...
SO_32's user avatar
  • 163
0 votes
1 answer
136 views

How can I determine the prematch and postmatch using egrep or ksh under linux

I am working on a issue at the office running on a linux system. I would like to be able to either use egrep or ksh pattern matching to determine not only the matched string but I also need to be able ...
Barry Kimelman's user avatar
0 votes
1 answer
155 views

How to delete multiline string with grep

Let's say I have file.txt with following content (simplified for example. In real it's much more complex): This line I DON'T want to remove \ common substring This line I want to remove \ ...
JaSON's user avatar
  • 123
2 votes
1 answer
169 views

Multiply found text in sed - is it possible?

This is what I have: echo -e "a\t4\nb\t7\nc\t2\nd\t12\n" | sed -rn 's/(.*)\t([0-9]*)$/\2\t\1\1\1\t/p' 4 aaa 7 bbb 2 ccc 12 ddd This I want: 4 aaaa 7 bbbbbbb 2 cc ...
xerostomus's user avatar
0 votes
1 answer
342 views

Add a list to specific lines of a (YAML) text file using python

I have an Ansible module file for NetBox with this content: - name: "Test NetBox modules" connection: local hosts: localhost gather_facts: False tasks: - name: Create device ...
mohammad's user avatar
1 vote
5 answers
322 views

list items correctly in bash with a space in their name

I have for example following Files: test1.txt test2.txt test space.txt I use the following command: output=$(ls -l --time-style=long-iso | awk '{print $8, $6}') And I get following output: test1.txt ...
Chuck's user avatar
  • 19
2 votes
2 answers
3k views

How to know a JSON array is an empty array in bash?

I use this code to consume GitHub API and automate some tasks: curl --silent -H 'Authorization: token github_access_token' 'https://api.github.com/orgs/OrganizationName/repos?per_page=100' Sometimes ...
Saeed Neamati's user avatar
3 votes
2 answers
731 views

bash script to compare fields of two files and replace columns

I have two files. File A: newname,1,string newname,2,string newname,3,string ... name,65500,string File B; oldname,1,string oldname,2,string oldname,3,string ... oldname,65500,string I need a bash ...
masoud hanifehzadeh's user avatar
8 votes
7 answers
998 views

Compare new txt file with old txt file and remove all data that matches

I have a new file with the following data separated by a carriage return a a b c d d I have an old file that is also separated by a carriage return b d How do I remove b & d from the new file ...
Robert Carroll's user avatar
0 votes
2 answers
291 views

Shell or bash script to open multiple URLs saved in a .TXT file in a browser from the terminal?

Is there a shell or bash script to open multiple URLs saved in a .TXT file in a browser from the terminal? I know that can be done with "firefox $(cat url.txt)", but: I need something that ...
Jeisson Cadena's user avatar
5 votes
3 answers
1k views

how can I get unique ip addresses from /var/log/auth.log?

How do I only show the unique IP addresses in /var/log/auth.log ? I have a server getting hit constantly, and while I have set up fail2ban, the requests keep coming. So I started writing to ISPs to ...
j0h's user avatar
  • 14.8k
0 votes
2 answers
457 views

How to parse large files looking for a word

I have some extremely large json files I am not even really sure if some are even written correctly Therefore viewing the with a editor is not possible (it freezes vscode etc) I can view them with ...
KansaiRobot's user avatar
1 vote
1 answer
291 views

How to find and replace a string in all files on ubuntu including root files with another string

I am running Ubuntu 20.04 server. It is a fully configured LEMP email and web server. In my super long process of building this, i know I entered my servers IP address on one or many of my ...
DanRan's user avatar
  • 95
0 votes
2 answers
447 views

print multi-line text with spaces aligned on the first line

Issue: I write relatively complex functions where I have to make up the code with spaces. It is expected that the first line with the content (non-space character) will be determined, N spaces will be ...
Avraam's user avatar
  • 41
0 votes
1 answer
54 views

find a number after a special character and multiply it by awk

I have a file: "UDP", "x", 12880, "any", "any", x, 0.0.0.0/0, 0.0.0.0/0, {85000, drop, -1, fast+dsample:1000}, {1m, alert, -1, none} How can I change 85000 ...
masoud hanifehzadeh's user avatar
0 votes
1 answer
879 views

how can i find the average of data in a column in a txt file?

I need to write a script that can find the average of numbers in a column and then print out the value when run. I am quite new to Linux commands, and I've tried my best, but I don't even know how to ...
Mhmoud Njedeen's user avatar
5 votes
2 answers
771 views

How to add double quotes around string and number pattern?

Hi I need to add double quotes in a pattern in 300k lines. I'm trying to use sed and I read multiple inquiries here and other sources, but I can't seem to understand its syntax. I have: chr1 ...
Thalles's user avatar
  • 51
1 vote
1 answer
130 views

How can I maintain the original creation date or crtime of .txt files I create?

I take a lot of notes in plain .txt files that I write in the default text editor and it is very useful for me. There is only one thing missing to make my "workflow" perfect. I would really ...
JMike's user avatar
  • 51
5 votes
1 answer
486 views

combine single text with multiple lines of file

I am trying to combine work tvg-name with multiple lines of a file tr '\n' '~' < file1.txt | sed s/~/tvg-name/g results in only first single-line print tvg-namegama1 my file1.txt has more than ...
Rizwan.A's user avatar
1 vote
2 answers
133 views

Unique values separated by semicolons (thumbnailers)

The file /usr/share/thumbnailers/totem.thumbnailer contains: [Thumbnailer Entry] TryExec=/usr/bin/totem-video-thumbnailer Exec=/usr/bin/totem-video-thumbnailer -s %s %u %o MimeType=application/mxf;...
Mario Palumbo's user avatar
0 votes
1 answer
25 views

Compare two files and remove every line that doesn't match with file B

I have two files. file A: Modern Tales: Age Of Invention Playstation 4, 1935, 3478-3480 Modern Tales: Age Of Invention Xbox One, 3074 Moero Chronicle Switch, 6667, 12400, 28910, 29900, 29901, 29920 ...
masoud hanifehzadeh's user avatar

1
2 3 4 5
24