Questions tagged [text-processing]

Questions about modifying, creating or parsing plain text files. This is a very broad tag, if your question is about using a specific tool, please tag it with the tool's name as well (e.g. sed, awk, perl, etc.)

Filter by
Sorted by
Tagged with
0 votes
1 answer
13 views

Replace simple text in the end if line with sed

I have that text #!/usr/bin/env python and I want to replace python with python3 (without regex) but I want to make sure that this text in the end of line. I tried this command: sed -i 's/python\n/...
user avatar
0 votes
0 answers
20 views

Copy text from one file to another file in specific line [duplicate]

I would like to copy text with variable (for example uname) from one file to another file but in specific line. I tried use awk and sed, but variables doesn't work so I need firstly add variables to ...
user avatar
  • 1
0 votes
1 answer
35 views

How to sort Time/Date output?

Let's have 16:00_17/06/2022 12:31_10/05/2022 10:34_07/03/2023 11:17_12/04/2022 12:37_10/05/2022 I would like to sort this output in ascending order to get this: 11:17_12/04/2022 12:31_10/05/2022 12:...
user avatar
  • 187
2 votes
2 answers
124 views

Using grep to find tag and get arbitrary string after it

So I have this pattern, the whole thing is one line <img itemprop="image" class="hovered__image jsOpenGallery lazyload" data-src="//static.yellowpages.ca/ypca/ypui-6.65.0....
user avatar
3 votes
2 answers
178 views

Trying to uncomment unattended-upgrades lines with script, need help

I've recently started working as an IT person in a all Linux company, and noticed some of the tasks we do can be 'easily' automated. Today's task to automate is installing software and configuring /...
user avatar
0 votes
0 answers
32 views

How to remove all lines containing emojis from a text file?

I have text files in which some lines includes emojis or other non Ascii characters, like the list in this screenshot: I need a way to remove the whole lines containing an emoji or more. Even the ...
user avatar
0 votes
2 answers
15 views

Return rest of the line after string in other file

main.sh file: function checkSettings() { setting="$1 : " #| sed -e "s/\b\(.\)/\u\1/g" # awk -F '${setting}' '{print $2}' settings.tropx # sed 's/.*${setting} //' settings....
user avatar
2 votes
1 answer
332 views

How to find duplicate lines in very large (65GB) text files?

I have 10 text files, that contain 65 characters of hex value in each line. Each text files is 6.5GB in size (having 99999999 lines) i.e.file1-6.5GB, file2-6.5GB, file3-6.5GB, file4-6.5GB, ... file10-...
user avatar
  • 53
6 votes
2 answers
347 views

How do I grep for arabic characters with diacritical marks?

I have large TXT files in arabic Tashkil and I'm trying to find lines that contain specific pattern mashkula with َ ً ُ ٌ ّ ْ ٍ , I've tried the following grep syntax: cat file.txt | grep "...
user avatar
  • 187
1 vote
1 answer
193 views

remove a set of characters from a .txt file

I have a text file (more then 1GB in size) and it contains lines like these: ...
user avatar
  • 53
0 votes
2 answers
43 views

How do I get variable substitution to work with this sed command

I have this script: PATTERN=xAB dd if=/dev/zero count=1 bs=10 | sed "s/\x0/\${PATTERN}/g" > test_ab hexdump -C test_ab output I get is: $ ./make_test 1+0 records in 1+0 records out 10 ...
user avatar
  • 173
1 vote
2 answers
226 views

How can I use bash to replace a string in a file from another file

How can I use bash to replace a string in a file from another file For example: There is a path: \\Hostname\example\example.txt I want to replace the Hostname with 162 hostnames from another file that ...
user avatar
0 votes
2 answers
38 views

bash: do not show the part of the line that is in the search terms of the grep command

For example I have a directory: /A/B/C/D/E/F/G.txt Not knowing a priori /E/F/G.txt, I need to isolate that part. For ease of example: echo '/A/B/C/D/E/F/G.txt' | grep -o '/A/B/C/D' Shows: /A/B/C/D ...
user avatar
0 votes
1 answer
75 views

EOL conversion in bash

Let's say that i have a file that was created with Unix EOL format. Now, before sending that file to a windows, EOL conversion must be done. I need to do this in bash and this is what i have so far: ...
user avatar
  • 103
0 votes
0 answers
30 views

Find additions , deletions and updations between two text files

I have two text files that each have two space separated columns . File 1: Mary 123 Jack 836 Joan 936 File 2: Mary 123 Jack 837 Henry 430 What I'm looking for is to find names unique to both files ...
user avatar
0 votes
2 answers
44 views

How to insert char before every digit?

In a text file i have this: txt txt 12345 12345678 txt 1234 i need to insert ; before every number like this: txt txt;12345;12345678 txt;1234 how to do that with sed, awk or perl?
user avatar
  • 187
0 votes
1 answer
17 views

removing entire line with sed using wild characters

E219 E36 C5X C201 C19 C5X E229 E36 C5X E239 E36 C5X C311 C19 C5X In this file example, I need to remove all lines containing C[0-9][0-9][0-9]. How I can do this with sed command? The ...
user avatar
  • 284
0 votes
6 answers
176 views

delete 50 random rows in a txt files that have their second column labeled as 0

I want to use the terminal commands for deleting 50 random rows from a txt file that is tab separated and has these sample rows where the second column is 0. The value in the second column is either 0 ...
user avatar
  • 3,631
0 votes
2 answers
77 views

need help MD5sum hash

Guys I have a question I have a txt file full of names and I want to hash to md5sum for all the names in the txt at once with one by command Can you help with that and thanks in advance. Thank you ...
user avatar
5 votes
3 answers
696 views

How to break string into N 80 character lines?

I want to display a welcome message in the start of my script: echo "Running $0 $@" But $@ might be very long. How can I break this line into multiple up-to-80 character lines?
user avatar
0 votes
0 answers
27 views

find replace text in file

I have one problem the edit info in file .php the command I used is sed -i 's/\TCG\Voyager\Models\User/Authenticatable/' "app/Models/User.php" not edit segment in my shell script other ...
user avatar
3 votes
2 answers
613 views

Append the name of each text file to the last line of the file

I have ten files: firstFile.txt,...,tenthFile.txt On the last line of the file I would like to append the name of the file. i.e. for the first one the last line would be firstFile I would like to ...
user avatar
  • 33
2 votes
3 answers
92 views

How to grep a string from a file starting from a pattern and ending from a pattern

I have this script: #!/bin/bash user=datab-admin-role allusers="datab-admin-111role datab-admin-112role datab-admin-113role " if(...) then.. ... I want to grep only the first occurrence of ...
user avatar
  • 121
0 votes
0 answers
29 views

How can I diff two files, deleting lines in file1 from file2 [duplicate]

I want to take file1 and remove all the lines that are contained in file2, ideally keeping the filename file1 once done, keeping the unmodified lines of anything remaining. I've tried using diff as ...
user avatar
  • 1
1 vote
3 answers
44 views

How can I remove a string from every line of a file in the second column of a file

The file structure is like this: 34945,/data/file/system01/data/system/backups/filename.txt 393092,/data/file/system-03stby/data/system/backups/filename.extra.txt The result that I am looking for is ...
user avatar
1 vote
1 answer
77 views

Read from a text file and get the corresponding row from a csv file

I have a text file named train_ids.txt, and a csv file named dataset.csv. The text file contains ids by this way: dish_1.png dish_5.png The input csv file has a lot of colums and rows, but the first ...
user avatar
4 votes
2 answers
87 views

How do I combine two files while excluding lines that exist in both files?

I have 2 files that can not be sorted. Both of them have a list of words per lines. I am trying to compare both files and create a new one without any duplicate lines that get matched between both ...
user avatar
-1 votes
2 answers
54 views

How to remove specific character from file

I have a file like this. [Guest] [AAD_987d7f2f57d2] [mhope] [SABatchJobs] [svc-ata] [svc-bexec] [svc-netapp] [dgalanos] [roleary] [smorgan] How to remove the '[' and ']' from the file so i can get ...
user avatar
  • 11
3 votes
1 answer
144 views

How to change `lsblk` sort order?

Change lsblk sort order lsblk sorts partitions in an order that is hard to follow. A method of sorting it without having to write a bash script or python script would be the first choice. A similar ...
user avatar
0 votes
1 answer
48 views

sed and quotation marks

I get data files that contain both of these (with the commas and the quotation marks) ,"ncal", ,"NCAL", I want to replace all occurrences of ,"ncal", with ,"NCAL&...
user avatar
3 votes
4 answers
912 views

Is there any way I can search in a file for Uptime that is greater than 100 days?

Is there any way I can search in a file for Uptime that is greater than 100 days via Linux Bash commands? For example, file1 contains: Uptime is 172 days Uptime is 562 days Uptime is 30 days ...
user avatar
  • 31
0 votes
2 answers
60 views

sed with ANSI color escapes fail

I do not know why this does not work: echo -e "$(echo "This is an uncolored text" | sed "s/{This is an uncolored text}/{This is an \033[0;34uncolored text\033[0m}/g")" ...
user avatar
1 vote
1 answer
135 views

How to make `ncdu` show a quick summary of disk usage and exit?

I'd like to make the ncdu NCurses Disk Usage tool analyze a path, display the output, and exit, for scripting purposes. Normally it is a human-interactive tool, but I'd like to remove the human-...
user avatar
0 votes
1 answer
85 views

Strip ANSI sequences from string through bash-python code

I try to use bash to strip ANSI color escape sequences from a string without success. I tried already some regex-based code. #!/bin/bash Blue='\033[0;34m' # Blue Clear='\033[0m' # ...
user avatar
4 votes
1 answer
136 views

(LibreOffice) Replace every character in a selection with *

I want to shadow portions of a .odt document with stars for every character in the selection area. I guess it should be possible using the find and replace tool in LibreOffice. Just need to know how ...
user avatar
1 vote
1 answer
290 views

Bash string split by delimiter constrained by number of characters

I want to split a long text separated by spaces through bash, but I fail. The command below splits into characters, but not delimiters. echo "The quick fox jumped over the lazy dog" | fold -...
user avatar
0 votes
2 answers
37 views

How can I capitalise letters after a certain character

How can I capitalize letters before a certain character? (^) I am trying to do the reverse of the above link. I want to capitalise everything after a certain character or word. It can be using awk, ...
user avatar
4 votes
2 answers
817 views

Using sed with a variable inside double quote

I have a file input.xml which contains a line: <exciton lambda="1" fix="hole"/> in that line I want to replace 1 with 2 but I can't use just 1 as key word since there are ...
user avatar
0 votes
1 answer
68 views

Using awk/sed/perl i want to remove the self closing tag and its content and add an empty space

I have the following string: <Sub4Topic hi="1d9bhb" cbdshfghfgs(C)(2)(d)"><step stepid=" ajhdjgjgjfhgjhgrhghr" stepid="du" /> Here I want the step tag ...
user avatar
  • 11
0 votes
0 answers
33 views

Analyse DMARC reports

I receive Google's DMARC reports daily as a zip file containing an XML file. I manually download each zip file ~/Documents/DMARC. I want to aggregate the information from all the saved DMARC reports ...
user avatar
  • 1,078
0 votes
2 answers
61 views

Unable to see and edit the character that is typed when you press the Enter key in simple text editors like KWrite [closed]

I'm a newbie to the KDE Plasma interface. I find myself unable to see and edit the character that is typed when you press the Enter key in simple text editors like KWrite.
user avatar
  • 21
0 votes
1 answer
37 views

Filter blocks of log-output from a log stream

Here's my task: I've got a source stream of live log output from a messaging process. Lots of output is irrelevant to me but there are sections i want to collect and evaluate separately. Those blocks ...
user avatar
  • 181
3 votes
1 answer
199 views

Replace a text within single quotes with another string using sed(just a first occurrence of regex)

I have a file with the following content: $ cat file.txt CREATE COLLATION public.collation123 (provider = libc, locale = 'fr-FR'); CREATE COLLATION public.collation_1 (provider = libc, locale = 'bas'...
user avatar
0 votes
1 answer
48 views

How can I pad numbers with leading zeroes using the seq command?

I had to generate a text file for a given range of numbers. As usual I had to use seq range_start range_end > text_file.txt to put it in a text file. I gave the seq range where the starting point ...
user avatar
  • 366
4 votes
2 answers
432 views

How do I make a Markdown table of two columns out of a list whose items alternate to each column?

I have a long list of common abbreviations for words in periodical titles. In the list, the full word is followed by its abbreviation. For example: Administration Admin. Applied Appl. Administrative ...
user avatar
1 vote
1 answer
40 views

How to count the number of consecutive identical lines

I have a file that looks like this: 2000 2000 2001 2001 2001 2001 2002 2002 I need a script to show me this: 2000 - 2 2001 - 4 2002 - 2 I prefer using sed or awk
user avatar
  • 139
2 votes
2 answers
86 views

Filtering content of command with grep

I am trying to filter out a the output of ssh-keyscan. The goal of this is to filter the output so I can use it in my python code to identify hosts connected to my VPN. Normally I would use grep to ...
user avatar
  • 173
1 vote
1 answer
896 views

Apply sed operations only to lines beginning with a particular string

I have the following file format Received from +11231231234 at 2021-10-10T19:56:50-07:00: This is a message that contains words like from, at, etc. Sent to +11231231234 at 2021-10-11T06:50:57+00:00: ...
user avatar
  • 11
0 votes
1 answer
62 views

How to append to the end of a line before the first pattern match?

I have a file containing some random data like: number1 number2 number3 &END I want to add a \ to the end of the line containing number3, but I cannot match a pattern on that line as that number ...
user avatar
0 votes
1 answer
57 views

awk -- duplicating a column and also modifying its text

awk {'printf ("%5s\t%s\n", $0, $NF)'} test_VN.txt works for duplicating a column in a text file however, how can I add _VN before the .jpeg extension of the duplicated column? For example, ...
user avatar
  • 3,631

1
2 3 4 5
22