Questions tagged [text-processing]

Questions about modifying, creating or parsing plain text files. This is a very broad tag, if your question is about using a specific tool, please tag it with the tool's name as well (e.g. sed, awk, perl, etc.)

Filter by
Sorted by
Tagged with
-1 votes
1 answer
28 views

Trying to remove all text except for IPs, data is in a file I downloaded

This is what I normally use to normalize the data, but it is not working. I need to keep only the IPs, one IP per line, and everything else must go. I'm using this in an .sh shell script. grep -oE '[...
  • 1
0 votes
0 answers
11 views

How to match results with grep from a list of words in a file? [duplicate]

for this example: grep 'word1\|word2\|word3' /path/to/file can these be placed in a file? something like: grep '<file_with_a_list_of_words>' /path/to/file for a large list of words to match.
1 vote
2 answers
190 views

awk or sed command to replace line break plus text containing spaces

An answer to another question suggests sed -i 's/original/replacement/g' file.txt to replace specific words in a text file. My starting situation looks like this: Item: PRF Type: File ...
0 votes
0 answers
72 views

Tail in combination with sort

I use tail in a variable to echo contents of multiple files in a folder. VAR="$(tail -n +1 DIR/*/file)" This outputs all the files contents with the needed filename on top like ==> /...
  • 101
6 votes
1 answer
279 views

How to sort by year-month-day from ascending using the "sort" command?

I want to sort dates in the format 31.12.2023. Lowest year should be displayed first then lowest month then lowest day. Is the following code fitting my needs? Seems good on first look. sort -t '.' -k ...
  • 101
-4 votes
1 answer
86 views

count frequency values of all csv

I have a large csv file (without header or index), for example A T C G G T A C CT T A G G G G G I want to count all values in all csv (not for special column or row), the output will be: A 3 T 3 C 2 ...
0 votes
6 answers
156 views

Cut last two lines of a file and append to another file repeatedly

I tried the following command to cut out the last two lines of a file and append them to a new file tail -n -2 file1 >> file2 && head -n -2 file1 > tmp && mv tmp file1 With ...
  • 101
-1 votes
1 answer
58 views

Remove last occurrence in a file with sed

I want to remove the very last occurrence of "(* cds_ams_schematic *)" in a file using sed command. `view schematic `timescale 1ns / 1ns (* cds_ams_schematic *) module ... endmodule // ...
2 votes
1 answer
157 views

Grep only until certain symbol

I try to grep some string and want grep to stop at a '+' symbol. The searched string could be "foo" or "foo+bar". Grep treats all findings no matter what comes after foo and the -w ...
  • 101
0 votes
1 answer
181 views

How to use "sed" for replacing xml tag in multiple lines xml content on bash

I want to use sed to replace the XML tag on bash. Here is the example xml content for testing: <xml-content> <validation> <timeout>2880</timeout> <subject>example</...
  • 127
2 votes
1 answer
38 views

How to search output of help message of program (using e.g. grep)

I have a directory with executables and I want to search for a word in the output of the help messages of these executables, i.e. the text that is printed in the console after using the command ./...
  • 477
0 votes
1 answer
60 views

Using awk to parse for conditions over multiple lines? (ifconfig output)

I'm trying to parse ifconfig output with awk. Right now my script is: # Setup ifconfig > ip.txt ans=$(cat ip.txt | awk '$1 == "inet" {print $2}') #output echo "Here are your IPs:&...
0 votes
2 answers
100 views

Ubuntu Bash - print specific word from a file

I need some help with some bash. I've only learnt like grep, awk, find, sed etc. Say I have a file named people.txt with different names on a line on multiple lines but a name appears in every line. ...
0 votes
1 answer
193 views

Can globbing be used to search file contents?

I basically understand that globbing is used for filename expansion in the shell. I also understand that regular expressions are used for text pattern matching within a file. However, is globbing also ...
  • 129
1 vote
2 answers
513 views

How to delete a range of lines in a text file?

I want to know how to quickly delete text with Bash. For example, I have a file named file.txt with a total of 500 lines. I want to delete lines 80 to 146 - how do I do that? What's is a terminal ...
  • 177
1 vote
2 answers
42 views

How to monitor a growing log file for specific string and print the line to terminal in realtime?

For example. I have nohup.out.log file which is growing over time. I want to monitor this file and check if the string test error occurs in this file and print the corresponding lines to terminal or ...
4 votes
4 answers
1k views

Convert text file to CSV on shell

I would like to convert a text file into CSV format. Here is an excerpt from the file: {"Outdated":false,"Watt":233,"Timestamp":1669647142,"A_Plus":6523.896,&...
0 votes
0 answers
16 views

How to deal with large formatted documents quickly in Ubuntu?

I have a .docx file that have a size of 200MB. To open or save such huge files, MS Office on Windows 10 only need several seconds, but Libre Office on Ubuntu 22.04 take dozens of seconds. (FYI: iWork ...
  • 263
2 votes
2 answers
481 views

Replace a line of text in a file, with the contents of another file

I have two files, let's call them foo.py and fixed_rule_map. I need to: find the line in foo.py that contains the pattern rule_map = . It would be enough to find the pattern rule_map, if that's ...
  • 285
2 votes
1 answer
191 views

How to log URL to a file only if it’s HTTP status code is 200?

I want to make a script so I can check if I can connect to several URL. For example I do curl -i -L --silent URL | grep "HTTP/2 200" >> *.txt I want to combine it with an if statement ...
  • 21
2 votes
1 answer
208 views

Oneliner to get the command which started a process on a certain port

I would need a one-line command which, given a port number, returns information on the command which started the process running on that port. Usually, I get this information by running a sudo netstat ...
  • 165
5 votes
2 answers
616 views

How do I replace multiple fields in multiple XML files?

I have around 4000 XML files and I need to replace the value of both <filename> and <path> fields. I need to replace those fields dynamically. e.g. images0001.xml should have images0001 ...
0 votes
3 answers
154 views

How to select several words, and remove all lines that contain those words?

See screenshot. I want to get rid of the clutter, and leave only things like ID, and TEXT, but remove the other things, such as type, date, date_unixtime, edited, from, etc. I tried the following ...
user avatar
0 votes
1 answer
39 views

Extract [a-c] from file and save in another file

I have a file with alphabetically sorted lines. The file starts from 0-9 till a-z. Example 000dasd 001dss awer addd alko babb bobb ... zzasd Since the file is really big I want to divide it into ...
  • 3
1 vote
3 answers
228 views

To make separate files based on pattern within a single file

I have a file containing 2.3M lines. Which looks like: $less V2.fastq >TS19_EWP4IQK02JPFP5 CATGCTGCCTCCCGTAGGAGTTTGGTCCGTGTCTCAGTACCAATGTGGGGGACCTTCCTC ...
  • 59
-1 votes
2 answers
27 views

How to fail command if zero or more than one line in output

Let's say I have a command like git log --some-options and I expect it to produce exactly one line so I could feed it to xargs, like git log --some-options | xargs git branch --all --contains how do I ...
  • 191
0 votes
2 answers
43 views

How do you break up lines character-wise?

I have a big text file made up of 64 hex values. Each line is stored one after the other; some lines are incorrectly recorded with 63 character hex values, while others are stored with 61 character ...
  • 91
2 votes
4 answers
988 views

How to print the content from a text file with one blank line after each line?

This is what I type and the output I get: $ cat helloworld.txt Hello World! I'm Kanladaporn Sirithatthamrong 6480952 But this is what I want an output to be: Hello World! I'm Kanladaporn ...
user avatar
0 votes
2 answers
52 views

sed / awk / etc - Sorting merged text before paste OR paste special: new line

My scenario: On an internal platform a list of IPs is displayed like: 8.8.8.8 1.0.0.1 1.1.1.1 192.168.1.1 192.168.0.1 When I copy these values and put them (pasted) in a text file they are all pasted, ...
  • 143
0 votes
0 answers
29 views

script for heap utilization on ColdFusion server

I have a command which return the output for heap size utilization:- jcmd `jps -l | grep com.adobe.coldfusion.bootstrap.Bootstrap | cut -f1 -d ' '` GC.heap_info it gives output like this 16144: ...
3 votes
2 answers
3k views

How can I delete specific string from a file

I do have a bash script. I want to delete a specific string from a text file. I already found a command on the online but it seems it isn't that stable.. I've tried; mv .admins .adminsold sed -r "...
  • 379
6 votes
2 answers
6k views

How can I check if string exists in file

I'm currently writing a bash script that should check if the exact string 329, exists in myfile. I have searched through the web and found some answers, but I can't use -x parameters because I have ...
  • 379
3 votes
3 answers
873 views

How to insert an ascii character in a line position?

I would like to edit a list of MACs in bl:ab:la:bl:ab:la format >> in to blab.labl.abla format. Each MAC is in a line. I'm new in the use of AWK or SED but I'm almost sure that it can be ...
  • 113
0 votes
1 answer
170 views

grep command to get file list with concatenated custom text

I am trying to list of files' names with custom text(my another command) with it. I am using below command: grep -rl --include=*.php --include=*.html --include=*.js 'ajax.googleapis.com/ajax/libs/...
  • 219
0 votes
0 answers
77 views

PDF Post Processing with Evince (Document Viewer)

I have a scanned PDF document which I've opened with Ubuntu's document viewer (Evince). When I select a part of the text in the document, it automatically processes the scanned text and converts it to ...
2 votes
4 answers
744 views

how to separate each content in the file?

I have a text file (more then 1GB in size) and it contains lines like these: ...
  • 91
0 votes
2 answers
374 views

How to remove characters/strings/patterns between, before and after some other characters/strings/patterns in a fasta file using a Ubuntu command line

How can I change all the lines in a fasta file from this: >vsearch_derep1;size=1 331 95 544 TRINITY_DN40607_c0_g1_i1 len=2000path=[0:0-1097] ...
  • 13
0 votes
2 answers
295 views

Sed (or other command) replace character range with hex codes

Let's say we have a file with non-printable characters. I would like to visualize those using their hex codes. Arte there a single liners that can achieve it, without writing a full-blown program. say ...
  • 283
0 votes
1 answer
174 views

How can I extract text between lines containing timestamps?

I'm looking to text process an existing file and have the output of the alrorithm output to a new file. I thought it would be an easy task, but it's flummoxing me, probably because I don't know ls ...
0 votes
0 answers
188 views

How do I get formatted responses from terminal?

I'm struggling to understand the response that I receive from the terminal as it is all pasted in one line without indentation, line breaks, etc. Is there a way to enable that? Or some command? This ...
0 votes
2 answers
111 views

from single column file to multi column file

I have a file like this: azg 1234567890 asgz 1234367890 aztyu 1234377890 ast 1234367899 asgassfa 1134367890 and I want to create an output to look like this: did dialer-label 1234567890 ...
0 votes
0 answers
130 views

XML content replace

I Have json example of replacing the json content dynamically as below and printing the id after creating an issue in jira .. Same operation needs to be done for TestNGXML report Kindly suggest the ...
  • 1
0 votes
1 answer
1k views

Parse JSON in Bash

I'm making a polyglot program, which takes input and runs files. I would like to run a command for a list of languages. Currently, I only have JavaScript and Ruby supported. My JSON file named ...
0 votes
0 answers
43 views

Problems with text-input in Ubuntu

Rapid text input is no longer possible everywhere. This is especially the case with two consecutive identical keys. E.g. double w, like http://www..., double forward-slash "//" double ...
  • 1
0 votes
2 answers
393 views

Creating an html from the output of awk script

I am performing in an awk script a series of aggregations from a dataset and I get the following. Now I would like to create an html that collects this previous output in the form of a table and that ...
0 votes
1 answer
60 views

how to make a bash script that will autofill the words in given statement one by one? [duplicate]

this is the statement sudo update-alternatives --install "/usr/bin/word" "word" "/usr/lib/jvm/jdk1.8.0_version/bin/word" 1 where word needs to be replaced ...
2 votes
3 answers
1k views

How to find number of unique words in first row using bash command?

I want to find the number of unique words in my file named cdj.tsv. I can use head -n 1 cdj.tsv to get the first line. Now I want number of unique words in this line. How can I get that? Result of the ...
  • 59
0 votes
1 answer
2k views

Replace simple text in the end if line with sed

I have that text #!/usr/bin/env python and I want to replace python with python3 (without regex) but I want to make sure that this text in the end of line. I tried this command: sed -i 's/python\n/...
0 votes
0 answers
20 views

Copy text from one file to another file in specific line [duplicate]

I would like to copy text with variable (for example uname) from one file to another file but in specific line. I tried use awk and sed, but variables doesn't work so I need firstly add variables to ...
  • 1
0 votes
1 answer
78 views

How to sort Time/Date output?

Let's have 16:00_17/06/2022 12:31_10/05/2022 10:34_07/03/2023 11:17_12/04/2022 12:37_10/05/2022 I would like to sort this output in ascending order to get this: 11:17_12/04/2022 12:31_10/05/2022 12:...
  • 197

1
2 3 4 5
23