Questions tagged [text-processing]
Questions about modifying, creating or parsing plain text files. This is a very broad tag, if your question is about using a specific tool, please tag it with the tool's name as well (e.g. sed, awk, perl, etc.)
84
questions with no upvoted or accepted answers
4
votes
0
answers
2k
views
Remove last character of a text file
How may I remove the last character of a text file in the shell without an unwanted final nextline?
Can you believe I tried more than 3 hours...
Actually, only this works :
printf '%s\n' '$' 's/.$//...
- 67
3
votes
1
answer
198
views
searching for "created_at": followed in the next line by "retweeted": in lots of json files for a specific range
I have a large amount of json files and previously I was told to use the following line to search if a tweet is created at a specific time range in a date:
grep -wirnEzc '},.*created_at":\s"Wed Oct ...
- 4,545
2
votes
1
answer
771
views
What is the default encoding, used in text files in Ubuntu?
I created a normal text file "input.txt" in Ubuntu. I added some data to it. My questions are:
What is the default encoding, used in text files in Ubuntu?
How do I find its encoding ...
2
votes
2
answers
3k
views
Bash - putting the first word of every line into an array
I'm trying to put the first word from every sentence into an array.
What am I doing wrong?
#!/bin/bash
file_name=$1
content=$(cat $file_name)
content=${content//"\n"/" "}
content=${content//". "/"\n"...
- 21
2
votes
2
answers
593
views
remove the preceding "file://" when pasting in gnome terminal
I am using Ubuntu 14.04 but I guess this thing can be seen in almost all versions of Ubuntu.
When I copy a file from nautilus using Ctrlc and paste into gedit it pastes the text like /home/urvish/....
- 21
2
votes
1
answer
693
views
Change character encoding in a text file
Is there a way to change the character encoding of a text file? If so, how?
See, I am using a keyboard shortcut to call a shell command which itself uses xclip to copy the contents of a file into ...
- 1,572
2
votes
0
answers
740
views
applying a rich text file formatting profile to a plain text
I have an .rtf , ~70 pages dictionary manuscript at hand in the following format:
I have been asked to change those lines to:
without formatting it is an easy thing to do, but i couldn't figure out ...
- 93
2
votes
2
answers
208
views
How do I write a Bash script for this scenario?
I have these files in the directory home/acs.
TBBN027162G_38d8-2f08-0c40
TBBN027605G_38d8-2f08-1518
TBBN027941G_38d8-2f08-0d1c
TBBN027923G_38d8-2f08-35e4
I want to read TBBN027162G as service_id and ...
1
vote
1
answer
232
views
Deleting lines beginning with a string
I wrote a bash script that I use to backup certain directories to the cloud. I then direct that output to a log file.
The log file consists of lines starting with either "upload" or "...
- 51
1
vote
1
answer
130
views
How can I maintain the original creation date or crtime of .txt files I create?
I take a lot of notes in plain .txt files that I write in the default text editor and it is very useful for me. There is only one thing missing to make my "workflow" perfect. I would really ...
- 51
1
vote
2
answers
134
views
Unique values separated by semicolons (thumbnailers)
The file /usr/share/thumbnailers/totem.thumbnailer contains:
[Thumbnailer Entry]
TryExec=/usr/bin/totem-video-thumbnailer
Exec=/usr/bin/totem-video-thumbnailer -s %s %u %o
MimeType=application/mxf;...
- 287
1
vote
0
answers
59
views
Extracting specific data from different tables and merging it
I would like to extract specific rows from specific columns from different tables, based on the column header. However, my knowledge seems to be too limited to solve it myself (both in command-line ...
1
vote
1
answer
224
views
Extracting Specific Information from file
I have a file created from a command I run on my system. After the file is created I want to extract only specific information from that file but only it it meets certain criteria.
Here is what my ...
- 11
1
vote
0
answers
211
views
Ubuntu shift arrow key to mark text?
I just installed latest ubuntu 20.04.2 LTS on my dell inspiron as dual boot with windows 10.
I used to mark text with the keyboard using Shift+Arrow, Shift+Pos1 and Shift+End on the numeric keypad. (...
- 11
1
vote
1
answer
62
views
repeated set of lines
A set of 3 lines in my .txt file
many sets are one after the other.
For example:
1.message: afcdd77f17519b364c1992095399d
2.signature: bbfecd2f9f1c2fabedb3d15f310
3.-----------
1.message: ...
- 91
1
vote
1
answer
95
views
Print selected columns from a file if the row is mentioned in another file
My first file contains data like:
Gene symbol Gene name Chromosome Start End Strand Otter gene ID Otter transcript ID Ensembl gene ID Ensembl transcript ID Gene biotype Transcript biotype
...
1
vote
2
answers
121
views
Show only selected data in text file without removing any data from it
I have a file with dates (formatted YYYY M DD) and their max/min/avg temperature, alongside with the precipitation for each day. The dates are in the first three columns of strings (1900 9 19 for ...
- 11
1
vote
1
answer
628
views
How to fetch fasta sequences if name line matches from another file
I have a file of lines of headers (file 1) and another file of sequences in fasta format (file 2). I want to grep fasta sequences if a header line from file 1 matches file 2.
Example:
File 1:
>...
- 91
1
vote
2
answers
130
views
extract the oldest values from a master file per each code
I need help writing a script that would extract from a master file the oldest records and save them in a separate file. The file contains hundreds of ISIN codes and additional data. For each, there ...
- 19
1
vote
2
answers
88
views
Why Extented Regular expression works on command line input but not when reading from a file
Using Windows 10 developer mode. My regular expression works perfectly when I execute in command prompt. Substitutes mm/dd/yyyy hh:mm to yyyy-mm-dd within a string delimited by comma. It does not ...
- 11
1
vote
0
answers
908
views
Use column to create table with fixed width
Can column create columns of a fixed width? I couldn't find the option in the man page.
$ echo "a b c
d eeeeee f" | column -t
a b c
d eeeeee f
- 2,802
1
vote
1
answer
1k
views
How to compare and replace the value at particular location with awk
I want to compare two csv files with one common column and then replace the data from first file to second file at particular location.
file1.csv:
"id","name","marks","subject","result"
"1","a","","...
- 11
1
vote
2
answers
224
views
print a string from a specific file
Let us suppose that there is a command such as:
# uname -a
Linux (none) 2.6.24 #155 PREEMPT Thu Nov 16 09:41:07 IST 2017 ppc GNU/Linux
Now, my question is: Are there any commands which will only ...
- 11
1
vote
0
answers
3k
views
Capturing the "progress bar" shown in a terminal
I am looking for the "verbose" output of scp, as shown by the following command, but excluding all lines containing debug1::
scp -v /path/to/some_very_large_file user@host:/home/user/test.deleteme 2&...
- 681
1
vote
2
answers
5k
views
Grepping last 5 minutes log
I need to grep last 5 minutes log.
log file:
18-06-17 06:00:09 ID-5
18-06-17 06:00:11 ID-78
20-06-17 09:34:51 ID-Hello
21-06-17 09:20:49 link is down
22-06-17 06:00:11 ID-674
22-06-17 06:40:51 ID-2
...
- 353
1
vote
0
answers
178
views
Text document unreadable after sudden PC shutdown
I was working in a text document with Mousepad, when I accidentally disconnected the power cable, shutting down the computer. When I started my Xubuntu session again, I tried to open the document, but ...
user308164
1
vote
0
answers
677
views
tool for Cyrillic text recognition
Are there any tools to recognize cyrillic text from image.?
it seems that OCR works only with latin letters.
- 141
1
vote
1
answer
49
views
Can selected text be wrapped in quotes/brackets?
Is there any way to configure that when I select text (anywhere in GUI) and then press the " key, the selected text do not get replaced by the quote char but instead gets wrapped in quotes char?
...
- 113
1
vote
2
answers
1k
views
Extracting Text Strings in terminal
I am wondering if there is a way I can download (with wget probably) an html file lets say http://vodlocker.com/embed-wrdlm4dbigu4-850x450.html
Here is the html file in pastebin. Here is the important ...
user483937
1
vote
1
answer
86
views
Replace only certain matches of a string search
Further to my earlier question: Search and replace multiple files including subdirectories I wish to make some adjustment so I do not invalidate the CFML. Basically I am trying to avoid nested <...
- 113
1
vote
1
answer
788
views
Are there NLP tools for text analysis?
Are there any tools on Ubuntu that can be used for Natural Language Processing? I am not necessarily looking for R, python or java scripts (thought they could be valuable), but for command line tools. ...
- 28.7k
0
votes
1
answer
210
views
How to import an mbox file into the Pine or Alpine mail client?
I'm a light Alpine e-mail client user.
If I navigate into ~/mail and run mail -f baz I'll get an expected list of old alpine/pine messages.
Alpine doesn't use mbox.
I added an mbox backup to that ...
- 1,158
0
votes
1
answer
35
views
Join details from two lines based on a match of the first two or three fields
I'm seeking help to extract / merge data from a single csv file.
The file contains the arrival and departure times of ships over a few months. The details of the arrival and departure are on separate ...
- 11
0
votes
0
answers
26
views
Re-arranging a table of values with VIM
I'm often in the situation of having to re-arrange a table of values, from this format:
a = 1 b = 2 c = 3
d = 4 e = 5 f = 6
to this format
a = 1
b = 2
c = 3
.
.
.
in VIM. The fastest way to make it ...
- 181
0
votes
1
answer
348
views
Add a list to specific lines of a (YAML) text file using python
I have an Ansible module file for NetBox with this content:
- name: "Test NetBox modules"
connection: local
hosts: localhost
gather_facts: False
tasks:
- name: Create device ...
- 9
0
votes
0
answers
186
views
Tail in combination with sort
I use tail in a variable to echo contents of multiple files in a folder.
VAR="$(tail -n +1 DIR/*/file)"
This outputs all the files contents with the needed filename on top like
==> /...
- 111
0
votes
2
answers
174
views
Ubuntu Bash - print specific word from a file
I need some help with some bash. I've only learnt like grep, awk, find, sed etc.
Say I have a file named people.txt with different names on a line on multiple lines but a name appears in every line. ...
0
votes
0
answers
36
views
How to deal with large formatted documents quickly in Ubuntu?
I have a .docx file that have a size of 200MB.
To open or save such huge files, MS Office on Windows 10 only need several seconds, but Libre Office on Ubuntu 22.04 take dozens of seconds.
(FYI: iWork ...
- 313
0
votes
2
answers
70
views
sed / awk / etc - Sorting merged text before paste OR paste special: new line
My scenario:
On an internal platform a list of IPs is displayed like:
8.8.8.8 1.0.0.1 1.1.1.1 192.168.1.1 192.168.0.1
When I copy these values and put them (pasted) in a text file they are all pasted, ...
- 173
0
votes
0
answers
398
views
How do I get formatted responses from terminal?
I'm struggling to understand the response that I receive from the terminal as it is all pasted in one line without indentation, line breaks, etc. Is there a way to enable that? Or some command? This ...
0
votes
0
answers
818
views
How to remove all lines containing emojis from a text file?
I have text files in which some lines includes emojis or other non Ascii characters, like the list in this screenshot:
I need a way to remove the whole lines containing an emoji or more. Even the ...
- 19.5k
0
votes
0
answers
96
views
find replace text in file
I have one problem the edit info in file .php
the command I used is
sed -i 's/\TCG\Voyager\Models\User/Authenticatable/' "app/Models/User.php"
not edit segment in my shell script
other ...
0
votes
0
answers
21
views
Sorting textfile with rows of [word number] numerically
I have a .txt file that consists of:
Aa 6
Bb 3
Aa Aa 2
Bb Bb 10
I need to sort it so the result is:
Bb Bb 10
Aa 6
Bb 3
Aa Aa 2
This only gives the alphabetical order:c
cat .txt | sort -n
Can you ...
0
votes
0
answers
45
views
Find specific numbers of capitalized words next to each other
Let's say I have a .txt file:
Aaa
Aaa Bbb
Aaa Bbb Ccc
I want to find the row that has one capitalized word, two capitalized words, and three capitalized words. So one grep to find Aaa, one grep to ...
0
votes
1
answer
75
views
Extract one column of strace -c
I am trying to get the system call names for common commands and don't have list of commands.
Do you have any idea how can I get list of system calls for some commands.
Finally I tried to do such ...
- 11
0
votes
0
answers
203
views
Appending string or character to terminal output
I need to run a command in a terminal to continuously receive incoming messages from elsewhere on the network. I have no control over the formulation of the length of the message. The command, however,...
- 125
0
votes
0
answers
257
views
Simple note taking app on Ubuntu that allows markdown and copy paste of tables in spreadhseet format
(tag suggestions welcome)
I've been exploring note taking apps, including those listed here and here. Preference is a foss app. I installed Simplenote which seems nice but already does more than I ...
- 375
0
votes
1
answer
165
views
How to generate a new log file containing the MAC adress based on pihole.log which didn't contain it by default
I have a /var/log/pihole.log that contains lines like these:
Nov 29 18:49:33 dnsmasq[13568]: query[AAAA] firebaselogging-pa.googleapis.com from 192.168.0.10
Nov 29 18:49:33 dnsmasq[13568]: query[A] ...
0
votes
2
answers
249
views
Remove LF from a specific column using sed
I have a file which contains data separated by |.
Example: (Input)
a|b|c|d`LF`
D|e
a1|b1|c1|d1`LF`
d2|e1\n
Output: (Removed LF from 4th column)
a|b|c|dD|e
a1|b1|c1|d1d2|e1\n
I want to remove LF from ...
- 111
0
votes
0
answers
2k
views
how to make awk extract text in xml with a tag that includes quotation marks
Without quotes, it works fine.
If the line in cust.xml is : \<span style=CustomerId/>999\</CustomerId> <br>
$ awk -F "[><]" '/span style=CustomerId/{print $3}' cust....
- 115