Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Make your voice heard. Take the 2019 Developer Survey now

Questions tagged [text-processing]

Questions about modifying, creating or parsing plain text files. This is a very broad tag, if your question is about using a specific tool, please tag it with the tool's name as well (e.g. sed, awk, perl, etc.)

0
votes
1answer
36 views

Replace spaces with hyphens in a text file

I have a text file as follows: 2018 11 20 2018 11 21 2018 11 22 2018 11 23 2018 11 24 2018 11 25 2018 11 26 2018 11 27 This file consists of year,month and day with three columns. I need to add ...
2
votes
1answer
78 views

Search for text in files of same name in multiple folders of various depths

Situation: In Linux, I have a parent folder with almost 100 folders of various names. Each folder has a file ResourceParent.xml and hundreds of of different version numbers each of which has its own ...
2
votes
3answers
98 views

Move specific column from csv file in front (select column by name)

Consider this data: #!/usr/bin/env bash cat > example_file.txt <<EOL group, value, price 1, 3.21, 3.21 1, 3.42, 4.11 1, 3.5, 1.22 2, 4.1, 9.2 2, 4.2, 2.11 EOL I want to move the 'value' ...
3
votes
2answers
269 views

extracting numbers whose lengths are fixed from text file

I have a text file. This file includes characters and numbers as follows: ANKR00TUR_R_20183240000_01D_30S_MO.rnx: 2018 11 20 00 00 0.0000000 GPS TIME OF FIRST OBS brmu3350....
0
votes
2answers
52 views

creating text file with regards to file names

I have a text file (digit.txt) consisting of one column includes three digit numbers whose range are 001-...365. In the same directory, I have data text files. The file name of these data files as ...
3
votes
1answer
202 views

Duplicate rows in text file

I need to duplicate rows in text file with a specific number of times. For example my data file is: jplg3350.18i jplg3360.18i jplg3370.18i I need to duplicate the lines three times as follows; ...
6
votes
2answers
313 views

How can I extract records with a column in common?

I have two tables with different numbers of columns and rows. I want to find rows using a common column (column B is in common). Here is an example. Could you please help? file1.txt A B C D a ...
6
votes
2answers
86 views

Sed script crashing on big file

I have a shell script which is in essence a sed script with some checks. The goal of the script is to convert the header of a file from. &FCI NORB=280, NELEC=78, MS2=0, UHF=.FALSE., ORBSYM=1,1,1,...
0
votes
2answers
145 views

use sed to replace each white space with a backslash

just want to escape spaces in windows filepath. I'm trying this echo "111 1111 "| sed -e "s/[[:space:]]/\\ /g" that only match spaces but do not replace.
0
votes
1answer
29 views

Changing text on a file using scripts [duplicate]

I want to change the PASS_MIN_DAYS = (whatever value is set) to PASS_MIN_DAYS = 15 using a script.The file it's in is /etc/login.defs.
1
vote
2answers
54 views

Sort files by unpadded leading numbers

I have the following files $ ls *.md | sort -n 1.pythonPrimer.md 10.mapsHashTablesAndSkipLists.md 11.searchTree.md 12.sortAndSelection.md 13.textProcessing.md 14.graphAlgorithms.md 15....
0
votes
0answers
45 views

How to print column number of last character of line in file

I want to print the column number of last character of line. I have a file. So I want to print the column number of last character of line which has highest number in file. Thanks.
2
votes
3answers
237 views

changing individual letter position with bash

If I have a file called text.txt and it contains one line of text, ijbiaubiu is it possible to move each letter one space forward or back in the alphabet in order to encrypt it manually, with what I ...
1
vote
2answers
59 views

sed add line (without spaces at the beginning ) after match (script using variables )

Please help with spaces when script add line after searched pattern $ cat test.txt test server:1 prod server 1 $ cat ./check.sh #!/bin/bash # search=" test server:1" add=" ...
1
vote
1answer
31 views

Extracting text based on partial ID

I have a fastq file. Each entry in the file has four lines as following @0e249669-03e7-47e0-9bc2-a9bcaa35e198 runid=435beb2485d20d8e06f3b38ed907a08f8147fa77 read=100 ch=83 start_time=2018-10-27T12:...
1
vote
3answers
121 views

extract substring (filename) with bash/grep/awk/etc

I have a console output process string like this: plex 24029 33.3 3.4 330284 63656 ? Sl 22:37 6:01 /volume1/@appstore/Plex Media Server/Plex Transcoder -codec:0 mpeg2video -codec:1 ...
3
votes
3answers
377 views

How to merge two files as alternative lines?

I know how to merge two tables to print alternative lines in a new file, but I want to merge every two lines from file1.txt with one line from file2.txt. As an example: file1.txt is A a aa B b bb C ...
0
votes
1answer
62 views

Why isn't grep working with iostat?

I'm trying to collect some disk usage statistics as specified here, for using with this nagios plugin In short it should get the tps for the second lecture in the output comming from iostat like ...
0
votes
1answer
40 views

Text file does not open

Scanning via sudo date >> clam && clamscan -r -i --detect-pua --remove /home >> clam && date >> clam Returns a file that does not open Error message: ...
2
votes
2answers
99 views

Find Missing Numbers [closed]

I have a big list of ordered files with name like this (videos) S1-E18-(Date)-(Title)-(Random numbers).mp4 Here is the example of list S1-E1-20100526-title-of-video-1400316375.mp4 S1-E3-20100547-...
0
votes
2answers
75 views

Can I run two different awk commands in 1

I can use this awk command to get last line of log file: awk 'END{print}' /home/jj1/.pia_manager/log/pia_nw.log And I can use this awk command to extract a substring: awk -F"[()]" '{print $2}' /...
0
votes
1answer
73 views

Convert output of shell script to .csv

I want to convert the output of the below patchhistory.sh shell script to csv, so that I can get the information in column format like this: servername patch history ...
5
votes
2answers
326 views

How do I save some lines but not all?

In Terminal how do I save lines 3, 4 and 5 from an existing file to a new file in a single command?
0
votes
1answer
45 views

Help with adapting a script to Multiple Outputs

This script pulls out the matching "Group" Sports from my file to the output emby_sports.m3u. I have all the usual groups I.E. "News", "General Streams", "Movies". Is it possible to adapt this ...
0
votes
1answer
36 views

sed new lines with \n as part of the line text

trying to write a new sed command that will insert new lines before another line. the code i have tried and is failing is: sudo sed -i '/# onlynet=ipv4/i echo "rpcallowip='$WebInternalIP'\n";\necho ...
0
votes
1answer
55 views

How can I use cron to edit a line in a file?

I'm using Ubuntu terminal and I want to tell this to my crontab; Go to test.cfg file, open it, replace line 35 with x. Is this possible? I have looked around but couldn't find anything.
1
vote
2answers
64 views

Why Extented Regular expression works on command line input but not when reading from a file

Using Windows 10 developer mode. My regular expression works perfectly when I execute in command prompt. Substitutes mm/dd/yyyy hh:mm to yyyy-mm-dd within a string delimited by comma. It does not ...
0
votes
1answer
42 views

Constructing an address book with Bash - editing the fields with sed produces the wrong result

I want to use Bash to make an address book but I'm having problems editing fields. The fields are name;number;mail. This is my code. I want to edit for example the name, or the name and the number,...
1
vote
2answers
152 views

How can I replace three columns 'XXX' with the number?

I have a .pdb file that looks like this: ATOM 1 N PRO 137 -17.579 -25.693 -70.648 ATOM 2 CA PRO 137 -18.871 -26.246 -70.218 ATOM 3 C PRO 137 -19.596 -25.304 -69....
1
vote
2answers
58 views

Merge expressions with a one-to-one join

I have two text tables as below: file1.txt 1 A a 2 B b 3 C c 4 D d file2.txt A x i C z iii D w iiii B y ii I want to have final.txt 1 A a x i 2 B b y ii 3 C c z iii 4 D d w iiii
0
votes
2answers
104 views

How can I paste a specific string into my text by a shortcut key?

I want to use keyboard combination Alt-E to write my email address, rather than have to type it out all the time. I know how to create a custom key in unity-control-center Keyboard Shortcuts Custom ...
1
vote
2answers
61 views

How can I decode a file where each letter has been replaced with the letter 13 letters ahead of it in the alphabet?

There's a text affected by ROT13, called rot.txt. ROT13 (rotate by 13 places) replaces a letter with the letter 13 letters after it in the alphabet. How do I write the command to view the actual text? ...
1
vote
1answer
55 views

How can I use the grep --color command to highlight brackets?

I want to highlight every [ in any color so i can see the hidden message here: [###[#[[[#[###[#[[[#[[[#[###[#[[[ [###[##[##[[##[#[####[##[[##[##[# [######[##[#[#[#[[[##[##[#[#[##[# [######[##[##[[#[#...
0
votes
2answers
38 views

Script that should add 1 to a variable adds 2 instead

I need help with a simple script but what I got adds 2 to the Variable instead of 1. Can I get some help so it's only 1? var=1 while read -r LINE || [ -n "$LINE" ]; do LINE=`echo $LINE | sed -e "s/...
1
vote
1answer
61 views

How can I replace [0] with ascending numbers in a text file?

I want to change a file with text like this Application[0]=test_app1 Application[0]=test_aap2 Application[0]=test_app3 Application[0]=test_app4 .. to have ascending numbers like this: Application[0]...
0
votes
1answer
57 views

Capturing fdisk output into an array and printing it

I want to get a list of partitions on the system... I use this command sudo fdisk -l | grep '^/dev' | awk '{print $1}' The output is /dev/sda1 /dev/sda2 I need to get this result as first ...
0
votes
2answers
33 views

How to remove first X lines from file up to the first occurrence of particular string?

I have many PostgreSQL dump files such this and because I import these files automatically into another database server (H2) and SET command is not compatible with this database, I need to remove all ...
2
votes
1answer
219 views

How to add Cyrillic in Ubuntu 18.04?

I have already installed Russian keyboard layout. In language support -> installed languages I already have Russian. But when I open text file with Cyrillic I have encoding problems. This files was ...
-1
votes
1answer
424 views

merge two files with two common fields

File1 #CHROM POS ID REF_Zv ALT_lm chr1A 219620 . T A chr1A 219648 . A G chr1A 219867 . A G file2 #CHROM POS ...
-1
votes
1answer
98 views

re.search to get decimal number [closed]

I am trying to get a number from a text file. I need the full decimal number. The line looks like: corner_lat: 49.0425000 decimal degrees I'm trying this if "corner_lat" in line: ...
-1
votes
2answers
58 views

grep and sed only the numbers from a text file's line

I'd like to extract only a number from a text file. It is like: corner_lat: 49.0425000 decimal degrees I am using the following expression : grep corner_lat EQA.dem_par | sed "s,[^0-9]*,," ...
0
votes
0answers
28 views

How to recursively go through all text files in a directory to fetch the desired line, and write these lines to same text file?

I have a directory which has text files (HTML pages). I wish to use SED to go through these text files, find the specific line, and write the line to a single text file. So for example I have HTMLS ...
5
votes
5answers
365 views

Capture and compile a list of names from a log file

I need a one-line command to compile and print all of the Expendable Launch Vehicle names listed in a log file. The ELV names are all listed in capital letters under the /elv directory. The output ...
3
votes
3answers
246 views

Compare two files line by line and show only difference from file 2

I want to compare two files line by line without sorting and show only difference from file2. file 1.txt: one two three four five six seven eight nine ten file 2.txt: one five three four five ...
1
vote
2answers
69 views

I want to extract link from a HTML page to download my images, and I have several thousand of these HTML files. How do I go about this?

so I have HTML files which have a specific portion I'd like to extract. These HTML addresses are in a text file. A sample HTML webpage, taken from this textfile would look like this, and I'd want to ...
1
vote
1answer
42 views

delete specified lines in files under directories

I have a directory, and it has many sub-directories. home/folderA/* home/folderB/* What I want to do is to delete all lines that have certain text 'password' in files in each sub-directory. ...
0
votes
1answer
67 views

change string/value in 3rd column based on value change in 6th column - awk

I am trying to figure out how to use awk to force the change of consecutive numbering in 3rd column of my file based on the value change in my 6th column, but preserving the spacing?? Can somebody ...
3
votes
2answers
53 views

Extract string from `pactl` output and pass as variable to shell script

I am trying to write a script that allows me to change Bluetooth speaker volume from the shell. I can hard code the value of the sink and it works just fine: #!/usr/bin/env bash for level in "$@" do ...
0
votes
1answer
54 views

How to print next characters after a pattern search in Unix?

I have bunch of files in a folder, need to search with the keywords 'PO' and SUPP to print the next chars in a output file. Please help. file 1: PO 0962690 SUPP 212055 20180820000000 8709384 ...
0
votes
0answers
25 views

Use column to create table with fixed width

Can column create columns of a fixed width? I couldn't find the option in the man page. $ echo "a b c d eeeeee f" | column -t a b c d eeeeee f