Questions tagged [awk]
awk is a special-purpose, interpreted programming language for extracting and reporting data.
493
questions
0
votes
2
answers
69
views
Creating an html from the output of awk script
I am performing in an awk script a series of aggregations from a dataset and I get the following.
Now I would like to create an html that collects this previous output in the form of a table and that ...
- 3
0
votes
1
answer
38
views
error trying to redirect output of awk script to a new file
I am working on the following code in an awk script and I need the output to be redirected to another file within the same script.
What I try to do in the first block is to round the decimals of the ...
- 3
0
votes
1
answer
14
views
Delete all lines from a line with certain string until another line with a diffrent string in a file
file.txt:
start some text works "HELLO" foo test
sum
other
end
start som "other" line with text 'also'
thing
foo
end
and I want to delete the line in which "HELLO&...
5
votes
3
answers
786
views
How can I capitalise a word proceeding a specific word
Lets say I want to capitalise the first word following the word jumped. Example below
Original text:
the cow jumped over the moon
Amended text
the cow jumped OVER the moon
I have tried awk and sed ...
- 51
1
vote
2
answers
39
views
How to get the average ping time?
Here's a real result of my ping 192.168.1.1 command:
64 bytes from 192.168.1.1: icmp_seq=964 ttl=64 time=1018 ms
64 bytes from 192.168.1.1: icmp_seq=965 ttl=64 time=921 ms
64 bytes from 192.168.1.1: ...
- 257
0
votes
1
answer
38
views
awk warning on escape sequence
I am firing an awk command with the character class ere.
cere='^[[:space:]]*([#;!]+|@c|\/\/)[[:space:]]*'
This gives the awk warning.
awk: warning: escape sequence `\/' treated as plain `/'
Is this ...
- 35
0
votes
6
answers
178
views
delete 50 random rows in a txt files that have their second column labeled as 0
I want to use the terminal commands for deleting 50 random rows from a txt file that is tab separated and has these sample rows where the second column is 0. The value in the second column is either 0 ...
- 3,661
0
votes
2
answers
65
views
How to print specific words from a file
I have a file which is like this
user:[dgalanos] rid:[0xa35]
user:[roleary] rid:[0xa36]
user:[smorgan] rid:[0xa37]
How can i print only the usernames like this
dgalanos
roleary
smorgan
I tried
cat ...
- 11
0
votes
1
answer
501
views
Extract keys from yaml file using awk
Let's say I have a yaml file like this one:
foo:
bar: 1
env:
firstVar: true
secondVar: 20
aa_thirdVar: 'hello'
aa_fourthVar: false
and I need to get the keys under the env key except for ...
- 101
0
votes
0
answers
69
views
Compare two lists, file1 and file2, get the IDs in file2 present in the usernames in file1
Please I need help...
I have two lists, file1 and file2. The list file1 has only usernames, while file2 has the same usernames but with additional columns (username|private yes or no|id).
Basically I ...
- 1
3
votes
1
answer
206
views
Replace a text within single quotes with another string using sed(just a first occurrence of regex)
I have a file with the following content:
$ cat file.txt
CREATE COLLATION public.collation123 (provider = libc, locale = 'fr-FR');
CREATE COLLATION public.collation_1 (provider = libc, locale = 'bas'...
0
votes
0
answers
32
views
Using sed or awk to remove near-duplicates
I currently use the following to get as close as I can do to a file
cut -d ' ' -f 3- /var/log/issues.log | sed -E 's/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}//g' | sort -u
So far it gets rid of ...
- 131
1
vote
1
answer
41
views
How to count the number of consecutive identical lines
I have a file that looks like this:
2000
2000
2001
2001
2001
2001
2002
2002
I need a script to show me this:
2000 - 2
2001 - 4
2002 - 2
I prefer using sed or awk
- 139
2
votes
1
answer
157
views
using variable in awk
Hello I would like to count number of words with specific lengths. I'm using this command.
awk 'length == 2' mydict.txt | wc -l
this code gives what I want but if try to put variable instead of ...
- 33
0
votes
1
answer
57
views
awk -- duplicating a column and also modifying its text
awk {'printf ("%5s\t%s\n", $0, $NF)'} test_VN.txt
works for duplicating a column in a text file however, how can I add _VN before the .jpeg extension of the duplicated column?
For example, ...
- 3,661
0
votes
0
answers
60
views
How to sort with filter awk?
I need to sort the list of files in the current directory by accessibility.
I created directory, files and changes accesibilty for some of them:
enter image description here
But it should look like:
...
- 1
0
votes
1
answer
61
views
playing with awk command trying to remove first line in input
below is my input file, results keep returning "Breed has Votes votes"
Breed, Votes
Black Lab, 30
Chihuahua, 2
Pug, 1
Corgi, 45
Shar Pei, 21
Shih Tzu, 5
...
0
votes
1
answer
48
views
awk command return not found on Ubuntu desktop but work well on Centos
I create test.sh to read a file with content
item1:return1
item2:return2
...
item10:return10
test.sh content
VAR1=$1
VAR2= `grep "^${VAR1}:" /home/path/file | awk '{print $2;}'`
echo "...
0
votes
0
answers
46
views
How to check whether one number range from one file is the subset of other number range from other file?
I'm trying to find out whether range1 numbers [both columns a and b] are the subset or lying between range2's columns [both columns b and c].
range1
a b
15 20
8 10
37 44
32 ...
- 11
1
vote
1
answer
78
views
Awk command with word as delimitor
How to I use awk to cut based on a word that keeps everything after it :
Example:
File.txt >
XXXXWORDNNNNNN
XXXWORDNNNNNNN
XXWORDNNNNNNNN
XWORDNNNNNNNNN
WORDNNNNNNNNNN
output desired:
File.txt &...
0
votes
0
answers
68
views
Redirecting output from printf to /dev/tty in awk
I have a question on awk.
#!/usr/bin/nawk -f
{
if (NF != 4) {
error("Expected 4 fields");
} else {
print;
}
}
function error ( message ) {
if (FILENAME != &...
- 123
-1
votes
4
answers
165
views
How to cut part of line?
I need to cut line in specific moment, I want to display GPU name, but only name, nothing else.
inxi -Gx | grep Device showing:
Device-2: NVIDIA GK107GLM [Quadro K1100M] vendor: Dell driver: nouveau
...
0
votes
1
answer
94
views
Find and replace the entire line of the source file [closed]
I am working on a batch script which will call the .awk file to execute the find and replace.
batch file script.cmd file -->
@gawk -f "%Modify.awk%" "%Temp.csv%" < "%...
- 1
0
votes
1
answer
77
views
tweak of two files
I am trying to merge two csv files based on 1st column (include both "exact string match" and "partial match") and if doesn't match - add those string but with blank 2nd column ...
- 69
1
vote
3
answers
90
views
Keep header from file1 and add column from file2 to file1
I need to copy a specific column from file2 to my file1, while keeping the header from file1.
File1:
##header
##header
##header
A1 B1 C1
A2 B2 C2
File2:
D1 E1 F1
D2 E2 F2
newfile:
##header
##header
#...
- 21
1
vote
1
answer
195
views
Changes in AWK on Ubuntu 20.04 LTS?
I am working with some huge astronomical databases and I just realized that awk is not working as expected. I'm downloading and cutting some .gz files, but awk restrictions are weird, let me share a ...
2
votes
1
answer
367
views
How to keep original decimal numbers without rounding off
I am trying to compute the sum of the travel distance and it is computed perfectly in the text file. However, the results would be rounded off to 4 decimal places
For instance, whenever I tried to add ...
- 35
1
vote
1
answer
341
views
How to print awk's results with different colors for different fields?
ads2 cls create | awk -F [\"] '/^>>> cluster established .../ { strt=1 } strt!=1 { next } $1 ~ "name" { cnt++;nam[cnt]=$2 } $1 ~ "state" {stat[cnt]=$2;print -e "...
- 119
1
vote
1
answer
932
views
awk - compare two files and print all columns from both files
I want to compare two files
File 1:
evm.TU.PTPU-T1. PF00808
evm.TU.PTP-T1 PF00498
evm.TU.PTPX-T1 PF00250
evm.TU.PAN-T1 PF00817
File 2:
PF00808 CL0012 Histone CBFD_NFYB_HMF ...
- 29
0
votes
3
answers
2k
views
how to split a string from a column using awk
I am a noob in Linux. I have a file like this:
col1 col2 col3
ID1234567-DNA_A01 chr1_10203040_T/C gene 0
ID1234568-DNA_A02 chr1_10203050_T/A gene 0
...
- 3
0
votes
1
answer
86
views
Extract frequency value from last column depending on what letter is there to plot
I have a tab-delimited file that has frequency values in the last column (actually the values are tab-delimited as well). I have mostly two values (e.g., A:0 G:1) but some rows have three values (e.g....
6
votes
2
answers
2k
views
awk if find pattern skip n lines
How to skip N lines when a pattern is found in awk?
awk '{if ($0~/pattern/) skip N lines; else print $0}'
- 181
0
votes
0
answers
268
views
what NF stands for in awk?
I know NF is the last element, as explained here:
NF is a predefined variable whose value is the number of fields in the current record. awk automatically updates the value of NF each time it reads a ...
- 1,009
1
vote
1
answer
411
views
Why GSUB does not work in AWK?
I am trying to do a substitution with AWK by passing two variables that are going to be used in the substitution.
My initial try was:
FILE=a
cat $FILE | awk -v var1="$Linea_SF" -v var2="...
- 296
0
votes
1
answer
80
views
awk -v var=$var /var/ is not working as expected
Portion of file:
<style:style style:name="P15" style:family="paragraph" style:parent-style-name="Table_20_Contents">
<style:paragraph-properties fo:text-...
- 115
0
votes
1
answer
132
views
Pass a variable in awk command
spke="myvaribale"
var="cidr"
val=$azure_spoke2_vnet_cidr
clp="azure_vnets"
I am placing these variable in this command : I dont want to write this myvaribale in ...
1
vote
1
answer
242
views
merge partial duplicate rows in a table
Input is a settings file with multiple rows separated by tab.
h1 h2 h3 x y h4 h5 h6 h7
a b c h4 true
a b c h5 false
a b d h4 false
m n o h6 true
m n o h7 false
p q r h4 true
...
0
votes
1
answer
85
views
awk parameter substitution fails
I am on Ubuntu 20.04 Mate Desktop environment.
I encounter a strange failure when I attempt to pass the date string to regex query in the following line :
date="2020-12-11"
cat /var/log/...
- 4,729
0
votes
4
answers
298
views
Awk/Sed commands find and replace pattern
I have a file with data like the following. xyz can be anything, so I want to replace it.
variable "azure_spoke_gateways" {
default = {
spoke1 = {
name = "xyz"
...
0
votes
1
answer
110
views
awk remove duplicate words
I want to extract installed packages in a specific date to remove them easily.
I can list them in a line with the following command:
cat /var/log/dpkg.log | awk '/ installed / && /2020-11-23 ...
- 4,729
0
votes
1
answer
192
views
AWK to compare columns in file1 to file2
I have a simple problem but have not been able to solve. I want to compare columns 2, 3, 4, 5 and 6 of File1 to the same columns of File2 and if they match then add Column1 of File2 to Column7 of ...
- 1
0
votes
1
answer
24
views
Sed insists on doing partial matches
Ok, consider this example file:
#!/usr/bin/env bash
cat > example_file.txt <<EOL
group, , price
1, S_BRN_, 3.21
1, S_WBS_BRN_, 4.11
1, S_WBS_BRN, 1.22
2, S_BRN_WBS, 9.2
2, F_WBS_BRN_, 2.11
2, ...
- 12.8k
0
votes
2
answers
147
views
awk condition is not working properly
These are the data on which I am applying an awk command:
~$ cat lab4.data
Mike Harrington: (510) 548-1278:250:100:175
Christian Dobbins: (408) 538-2358:155:90:201
Susan Dalsass: (206) 654-6279:250:60:...
0
votes
1
answer
105
views
How to use awk to edit a specifc line set by a variable (awk NR, -i inplace and -v var)
If file is:
this is line one
this is line two
this is line three
I want to do something like:
awk -i inplace -v line_number=$line_number 'NR == line_number {print $1}' file
Lets say line_number=1. ...
- 115
0
votes
2
answers
109
views
Remove LF from a specific column using sed
I have a file which contains data separated by |.
Example: (Input)
a|b|c|d`LF`
D|e
a1|b1|c1|d1`LF`
d2|e1\n
Output: (Removed LF from 4th column)
a|b|c|dD|e
a1|b1|c1|d1d2|e1\n
I want to remove LF from ...
- 111
0
votes
1
answer
44
views
Define fields by content and remove separator inside
I have some comma separated files that unfortunately contain commas inside some of the strings. That makes sorting them non trivial. See previous question.
Regardless of the sorting, I figure it would ...
- 103
0
votes
1
answer
375
views
How to replace every 2nd line sed
How to replace every 2nd line of a text file with another text file list of lines.
Example:
cat filea.txt
I love Pink
I love Blue
I love Dogs
I love MOMO
cat bileb.txt
I hate yello
I hate white
I ...
- 11
0
votes
1
answer
64
views
Editing wc -l src Output
I am new to the Topic and working on a few bash commands.
I am stuck at a task where i Need t ouput my result in a specific Format.
wc -l folder/*
The Output gives me:
0 folder/file1
3 folder/file2
3 ...
- 1
2
votes
2
answers
3k
views
Convert HTML to text format
I want to extract the page content from this HTML file:
<BR />
<TABLE style=border-color:#32506d border=1 cellspacing=0>
<caption class=header style=background-color:#32506d><...
0
votes
2
answers
491
views
awk NR variable not working as expected, getting the whole line while asking for first field
echo cat:
cat Records.txt
echo ""
echo Using a digit for the second record:
record_id=$(awk 'NR==2{print $1; exit}' Records.txt)
echo $record_id
echo ""
a=2
echo a is set to $a
...
- 115