Questions tagged [awk]
awk is a special-purpose, interpreted programming language for extracting and reporting data.
511
questions
0
votes
1
answer
112
views
What does this command mean: `echo $line | awk -F "\t" '{print $1;}'`?
I'm new to this, and I was wondering specifically what awk -F "\t" means.
- 1
5
votes
3
answers
461
views
how can I get unique ip addresses from /var/log/auth.log?
How do I only show the unique IP addresses in /var/log/auth.log ?
I have a server getting hit constantly, and while I have set up fail2ban, the requests keep coming. So I started writing to ISPs to ...
- 14.6k
0
votes
1
answer
44
views
find a number after a special character and multiply it by awk
I have a file:
"UDP", "x", 12880, "any", "any", x, 0.0.0.0/0, 0.0.0.0/0, {85000, drop, -1, fast+dsample:1000}, {1m, alert, -1, none}
How can I change 85000 ...
1
vote
1
answer
74
views
Check a host file for modification and if missing add it
I'm using docker in which the container has its own /etc/hosts file. It gets cleared out when rebooting. I'd like to create a script to check for an entry and if missing add it.
So if /etc/hosts has ...
- 13
0
votes
2
answers
184
views
Append to a variable in awk
When using bash I can append to a variable with +=. I want to
do something similar in awk. What can I use ?
rl="?(+)*([$dgt])?([.,])+([$dgt])"
rl+="?([eE]?([-+])+([$dgt]))"
- 361
1
vote
1
answer
123
views
Conditionals in awk
In bash I can do the shortcut condition for an if statement
(( globrl == 1 )) && var=val
Is there something equivalent in awk, without having to use a direct if condition ?
- 361
0
votes
0
answers
102
views
Skip line at match
I am printing a multiline string mstr with colour codes defined in array tseq[knam], with knam being a colour identifier (e.g. Blu:, Grn:, etc).
For instance, with the multiline string mstr,
mstr= &...
- 361
-2
votes
1
answer
50
views
gsub not accepting fourth argument in gawk
Have been using gsub when using gawk, but is not accepting the 4th argument. How can I fix this so that I get a string without blanks ?
for ( knam in tseq ) {
## Remove any blanks from array ...
- 361
0
votes
1
answer
30
views
Appending a pattern to a preexisting regex
I have this regex that allows a space followed by visible characters
[[:space:]][[:graph:]]+$
How can I construct the possibilities of having either some pattern ptrn or the same pattern ptrn ...
- 361
1
vote
3
answers
184
views
Couldn't able to store a output of awk command
I have been trying to print git clone progress on more minimalistic way for my project.
Aim
Instead of printing a whole git clone output on screen
remote: Enumerating objects: 1845678, done.
...
- 13
1
vote
2
answers
958
views
awk or sed command to replace line break plus text containing spaces
An answer to another question suggests sed -i 's/original/replacement/g' file.txt to replace specific words in a text file. My starting situation looks like this:
Item: PRF
Type: File
...
- 331
0
votes
1
answer
196
views
Using awk to parse for conditions over multiple lines? (ifconfig output)
I'm trying to parse ifconfig output with awk.
Right now my script is:
# Setup
ifconfig > ip.txt
ans=$(cat ip.txt | awk '$1 == "inet" {print $2}')
#output
echo "Here are your IPs:&...
- 1
0
votes
3
answers
603
views
How to select several words, and remove all lines that contain those words?
See screenshot. I want to get rid of the clutter, and leave only things like ID, and TEXT, but remove the other things, such as type, date, date_unixtime, edited, from, etc.
I tried the following ...
user1613265
0
votes
2
answers
44
views
I can't change the value between two quotation marks with sed
How can I replace the word at the very end of the first line using sed?
test00 ; test01 ; 'test02 -test03="j32h"'
test04
The result should look like below;
test00 ; test01 ; 'test02 -test03=...
- 3
0
votes
0
answers
92
views
How do I underline formatted output in awk?
I am new to awk and having a hard time figuring out how to format output with underlines. It doesn't appear to be possible to use formatting escape sequences in awk e.g. \e[4m?
Is there a way to ...
2
votes
1
answer
177
views
Awk (mawk) floors decimal numbers to zero
It seems to me like a bug in awk (mawk 1.3.4.20200120-2) on Ubuntu 20.04. I can display required number of decimals with printf (%.*n*f), but the decimals are always rounded to zero, as if there was ...
- 61
0
votes
1
answer
84
views
echo | awk '{printf "%5s\n", "howardstone"}' What "%5s" specifically does in code?
echo | awk '{printf "%5s\n", "howardstone"}'
What does "%5s" specifically do in this code?
- 9
7
votes
1
answer
769
views
awk works in different ways on desktop and server
Just a simple question.
Why this bash command works properly on Ubuntu Server and finds out the ISBN code
and doesn't work on my Ubuntu Desktop? Both systems are 22.04 LTS.
echo "ISBN 5-02-013850-...
- 333
0
votes
1
answer
2k
views
awk: line 1: syntax error at or near ,
I try to use the awk command on my file in dpkg and I get that error:
awk '{print $1,$6}' cmd.txt > cmd2.txt
File content:
Port Protocol Type Board Name FQBN ...
-1
votes
1
answer
7k
views
dpkg: error processing package php8.1-common
I know that my error is a very classic one, and I encountered it without doing it on purpose and i need to resolve the problem because i'm working with php.
❯ php
zsh: command not found: php
and:
❯ ...
0
votes
2
answers
657
views
Creating an html from the output of awk script
I am performing in an awk script a series of aggregations from a dataset and I get the following.
Now I would like to create an html that collects this previous output in the form of a table and that ...
- 7
0
votes
1
answer
385
views
error trying to redirect output of awk script to a new file
I am working on the following code in an awk script and I need the output to be redirected to another file within the same script.
What I try to do in the first block is to round the decimals of the ...
- 7
0
votes
1
answer
555
views
Delete all lines from a line with certain string until another line with a diffrent string in a file
file.txt:
start some text works "HELLO" foo test
sum
other
end
start som "other" line with text 'also'
thing
foo
end
and I want to delete the line in which "HELLO&...
- 115
5
votes
3
answers
920
views
How can I capitalise a word proceeding a specific word
Lets say I want to capitalise the first word following the word jumped. Example below
Original text:
the cow jumped over the moon
Amended text
the cow jumped OVER the moon
I have tried awk and sed ...
- 51
1
vote
2
answers
375
views
How to get the average ping time?
Here's a real result of my ping 192.168.1.1 command:
64 bytes from 192.168.1.1: icmp_seq=964 ttl=64 time=1018 ms
64 bytes from 192.168.1.1: icmp_seq=965 ttl=64 time=921 ms
64 bytes from 192.168.1.1: ...
- 673
1
vote
1
answer
2k
views
awk warning on escape sequence
I am firing an awk command with the character class ere.
cere='^[[:space:]]*([#;!]+|@c|\/\/)[[:space:]]*'
This gives the awk warning.
awk: warning: escape sequence `\/' treated as plain `/'
Is this ...
- 361
0
votes
6
answers
328
views
delete 50 random rows in a txt files that have their second column labeled as 0
I want to use the terminal commands for deleting 50 random rows from a txt file that is tab separated and has these sample rows where the second column is 0. The value in the second column is either 0 ...
- 4,349
0
votes
2
answers
1k
views
How to print specific words from a file
I have a file which is like this
user:[dgalanos] rid:[0xa35]
user:[roleary] rid:[0xa36]
user:[smorgan] rid:[0xa37]
How can i print only the usernames like this
dgalanos
roleary
smorgan
I tried
cat ...
- 21
0
votes
1
answer
2k
views
Extract keys from yaml file using awk
Let's say I have a yaml file like this one:
foo:
bar: 1
env:
firstVar: true
secondVar: 20
aa_thirdVar: 'hello'
aa_fourthVar: false
and I need to get the keys under the env key except for ...
- 101
3
votes
1
answer
738
views
Replace a text within single quotes with another string using sed(just a first occurrence of regex)
I have a file with the following content:
$ cat file.txt
CREATE COLLATION public.collation123 (provider = libc, locale = 'fr-FR');
CREATE COLLATION public.collation_1 (provider = libc, locale = 'bas'...
- 31
0
votes
0
answers
69
views
Using sed or awk to remove near-duplicates
I currently use the following to get as close as I can do to a file
cut -d ' ' -f 3- /var/log/issues.log | sed -E 's/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}//g' | sort -u
So far it gets rid of ...
- 131
1
vote
1
answer
222
views
How to count the number of consecutive identical lines
I have a file that looks like this:
2000
2000
2001
2001
2001
2001
2002
2002
I need a script to show me this:
2000 - 2
2001 - 4
2002 - 2
I prefer using sed or awk
- 139
2
votes
1
answer
517
views
using variable in awk
Hello I would like to count number of words with specific lengths. I'm using this command.
awk 'length == 2' mydict.txt | wc -l
this code gives what I want but if try to put variable instead of ...
- 33
0
votes
1
answer
195
views
awk -- duplicating a column and also modifying its text
awk {'printf ("%5s\t%s\n", $0, $NF)'} test_VN.txt
works for duplicating a column in a text file however, how can I add _VN before the .jpeg extension of the duplicated column?
For example, ...
- 4,349
0
votes
1
answer
533
views
playing with awk command trying to remove first line in input
below is my input file, results keep returning "Breed has Votes votes"
Breed, Votes
Black Lab, 30
Chihuahua, 2
Pug, 1
Corgi, 45
Shar Pei, 21
Shih Tzu, 5
...
0
votes
1
answer
184
views
awk command return not found on Ubuntu desktop but work well on Centos
I create test.sh to read a file with content
item1:return1
item2:return2
...
item10:return10
test.sh content
VAR1=$1
VAR2= `grep "^${VAR1}:" /home/path/file | awk '{print $2;}'`
echo "...
0
votes
0
answers
62
views
How to check whether one number range from one file is the subset of other number range from other file?
I'm trying to find out whether range1 numbers [both columns a and b] are the subset or lying between range2's columns [both columns b and c].
range1
a b
15 20
8 10
37 44
32 ...
- 111
1
vote
1
answer
1k
views
Awk command with word as delimitor
How to I use awk to cut based on a word that keeps everything after it :
Example:
File.txt >
XXXXWORDNNNNNN
XXXWORDNNNNNNN
XXWORDNNNNNNNN
XWORDNNNNNNNNN
WORDNNNNNNNNNN
output desired:
File.txt &...
0
votes
0
answers
178
views
Redirecting output from printf to /dev/tty in awk
I have a question on awk.
#!/usr/bin/nawk -f
{
if (NF != 4) {
error("Expected 4 fields");
} else {
print;
}
}
function error ( message ) {
if (FILENAME != &...
- 123
-1
votes
4
answers
1k
views
How to cut part of line?
I need to cut line in specific moment, I want to display GPU name, but only name, nothing else.
inxi -Gx | grep Device showing:
Device-2: NVIDIA GK107GLM [Quadro K1100M] vendor: Dell driver: nouveau
...
0
votes
1
answer
303
views
Find and replace the entire line of the source file [closed]
I am working on a batch script which will call the .awk file to execute the find and replace.
batch file script.cmd file -->
@gawk -f "%Modify.awk%" "%Temp.csv%" < "%...
- 1
0
votes
1
answer
79
views
tweak of two files
I am trying to merge two csv files based on 1st column (include both "exact string match" and "partial match") and if doesn't match - add those string but with blank 2nd column ...
- 69
1
vote
3
answers
93
views
Keep header from file1 and add column from file2 to file1
I need to copy a specific column from file2 to my file1, while keeping the header from file1.
File1:
##header
##header
##header
A1 B1 C1
A2 B2 C2
File2:
D1 E1 F1
D2 E2 F2
newfile:
##header
##header
#...
- 21
1
vote
1
answer
452
views
Changes in AWK on Ubuntu 20.04 LTS?
I am working with some huge astronomical databases and I just realized that awk is not working as expected. I'm downloading and cutting some .gz files, but awk restrictions are weird, let me share a ...
2
votes
1
answer
630
views
How to keep original decimal numbers without rounding off
I am trying to compute the sum of the travel distance and it is computed perfectly in the text file. However, the results would be rounded off to 4 decimal places
For instance, whenever I tried to add ...
- 35
1
vote
1
answer
1k
views
How to print awk's results with different colors for different fields?
ads2 cls create | awk -F [\"] '/^>>> cluster established .../ { strt=1 } strt!=1 { next } $1 ~ "name" { cnt++;nam[cnt]=$2 } $1 ~ "state" {stat[cnt]=$2;print -e "...
- 129
1
vote
1
answer
2k
views
awk - compare two files and print all columns from both files
I want to compare two files
File 1:
evm.TU.PTPU-T1. PF00808
evm.TU.PTP-T1 PF00498
evm.TU.PTPX-T1 PF00250
evm.TU.PAN-T1 PF00817
File 2:
PF00808 CL0012 Histone CBFD_NFYB_HMF ...
- 29
0
votes
3
answers
6k
views
how to split a string from a column using awk
I am a noob in Linux. I have a file like this:
col1 col2 col3
ID1234567-DNA_A01 chr1_10203040_T/C gene 0
ID1234568-DNA_A02 chr1_10203050_T/A gene 0
...
- 3
0
votes
1
answer
93
views
Extract frequency value from last column depending on what letter is there to plot
I have a tab-delimited file that has frequency values in the last column (actually the values are tab-delimited as well). I have mostly two values (e.g., A:0 G:1) but some rows have three values (e.g....
6
votes
2
answers
4k
views
awk if find pattern skip n lines
How to skip N lines when a pattern is found in awk?
awk '{if ($0~/pattern/) skip N lines; else print $0}'
- 181