Questions tagged [awk]
awk is a special-purpose, interpreted programming language for extracting and reporting data.
513
questions
3
votes
3
answers
887
views
Remove spaces from the 3rd line onwards in a file on linux
I need to remove all spaces from the 3rd line onwards. The spaces in the first two lines should not be removed. Any option to exclude the first two lines in sed or awk?
Input file:
This is the first ...
- 51
3
votes
2
answers
467
views
How to apply AWK to a directory path?
I have this command:
Folder=./GAS/NNN_/Neutral
awk -F / '{print $(NF-1)}' $Folder
But I got: awk: read error (Is a directory)
I have tried:
awk -F / '{print $(NF-1)}' $(Folder)
awk -F / '{print $(NF-...
- 352
0
votes
1
answer
172
views
AWK issue on Bash script
I need to shortlist the logs between the particular timestamps.
I can achieve it by using the below command.
awk '$a>="Feb 5 10:44:*" && $a<="Feb 5 10:54*"' /var/...
-1
votes
1
answer
525
views
What does this command mean: `echo $line | awk -F "\t" '{print $1;}'`?
I'm new to this, and I was wondering specifically what awk -F "\t" means.
- 1
5
votes
3
answers
2k
views
how can I get unique ip addresses from /var/log/auth.log?
How do I only show the unique IP addresses in /var/log/auth.log ?
I have a server getting hit constantly, and while I have set up fail2ban, the requests keep coming. So I started writing to ISPs to ...
- 15k
0
votes
1
answer
65
views
find a number after a special character and multiply it by awk
I have a file:
"UDP", "x", 12880, "any", "any", x, 0.0.0.0/0, 0.0.0.0/0, {85000, drop, -1, fast+dsample:1000}, {1m, alert, -1, none}
How can I change 85000 ...
1
vote
1
answer
107
views
Check a host file for modification and if missing add it
I'm using docker in which the container has its own /etc/hosts file. It gets cleared out when rebooting. I'd like to create a script to check for an entry and if missing add it.
So if /etc/hosts has ...
- 13
1
vote
2
answers
750
views
Append to a variable in awk
When using bash I can append to a variable with +=. I want to
do something similar in awk. What can I use ?
rl="?(+)*([$dgt])?([.,])+([$dgt])"
rl+="?([eE]?([-+])+([$dgt]))"
- 1
1
vote
1
answer
1k
views
Conditionals in awk
In bash I can do the shortcut condition for an if statement
(( globrl == 1 )) && var=val
Is there something equivalent in awk, without having to use a direct if condition ?
- 1
0
votes
0
answers
139
views
Skip line at match
I am printing a multiline string mstr with colour codes defined in array tseq[knam], with knam being a colour identifier (e.g. Blu:, Grn:, etc).
For instance, with the multiline string mstr,
mstr= &...
- 1
-2
votes
1
answer
93
views
gsub not accepting fourth argument in gawk
Have been using gsub when using gawk, but is not accepting the 4th argument. How can I fix this so that I get a string without blanks ?
for ( knam in tseq ) {
## Remove any blanks from array ...
- 1
0
votes
1
answer
32
views
Appending a pattern to a preexisting regex
I have this regex that allows a space followed by visible characters
[[:space:]][[:graph:]]+$
How can I construct the possibilities of having either some pattern ptrn or the same pattern ptrn ...
- 1
1
vote
3
answers
258
views
unable to store output of awk command
I am trying to print git clone progress in a minimalistic way for my project.
Aim
Instead of printing a whole git clone output on screen
remote: Enumerating objects: 1845678, done.
remote: ...
- 13
1
vote
3
answers
2k
views
awk or sed command to replace line break plus text containing spaces
An answer to another question suggests sed -i 's/original/replacement/g' file.txt to replace specific words in a text file. My starting situation looks like this:
Item: PRF
Type: File
...
- 350
0
votes
1
answer
314
views
Using awk to parse for conditions over multiple lines? (ifconfig output)
I'm trying to parse ifconfig output with awk.
Right now my script is:
# Setup
ifconfig > ip.txt
ans=$(cat ip.txt | awk '$1 == "inet" {print $2}')
#output
echo "Here are your IPs:&...
- 1
0
votes
3
answers
947
views
How to select several words, and remove all lines that contain those words?
See screenshot. I want to get rid of the clutter, and leave only things like ID, and TEXT, but remove the other things, such as type, date, date_unixtime, edited, from, etc.
I tried the following ...
user1613265
0
votes
2
answers
48
views
I can't change the value between two quotation marks with sed
How can I replace the word at the very end of the first line using sed?
test00 ; test01 ; 'test02 -test03="j32h"'
test04
The result should look like below;
test00 ; test01 ; 'test02 -test03=...
- 3
0
votes
0
answers
141
views
How do I underline formatted output in awk?
I am new to awk and having a hard time figuring out how to format output with underlines. It doesn't appear to be possible to use formatting escape sequences in awk e.g. \e[4m?
Is there a way to ...
2
votes
1
answer
295
views
Awk (mawk) floors decimal numbers to zero
It seems to me like a bug in awk (mawk 1.3.4.20200120-2) on Ubuntu 20.04. I can display required number of decimals with printf (%.*n*f), but the decimals are always rounded to zero, as if there was ...
- 61
0
votes
1
answer
111
views
echo | awk '{printf "%5s\n", "howardstone"}' What "%5s" specifically does in code?
echo | awk '{printf "%5s\n", "howardstone"}'
What does "%5s" specifically do in this code?
- 9
8
votes
1
answer
1k
views
awk works in different ways on desktop and server
Just a simple question.
Why this bash command works properly on Ubuntu Server and finds out the ISBN code
and doesn't work on my Ubuntu Desktop? Both systems are 22.04 LTS.
echo "ISBN 5-02-013850-...
- 353
0
votes
1
answer
2k
views
awk: line 1: syntax error at or near ,
I try to use the awk command on my file in dpkg and I get that error:
awk '{print $1,$6}' cmd.txt > cmd2.txt
File content:
Port Protocol Type Board Name FQBN ...
0
votes
1
answer
10k
views
dpkg: error processing package php8.1-common
I know that my error is a very classic one, and I encountered it without doing it on purpose and i need to resolve the problem because i'm working with php.
❯ php
zsh: command not found: php
and:
❯ ...
- 11
0
votes
2
answers
955
views
Creating an html from the output of awk script
I am performing in an awk script a series of aggregations from a dataset and I get the following.
Now I would like to create an html that collects this previous output in the form of a table and that ...
- 7
0
votes
1
answer
620
views
error trying to redirect output of awk script to a new file
I am working on the following code in an awk script and I need the output to be redirected to another file within the same script.
What I try to do in the first block is to round the decimals of the ...
- 7
1
vote
1
answer
1k
views
Delete all lines from a line with certain string until another line with a diffrent string in a file
file.txt:
start some text works "HELLO" foo test
sum
other
end
start som "other" line with text 'also'
thing
foo
end
and I want to delete the line in which "HELLO&...
- 125
5
votes
3
answers
976
views
How can I capitalise a word proceeding a specific word
Lets say I want to capitalise the first word following the word jumped. Example below
Original text:
the cow jumped over the moon
Amended text
the cow jumped OVER the moon
I have tried awk and sed ...
- 51
1
vote
2
answers
730
views
How to get the average ping time?
Here's a real result of my ping 192.168.1.1 command:
64 bytes from 192.168.1.1: icmp_seq=964 ttl=64 time=1018 ms
64 bytes from 192.168.1.1: icmp_seq=965 ttl=64 time=921 ms
64 bytes from 192.168.1.1: ...
- 885
1
vote
1
answer
3k
views
awk warning on escape sequence
I am firing an awk command with the character class ere.
cere='^[[:space:]]*([#;!]+|@c|\/\/)[[:space:]]*'
This gives the awk warning.
awk: warning: escape sequence `\/' treated as plain `/'
Is this ...
- 1
0
votes
6
answers
374
views
delete 50 random rows in a txt files that have their second column labeled as 0
I want to use the terminal commands for deleting 50 random rows from a txt file that is tab separated and has these sample rows where the second column is 0. The value in the second column is either 0 ...
- 4,635
0
votes
2
answers
2k
views
How to print specific words from a file
I have a file which is like this
user:[dgalanos] rid:[0xa35]
user:[roleary] rid:[0xa36]
user:[smorgan] rid:[0xa37]
How can i print only the usernames like this
dgalanos
roleary
smorgan
I tried
cat ...
- 21
0
votes
1
answer
3k
views
Extract keys from yaml file using awk
Let's say I have a yaml file like this one:
foo:
bar: 1
env:
firstVar: true
secondVar: 20
aa_thirdVar: 'hello'
aa_fourthVar: false
and I need to get the keys under the env key except for ...
- 101
3
votes
1
answer
845
views
Replace a text within single quotes with another string using sed(just a first occurrence of regex)
I have a file with the following content:
$ cat file.txt
CREATE COLLATION public.collation123 (provider = libc, locale = 'fr-FR');
CREATE COLLATION public.collation_1 (provider = libc, locale = 'bas'...
- 31
0
votes
0
answers
90
views
Using sed or awk to remove near-duplicates
I currently use the following to get as close as I can do to a file
cut -d ' ' -f 3- /var/log/issues.log | sed -E 's/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}//g' | sort -u
So far it gets rid of ...
- 151
1
vote
1
answer
367
views
How to count the number of consecutive identical lines
I have a file that looks like this:
2000
2000
2001
2001
2001
2001
2002
2002
I need a script to show me this:
2000 - 2
2001 - 4
2002 - 2
I prefer using sed or awk
- 181
2
votes
1
answer
1k
views
using variable in awk
Hello I would like to count number of words with specific lengths. I'm using this command.
awk 'length == 2' mydict.txt | wc -l
this code gives what I want but if try to put variable instead of ...
- 33
0
votes
1
answer
290
views
awk -- duplicating a column and also modifying its text
awk {'printf ("%5s\t%s\n", $0, $NF)'} test_VN.txt
works for duplicating a column in a text file however, how can I add _VN before the .jpeg extension of the duplicated column?
For example, ...
- 4,635
0
votes
1
answer
668
views
playing with awk command trying to remove first line in input
below is my input file, results keep returning "Breed has Votes votes"
Breed, Votes
Black Lab, 30
Chihuahua, 2
Pug, 1
Corgi, 45
Shar Pei, 21
Shih Tzu, 5
...
0
votes
1
answer
215
views
awk command return not found on Ubuntu desktop but work well on Centos
I create test.sh to read a file with content
item1:return1
item2:return2
...
item10:return10
test.sh content
VAR1=$1
VAR2= `grep "^${VAR1}:" /home/path/file | awk '{print $2;}'`
echo "...
0
votes
0
answers
64
views
How to check whether one number range from one file is the subset of other number range from other file?
I'm trying to find out whether range1 numbers [both columns a and b] are the subset or lying between range2's columns [both columns b and c].
range1
a b
15 20
8 10
37 44
32 ...
- 111
1
vote
1
answer
3k
views
Awk command with word as delimitor
How to I use awk to cut based on a word that keeps everything after it :
Example:
File.txt >
XXXXWORDNNNNNN
XXXWORDNNNNNNN
XXWORDNNNNNNNN
XWORDNNNNNNNNN
WORDNNNNNNNNNN
output desired:
File.txt &...
0
votes
0
answers
264
views
Redirecting output from printf to /dev/tty in awk
I have a question on awk.
#!/usr/bin/nawk -f
{
if (NF != 4) {
error("Expected 4 fields");
} else {
print;
}
}
function error ( message ) {
if (FILENAME != &...
- 123
-1
votes
4
answers
2k
views
How to cut part of line?
I need to cut line in specific moment, I want to display GPU name, but only name, nothing else.
inxi -Gx | grep Device showing:
Device-2: NVIDIA GK107GLM [Quadro K1100M] vendor: Dell driver: nouveau
...
0
votes
1
answer
315
views
Find and replace the entire line of the source file [closed]
I am working on a batch script which will call the .awk file to execute the find and replace.
batch file script.cmd file -->
@gawk -f "%Modify.awk%" "%Temp.csv%" < "%...
- 1
0
votes
1
answer
82
views
tweak of two files
I am trying to merge two csv files based on 1st column (include both "exact string match" and "partial match") and if doesn't match - add those string but with blank 2nd column ...
- 69
1
vote
3
answers
98
views
Keep header from file1 and add column from file2 to file1
I need to copy a specific column from file2 to my file1, while keeping the header from file1.
File1:
##header
##header
##header
A1 B1 C1
A2 B2 C2
File2:
D1 E1 F1
D2 E2 F2
newfile:
##header
##header
#...
- 21
1
vote
1
answer
648
views
Changes in AWK on Ubuntu 20.04 LTS?
I am working with some huge astronomical databases and I just realized that awk is not working as expected. I'm downloading and cutting some .gz files, but awk restrictions are weird, let me share a ...
2
votes
1
answer
742
views
How to keep original decimal numbers without rounding off
I am trying to compute the sum of the travel distance and it is computed perfectly in the text file. However, the results would be rounded off to 4 decimal places
For instance, whenever I tried to add ...
- 35
1
vote
1
answer
2k
views
How to print awk's results with different colors for different fields?
ads2 cls create | awk -F [\"] '/^>>> cluster established .../ { strt=1 } strt!=1 { next } $1 ~ "name" { cnt++;nam[cnt]=$2 } $1 ~ "state" {stat[cnt]=$2;print -e "...
- 127
1
vote
1
answer
3k
views
awk - compare two files and print all columns from both files
I want to compare two files
File 1:
evm.TU.PTPU-T1. PF00808
evm.TU.PTP-T1 PF00498
evm.TU.PTPX-T1 PF00250
evm.TU.PAN-T1 PF00817
File 2:
PF00808 CL0012 Histone CBFD_NFYB_HMF ...
- 29