Questions tagged [awk]
awk is a special-purpose, interpreted programming language for extracting and reporting data.
0
votes
0answers
30 views
Cat and awk Commands
What does these commands mean in linux? GBPUSD30MIN and USDJPY30MIN are text files with data having 4 columns and 1000 rows.
cat GBPUSD30MIN | awk '{ print NF }' | uniq -c
awk '{ print $4}' ...
-2
votes
0answers
31 views
Awk has a command line option I see [closed]
Used in lots of examples "--cpp", I cannot find a reference to this option in any of the man pages, is there a reference for it or documentation?
1
vote
3answers
57 views
How to add string at specific column in a file which is not a white space
How to add string at specific column in a file which is not a white space.
cat file.txt
abc
defg
xy
Now I want to add string "test? at column # 8 of file.txt. But column #8 is not a white space ...
0
votes
1answer
37 views
runaway regular expression error in awk
I have a script which converts expressions in different units to standard units like:
distance=1km -> distance=1000m.
This is my code:
#!/bin/bash
cat "$1" | grep '=' | awk -F= '{switch ($2) {
...
3
votes
3answers
292 views
Edit text file to add space
I have a text file which looks like this
../pool/main/a/aalib/libaa1_1.4p5-44build2_amd64.deb
../pool/main/a/accountsservice/accountsservice_0.6.45-1ubuntu1_amd64.deb
../pool/main/a/accountsservice/...
0
votes
1answer
19 views
Extract only package name without version using awk or sed
cat pkg.temp
ca-certificates-2017.2.14-65.0.1.el6_9.noarch
glusterfs-3.7.9-12.el6.x86_64
glusterfs-api-3.7.9-12.el6.x86_64
glusterfs-api-devel-3.7.9-12.el6.x86_64
glusterfs-client-xlators-3.7.9-12.el6....
0
votes
0answers
30 views
What does this command execute? - awk '{if ($2 ~ “variable1=”) {split($2,res,“=”); print res[2]}}
What does this command execute on a linux based system? i.e. shell script
awk '{if ($2 ~ "variable1=") {split($2,res,"="); print res[2]}}
0
votes
2answers
73 views
Can I run two different awk commands in 1
I can use this awk command to get last line of log file:
awk 'END{print}' /home/jj1/.pia_manager/log/pia_nw.log
And I can use this awk command to extract a substring:
awk -F"[()]" '{print $2}' /...
0
votes
0answers
22 views
What is awk -F / '{print $5}' [duplicate]
I have a command as following
grep "/elv/[A-Z].*.gif" NASA_access_log_Aug85.txt |awk -F/ '{print$5}' |sort -u | grep "[A-Z]" > ele.txt
I know that awk is for existing command but have no idea ...
0
votes
1answer
74 views
awk: cannot open “filename” for output (Too many open files)
So here is the culprit:
zcat Merged.csv.gz | awk -F, 'FNR == 1 {header = $0; next} !seen[$7]++ {print header | "gzip > data/S20180528_"$7".txt.gz"} {print | "gzip > data/S20180528_"$7".txt.gz";...
0
votes
1answer
52 views
Capturing fdisk output into an array and printing it
I want to get a list of partitions on the system...
I use this command
sudo fdisk -l | grep '^/dev' | awk '{print $1}'
The output is
/dev/sda1
/dev/sda2
I need to get this result as first ...
2
votes
1answer
63 views
How to split awk output to multiple lines for piping
I'm currently putting together a quick script that deploys multiple vagrant machines locally for dev purposes.
Par of the procedure consists of adding the ssh key to the vagrant boxes.
So far, ...
0
votes
0answers
26 views
How to recursively go through all text files in a directory to fetch the desired line, and write these lines to same text file?
I have a directory which has text files (HTML pages). I wish to use SED to go through these text files, find the specific line, and write the line to a single text file.
So for example I have HTMLS ...
0
votes
1answer
61 views
change string/value in 3rd column based on value change in 6th column - awk
I am trying to figure out how to use awk to force the change of consecutive numbering in 3rd column of my file based on the value change in my 6th column, but preserving the spacing?? Can somebody ...
0
votes
1answer
111 views
Python syntax error with awk, sprintf statment (2.7)
I am trying to use one of the shell commands to a python script, but struggling with some syntax error here.
The description of the command is from here
echo /dev/bus/usb/`udevadm info --name=/dev/...
-1
votes
2answers
642 views
Awk commands find and replace string and print every thing after key word [closed]
New to Linux commands.. What syntax would I use to find and replace a string with each line starting with "Data Centre:" Address information that is different on every line" but ends with the word ...
9
votes
2answers
732 views
Using AWK to get second column
I can not seem to get the awk command to get the second column of data.
Bash Code:
filter_data=$(awk "{if(/$filter:/) print $2}" < scanresults_temp.txt)
printf "$filter_data \n"
The $filter ...
0
votes
1answer
93 views
How to change a hypen to a period in a file and/or directory
I'm trying to do this in linux/bash. I have multiple directories. I don't mind doing each one individually as there are just a few. In each directory, there are files and folders with names like...
...
-1
votes
2answers
156 views
Insert a line of text after the line containing the last occurence of a specified word
Basically, I want to insert a line of text after the line containing the last
occurence of a specific string, without affecting the rest of the file.
I imagine I should proceed like this:
delete ...
0
votes
1answer
69 views
awk: Set value in second line, colunm (if it exists) to variable
Consider this file:
#!/usr/bin/env bash
cat > example_file.txt <<EOL
group, value
1, 3.21
1, 3.42
1, 3.5
2, 4.1
2, 4.2
EOL
Now I want to set the value in the second column, second row to a ...
3
votes
1answer
294 views
awk: pipe output of (conditional) print to gzip
consider this file:
#!/usr/bin/env bash
cat > example_file.txt <<EOL
group, value
1, 3.21
1, 3.42
1, 3.5
2, 4.1
2, 4.2
EOL
in the following script, I group the rows of this file by
the ...
0
votes
0answers
395 views
how to use gsub function awk to replace characters with empty value
for example my value alpha_1_2_u_345_cd.csv then i want to remove the characters present in using gsub in awk
output should be
12345,alpha_1_2_u_345_cd.csv
please let me know if there any evaluation ...
3
votes
3answers
187 views
Delete all lines from middle of a line matching a string until the second string match is found
I am trying to remove the error messages printed into my file.
I have this:
addr:1122c:1234:
addr:11230:5678:
addr:11223:01Error:abcdef(x, y) = z, value = a
Error:hijklm(v, q) = w, value = b
Error:...
1
vote
1answer
40 views
Could anyone explain me this AWK command please? [duplicate]
awk -F: '{ print $1 }' /etc/passwd
Hello Guys,
I am learning linux, i would ask weird doubts, please do not hesitate to answer. Can anyone explain clearly the above command.
1
vote
2answers
99 views
Parsing Text File By Column Pattern
I was wondering if there is an easy way to use awk or any other command to parse a text file such that a file of something like:
Step Temp Enthalpy
0 0 -368
100 1 -369
200 2 -372
300 6 ...
2
votes
2answers
114 views
Parsing with Awk
new to Ubuntu so sorry if this has already been asked but I would like a way to parse a text file that has a general format of:
-------- step 0 ---- cpu = Time_value -------
Energy = ...
1
vote
2answers
561 views
Delete multiple rows in csv file
I'm working on this assignment to delete rows from a CSV file with different customers. I've figured out how to delete one specific customer by using this code:
delete() {
awk -F "\"*;\"*" '$1 != '$...
2
votes
1answer
86 views
Add minutes to date time column
Thanks to various contributors resolving my previous request.
I need to add minutes to the date column to get new datetime stamp value. I have a file : afile.txt
1,2012-02-16,abc,aa,455,340
3,2015-...
1
vote
1answer
37 views
Appending specified property of a specified header within a ini file
I have a ini file containing Headers with respective property fields to it. For a given Header2 and a property2 how do I append AppendedValue to the property2 by either using sed or awk?
There may ...
1
vote
2answers
126 views
Problem with AWK inside bash
Hello there long history short: I'm trying to create a script to run through CI server to kill a specific process.
So searching over the internet I found out this command:
myuser@server:/home/myuser#...
11
votes
5answers
3k views
grep to return Nth and Mth lines before and after the match
I know that with grep I can use the fields -A and -B to pull previous and next lines from a match.
However they pull in all lines between the match based on however many lines are specified.
grep -...
1
vote
1answer
161 views
How to compare and replace the value at particular location with awk
I want to compare two csv files with one common column and then replace the data from first file to second file at particular location.
file1.csv:
"id","name","marks","subject","result"
"1","a","","...
0
votes
1answer
170 views
awk -F not working
When I do the following commands:
echo "1;2;3;4;5" > file.txt
Then:
awk -F; {print $1} file.txt
I am not getting the second element of file.txt.
What is wrong?
2
votes
2answers
269 views
Merge values of consecutive rows if they have the same values on a different column (AWK)
I need to combine the first value ($1) of consecutive rows if their fourth value ($4) is the same (I-PER).
I managed to filter the values I need simply using awk:
awk ' ($4 == "I-PER") {printf $1; ...
2
votes
1answer
299 views
Combine text files with the same header
With csplit I've been able to split a text file with a bunch of names and dates using these dates as a delimiter.
Examples of files:
File 'xx00':
1950
Frank
Church
James
McClure
File 'xx01':
1945
...
1
vote
1answer
28 views
Replace only characters after symbol and before another?
I'm having real difficulty with this. Apologies for the noobish question.
Lines grepped out yields this:
192.168.1.234/50112(LOCAL\username)
Desired output:
192.168.1.234-(LOCAL\username)
I know ...
2
votes
2answers
80 views
Joining two text files with multiple keys
I have two sets of data. and I want to join them by two key values (storm_ID, Cell_ID).
The first dataset looks like:
Storm_ID,Cell_ID,Wind_speed
2,10236258,27
2,10236300,58
2,10236301,25
3,...
3
votes
1answer
143 views
Checking if the 'block' has unique pattern
I have a file with number of 'blocks' (three blocks in this example) that look like this:
A4_RAT Amyloid-beta A4 protein; P08592 PDB; 1M7E; X-ray; 2.45 A; D/E/F=755-763.
...
-1
votes
2answers
1k views
How to replace every nth occurrence of pattern using sed ONLY
I have searched all over the web but i can't find any good solutions. I have read on many forums that sed is not the tool (but awk is) for this but it makes no sense to me why. I mean you can do a ...
9
votes
2answers
1k views
Combine two files with awk
File1.txt
item1 carA
item2 carB
item3 carC
item4 platD
item5 carE
File2.txt
carA platA
carB platB
carC platC
carE platE
Wanted output:
item1 platA
item2 platB
item3 platC
...
0
votes
2answers
67 views
How to delete lines where certain pattern appears on the specific position
I am having a file that looks like this:
PEBP1_HUMAN Homo sapiens P30086 PDB; 1BD9; X-ray; 2.05 A; A/B=1-187.
PDB; 1BEH; X-ray; 1.75 A; A/B=1-187.
PDB; ...
0
votes
0answers
156 views
gawk is crashing for complex regexp when LANG is set to .UTF-8 on 16.04
The following command will crash on my system when the LANG environment variable is set to eg. en_US.UTF-8, but will work fine with LANG=C:
/usr/bin/gawk '
BEGIN {
match(str, /^([^\1-\71\73-\...
0
votes
0answers
14 views
Merge files using a common column (using awk) [duplicate]
I have two text files:
file 1:
CHROM POS ID REF ALT
1 61442 rs2531261 A G
1 924629 rs28622096 A G,T
1 974662 rs2465135 G T
1 995481 rs9442393 T G
file 2
CHROM POS ID REF ALT FORMAT
1 61442 ...
0
votes
1answer
110 views
Merge files using a common column (awk)
I have two text files:
file 1:
#CHROM POS ID REF ALT
1 61442 rs2531261 A G
1 924629 rs28622096 A G,T
1 974662 rs2465135 G T
1 995481 rs9442393 T G
file 2
#CHROM POS ...
1
vote
2answers
146 views
AWK split output with the line
How can I split output lines using tailf and awk like this?
From this:
tail -f logfile.log | awk '/connected/ {print "\033[31m" $0 "\033[39m"}'
Output:
2018-01-31 10:00 user connected ...
0
votes
2answers
1k views
remove the Carriage return with bash
I have a Ldif file which I would extract some lines from.
the LDIF is 78 character by line, so whenever there is a line contain more than 78 character it's gonna be devided in two line.
what I want ...
1
vote
1answer
575 views
If two columns partially match, replace third with awk
File a.txt
chr:1:10539:A:C 10539 C A 0.545987 0.508902 0 0.36065 + 1
chr:2:13494:A:G 13494 A G 0.330493 0.0264746 0 0.733423 + 1
chr:7:13494:A:G 13494 A G 0.330493 0.0264746 0 0.733423 + 1
File b....
3
votes
1answer
1k views
awk store a pattern result to a shell array variable
I am trying to store the result of a pattern matched by awk to a shell array variable. Here's a simplified example of the same:
#!/bin/bash
declare -a array1=()
declare -a array2=()
READ_FILE="...
2
votes
1answer
208 views
Get Nth result of awk
I am trying to find text between two patterns and it works. But sometimes, I have more than one occurrence, and I would need to get all and use the one I want.
Command example:
awk '/>23958742<...
0
votes
1answer
149 views
How can I remove a string from the beginning of each line of text? [duplicate]
I grab the contents of a directory with du -a /test1 | cut -f2 and get this:
/test1/file1
/test1/directory1
/test1/directory1/file2
My hope is to use something like cut, sed, or grep to say "...
