Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [awk]

awk is a special-purpose, interpreted programming language for extracting and reporting data.

0
votes
0answers
30 views

Cat and awk Commands

What does these commands mean in linux? GBPUSD30MIN and USDJPY30MIN are text files with data having 4 columns and 1000 rows. cat GBPUSD30MIN | awk '{ print NF }' | uniq -c awk '{ print $4}' ...
-2
votes
0answers
31 views

Awk has a command line option I see [closed]

Used in lots of examples "--cpp", I cannot find a reference to this option in any of the man pages, is there a reference for it or documentation?
1
vote
3answers
57 views

How to add string at specific column in a file which is not a white space

How to add string at specific column in a file which is not a white space. cat file.txt abc defg xy Now I want to add string "test? at column # 8 of file.txt. But column #8 is not a white space ...
0
votes
1answer
37 views

runaway regular expression error in awk

I have a script which converts expressions in different units to standard units like: distance=1km -> distance=1000m. This is my code: #!/bin/bash cat "$1" | grep '=' | awk -F= '{switch ($2) { ...
3
votes
3answers
292 views

Edit text file to add space

I have a text file which looks like this ../pool/main/a/aalib/libaa1_1.4p5-44build2_amd64.deb ../pool/main/a/accountsservice/accountsservice_0.6.45-1ubuntu1_amd64.deb ../pool/main/a/accountsservice/...
0
votes
1answer
19 views

Extract only package name without version using awk or sed

cat pkg.temp ca-certificates-2017.2.14-65.0.1.el6_9.noarch glusterfs-3.7.9-12.el6.x86_64 glusterfs-api-3.7.9-12.el6.x86_64 glusterfs-api-devel-3.7.9-12.el6.x86_64 glusterfs-client-xlators-3.7.9-12.el6....
0
votes
0answers
30 views

What does this command execute? - awk '{if ($2 ~ “variable1=”) {split($2,res,“=”); print res[2]}}

What does this command execute on a linux based system? i.e. shell script awk '{if ($2 ~ "variable1=") {split($2,res,"="); print res[2]}}
0
votes
2answers
73 views

Can I run two different awk commands in 1

I can use this awk command to get last line of log file: awk 'END{print}' /home/jj1/.pia_manager/log/pia_nw.log And I can use this awk command to extract a substring: awk -F"[()]" '{print $2}' /...
0
votes
0answers
22 views

What is awk -F / '{print $5}' [duplicate]

I have a command as following grep "/elv/[A-Z].*.gif" NASA_access_log_Aug85.txt |awk -F/ '{print$5}' |sort -u | grep "[A-Z]" > ele.txt I know that awk is for existing command but have no idea ...
0
votes
1answer
74 views

awk: cannot open “filename” for output (Too many open files)

So here is the culprit: zcat Merged.csv.gz | awk -F, 'FNR == 1 {header = $0; next} !seen[$7]++ {print header | "gzip > data/S20180528_"$7".txt.gz"} {print | "gzip > data/S20180528_"$7".txt.gz";...
0
votes
1answer
52 views

Capturing fdisk output into an array and printing it

I want to get a list of partitions on the system... I use this command sudo fdisk -l | grep '^/dev' | awk '{print $1}' The output is /dev/sda1 /dev/sda2 I need to get this result as first ...
2
votes
1answer
63 views

How to split awk output to multiple lines for piping

I'm currently putting together a quick script that deploys multiple vagrant machines locally for dev purposes. Par of the procedure consists of adding the ssh key to the vagrant boxes. So far, ...
0
votes
0answers
26 views

How to recursively go through all text files in a directory to fetch the desired line, and write these lines to same text file?

I have a directory which has text files (HTML pages). I wish to use SED to go through these text files, find the specific line, and write the line to a single text file. So for example I have HTMLS ...
0
votes
1answer
61 views

change string/value in 3rd column based on value change in 6th column - awk

I am trying to figure out how to use awk to force the change of consecutive numbering in 3rd column of my file based on the value change in my 6th column, but preserving the spacing?? Can somebody ...
0
votes
1answer
111 views

Python syntax error with awk, sprintf statment (2.7)

I am trying to use one of the shell commands to a python script, but struggling with some syntax error here. The description of the command is from here echo /dev/bus/usb/`udevadm info --name=/dev/...
-1
votes
2answers
642 views

Awk commands find and replace string and print every thing after key word [closed]

New to Linux commands.. What syntax would I use to find and replace a string with each line starting with "Data Centre:" Address information that is different on every line" but ends with the word ...
9
votes
2answers
732 views

Using AWK to get second column

I can not seem to get the awk command to get the second column of data. Bash Code: filter_data=$(awk "{if(/$filter:/) print $2}" < scanresults_temp.txt) printf "$filter_data \n" The $filter ...
0
votes
1answer
93 views

How to change a hypen to a period in a file and/or directory

I'm trying to do this in linux/bash. I have multiple directories. I don't mind doing each one individually as there are just a few. In each directory, there are files and folders with names like... ...
-1
votes
2answers
156 views

Insert a line of text after the line containing the last occurence of a specified word

Basically, I want to insert a line of text after the line containing the last occurence of a specific string, without affecting the rest of the file. I imagine I should proceed like this: delete ...
0
votes
1answer
69 views

awk: Set value in second line, colunm (if it exists) to variable

Consider this file: #!/usr/bin/env bash cat > example_file.txt <<EOL group, value 1, 3.21 1, 3.42 1, 3.5 2, 4.1 2, 4.2 EOL Now I want to set the value in the second column, second row to a ...
3
votes
1answer
294 views

awk: pipe output of (conditional) print to gzip

consider this file: #!/usr/bin/env bash cat > example_file.txt <<EOL group, value 1, 3.21 1, 3.42 1, 3.5 2, 4.1 2, 4.2 EOL in the following script, I group the rows of this file by the ...
0
votes
0answers
395 views

how to use gsub function awk to replace characters with empty value

for example my value alpha_1_2_u_345_cd.csv then i want to remove the characters present in using gsub in awk output should be 12345,alpha_1_2_u_345_cd.csv please let me know if there any evaluation ...
3
votes
3answers
187 views

Delete all lines from middle of a line matching a string until the second string match is found

I am trying to remove the error messages printed into my file. I have this: addr:1122c:1234: addr:11230:5678: addr:11223:01Error:abcdef(x, y) = z, value = a Error:hijklm(v, q) = w, value = b Error:...
1
vote
1answer
40 views

Could anyone explain me this AWK command please? [duplicate]

awk -F: '{ print $1 }' /etc/passwd Hello Guys, I am learning linux, i would ask weird doubts, please do not hesitate to answer. Can anyone explain clearly the above command.
1
vote
2answers
99 views

Parsing Text File By Column Pattern

I was wondering if there is an easy way to use awk or any other command to parse a text file such that a file of something like: Step Temp Enthalpy 0 0 -368 100 1 -369 200 2 -372 300 6 ...
2
votes
2answers
114 views

Parsing with Awk

new to Ubuntu so sorry if this has already been asked but I would like a way to parse a text file that has a general format of: -------- step 0 ---- cpu = Time_value ------- Energy = ...
1
vote
2answers
561 views

Delete multiple rows in csv file

I'm working on this assignment to delete rows from a CSV file with different customers. I've figured out how to delete one specific customer by using this code: delete() { awk -F "\"*;\"*" '$1 != '$...
2
votes
1answer
86 views

Add minutes to date time column

Thanks to various contributors resolving my previous request. I need to add minutes to the date column to get new datetime stamp value. I have a file : afile.txt 1,2012-02-16,abc,aa,455,340 3,2015-...
1
vote
1answer
37 views

Appending specified property of a specified header within a ini file

I have a ini file containing Headers with respective property fields to it. For a given Header2 and a property2 how do I append AppendedValue to the property2 by either using sed or awk? There may ...
1
vote
2answers
126 views

Problem with AWK inside bash

Hello there long history short: I'm trying to create a script to run through CI server to kill a specific process. So searching over the internet I found out this command: myuser@server:/home/myuser#...
11
votes
5answers
3k views

grep to return Nth and Mth lines before and after the match

I know that with grep I can use the fields -A and -B to pull previous and next lines from a match. However they pull in all lines between the match based on however many lines are specified. grep -...
1
vote
1answer
161 views

How to compare and replace the value at particular location with awk

I want to compare two csv files with one common column and then replace the data from first file to second file at particular location. file1.csv: "id","name","marks","subject","result" "1","a","","...
0
votes
1answer
170 views

awk -F not working

When I do the following commands: echo "1;2;3;4;5" > file.txt Then: awk -F; {print $1} file.txt I am not getting the second element of file.txt. What is wrong?
2
votes
2answers
269 views

Merge values of consecutive rows if they have the same values on a different column (AWK)

I need to combine the first value ($1) of consecutive rows if their fourth value ($4) is the same (I-PER). I managed to filter the values I need simply using awk: awk ' ($4 == "I-PER") {printf $1; ...
2
votes
1answer
299 views

Combine text files with the same header

With csplit I've been able to split a text file with a bunch of names and dates using these dates as a delimiter. Examples of files: File 'xx00': 1950 Frank Church James McClure File 'xx01': 1945 ...
1
vote
1answer
28 views

Replace only characters after symbol and before another?

I'm having real difficulty with this. Apologies for the noobish question. Lines grepped out yields this: 192.168.1.234/50112(LOCAL\username) Desired output: 192.168.1.234-(LOCAL\username) I know ...
2
votes
2answers
80 views

Joining two text files with multiple keys

I have two sets of data. and I want to join them by two key values (storm_ID, Cell_ID). The first dataset looks like: Storm_ID,Cell_ID,Wind_speed 2,10236258,27 2,10236300,58 2,10236301,25 3,...
3
votes
1answer
143 views

Checking if the 'block' has unique pattern

I have a file with number of 'blocks' (three blocks in this example) that look like this: A4_RAT Amyloid-beta A4 protein; P08592 PDB; 1M7E; X-ray; 2.45 A; D/E/F=755-763. ...
-1
votes
2answers
1k views

How to replace every nth occurrence of pattern using sed ONLY

I have searched all over the web but i can't find any good solutions. I have read on many forums that sed is not the tool (but awk is) for this but it makes no sense to me why. I mean you can do a ...
9
votes
2answers
1k views

Combine two files with awk

File1.txt item1 carA item2 carB item3 carC item4 platD item5 carE File2.txt carA platA carB platB carC platC carE platE Wanted output: item1 platA item2 platB item3 platC ...
0
votes
2answers
67 views

How to delete lines where certain pattern appears on the specific position

I am having a file that looks like this: PEBP1_HUMAN Homo sapiens P30086 PDB; 1BD9; X-ray; 2.05 A; A/B=1-187. PDB; 1BEH; X-ray; 1.75 A; A/B=1-187. PDB; ...
0
votes
0answers
156 views

gawk is crashing for complex regexp when LANG is set to .UTF-8 on 16.04

The following command will crash on my system when the LANG environment variable is set to eg. en_US.UTF-8, but will work fine with LANG=C: /usr/bin/gawk ' BEGIN { match(str, /^([^\1-\71\73-\...
0
votes
0answers
14 views

Merge files using a common column (using awk) [duplicate]

I have two text files: file 1: CHROM POS ID REF ALT 1 61442 rs2531261 A G 1 924629 rs28622096 A G,T 1 974662 rs2465135 G T 1 995481 rs9442393 T G file 2 CHROM POS ID REF ALT FORMAT 1 61442 ...
0
votes
1answer
110 views

Merge files using a common column (awk)

I have two text files: file 1: #CHROM POS ID REF ALT 1 61442 rs2531261 A G 1 924629 rs28622096 A G,T 1 974662 rs2465135 G T 1 995481 rs9442393 T G file 2 #CHROM POS ...
1
vote
2answers
146 views

AWK split output with the line

How can I split output lines using tailf and awk like this? From this: tail -f logfile.log | awk '/connected/ {print "\033[31m" $0 "\033[39m"}' Output: 2018-01-31 10:00 user connected ...
0
votes
2answers
1k views

remove the Carriage return with bash

I have a Ldif file which I would extract some lines from. the LDIF is 78 character by line, so whenever there is a line contain more than 78 character it's gonna be devided in two line. what I want ...
1
vote
1answer
575 views

If two columns partially match, replace third with awk

File a.txt chr:1:10539:A:C 10539 C A 0.545987 0.508902 0 0.36065 + 1 chr:2:13494:A:G 13494 A G 0.330493 0.0264746 0 0.733423 + 1 chr:7:13494:A:G 13494 A G 0.330493 0.0264746 0 0.733423 + 1 File b....
3
votes
1answer
1k views

awk store a pattern result to a shell array variable

I am trying to store the result of a pattern matched by awk to a shell array variable. Here's a simplified example of the same: #!/bin/bash declare -a array1=() declare -a array2=() READ_FILE="...
2
votes
1answer
208 views

Get Nth result of awk

I am trying to find text between two patterns and it works. But sometimes, I have more than one occurrence, and I would need to get all and use the one I want. Command example: awk '/>23958742<...
0
votes
1answer
149 views

How can I remove a string from the beginning of each line of text? [duplicate]

I grab the contents of a directory with du -a /test1 | cut -f2 and get this: /test1/file1 /test1/directory1 /test1/directory1/file2 My hope is to use something like cut, sed, or grep to say "...