131

I would like to examine the network traffic being handled by a single process, but simple network captures won't work since I am dealing with such a busy system (lots of other traffic happening at the same time). Is there a way to isolate a tcpdump or wireshark capture to the networking traffic of a single specific process? (Using netstat is insufficient.)

16 Answers 16

184

To start and monitor an new process:

strace -f -e trace=network -s 10000 PROCESS ARGUMENTS

To monitor an existing process with a known PID:

strace -p $PID -f -e trace=network -s 10000
  • -f is for "follow new processes"
  • -e defines a filter
  • -s sets the limit of strings to more then 32
  • -p takes the process id to attach to
4
  • 4
    This is useful because it can be used without having root access or special permissions (on some Linux distributions, anyway - on Ubuntu you may need special permissions). Jun 7, 2012 at 9:24
  • 2
    This is also useful, because it can be run against an already-launched process and is available on virtually any Linux box.
    – zakmck
    Sep 22, 2015 at 9:24
  • any way to monitor all processes owned by a non-root user? Mar 26, 2020 at 21:39
  • Note that this only shows networking specific system calls, such socket, bind, connect, recvfrom, etc. It doesn't show any of the data sent/received using write/read system calls. To show those you need to do -e %network,read,write or -e %network,%desc but doing that will show all I/O not just network I/O, so it is quite a bit messier to sort through than using tcpdump or wireshark. Using strace is thus probably most useful when used in combination with those tools to determine which ports to filter on.
    – pavon
    May 24, 2023 at 20:21
73

I know this thread is a bit old but I think this might help some of you:

If your kernel allows it, capturing the network traffic of a single process is very easily done by running the said process in an isolated network namespace and using wireshark (or other standard networking tools) in the said namespace as well.

The setup might seem a bit complex, but once you understand it and become familiar with it, it will ease your work so much.

So as to do so:

  • create a test network namespace:

    ip netns add test
    
  • create a pair of virtual network interfaces (veth-a and veth-b):

    ip link add veth-a type veth peer name veth-b
    
  • change the active namespace of the veth-a interface:

    ip link set veth-a netns test
    
  • configure the IP addresses of the virtual interfaces:

    ip netns exec test ifconfig veth-a up 192.168.163.1 netmask 255.255.255.0
    ifconfig veth-b up 192.168.163.254 netmask 255.255.255.0
    
  • configure the routing in the test namespace:

    ip netns exec test route add default gw 192.168.163.254 dev veth-a
    
  • activate ip_forward and establish a NAT rule to forward the traffic coming in from the namespace you created (you have to adjust the network interface and SNAT ip address):

    echo 1 > /proc/sys/net/ipv4/ip_forward
    iptables -t nat -A POSTROUTING -s 192.168.163.0/24 -o <your internet interface, e.g. eth0> -j SNAT --to-source <your ip address>
    

    (You can also use the MASQUERADE rule if you prefer)

  • finally, you can run the process you want to analyze in the new namespace, and wireshark too:

    ip netns exec test thebinarytotest
    ip netns exec test wireshark
    

    You'll have to monitor the veth-a interface.

8
  • 5
    Great idea, but beware of the "limitations". Since this is a separate namespace, you cannot communicate with local processes in the default namespace using the loopback address(es) or UNIX domain sockets. The latter affects communication over D-Bus.
    – Lekensteyn
    Feb 15, 2015 at 14:53
  • @Lekensteyn you can still use unix domain sockets across network namespaces afaik. The file system is not isolated by them.
    – randunel
    Jan 29, 2016 at 17:42
  • 1
    @randunel I should have be more precise on that. What I meant to say is that Unix domain sockets in the "abstract socket namespace" (which does not use the filesystem) cannot directly be accessed between network namespaces. As a workaround, you can use a proxy such as socat.
    – Lekensteyn
    Jan 30, 2016 at 13:36
  • What IP address do you use with the --to-source argument to iptables? Is this the IP address of the interface you pass to the -o option, an IP address you make up, or ??? I tried the masquerade version that doesn't need --to-source, as described here, and that worked!
    – ntc2
    Feb 2, 2017 at 3:03
  • 3
    All of this seems to require root access. Oct 5, 2017 at 17:23
23

Indeed there is a way, using the Wireshark filters. But you cannot filter directly by process name or PID (because they are not a network quantities).

You should first figure out the protocols and the ports used by your process (the netstat command in the previous comment works well).

Then use Wireshark to filter the inbound (or outbound) port with the one you just retrieve. That should isolate the incoming and outcoming traffic of your process.

6
  • 7
    For a simple connection, this is possible, but I need to track DNS, HTTP, etc, that are all rushing past, so there isn't a simple way to just use netstat and simple network capture filters on a busy machine.
    – Kees Cook
    Nov 6, 2010 at 18:12
  • Ok, HTTP and DNS public ports are used by a lot of application, but the corresponding private port is unique. So why don't you try filtering by the private port?
    – OpenNingia
    Nov 8, 2010 at 11:13
  • 1
    Because rapid small requests won't be seen by netstat; I'll only be able to catch long-lived connections. :(
    – Kees Cook
    Nov 8, 2010 at 18:58
  • what if the process uses dynamic ports at run time, then your not going to be able to uses static port filters Nov 10, 2010 at 10:00
  • I think you've here the best answer... Sadly a network sniffing tool works at the lowest level of the net stack, trying to catch everything, it's completely unaware of processes running on the OS. It'd be extremely difficult to find out what's originated a certain call. A packet sniffer could eventually figure out (via the port number) a process ID but cannot figure out which process did a DNS lookup as this is completely independent (that's most probably the kernel net stack that triggered the call). But with filtering and stoping other processes you should be able to achieve your goal.
    – Huygens
    Nov 10, 2010 at 13:05
21
netstat -taucp | grep <pid or process name>

That will show the connections an application is making including the port being used.

4
  • 10
    This would show connections that exist for that instant, but it won't provide a log of the traffic itself.
    – Kees Cook
    Nov 6, 2010 at 17:36
  • 1
    Not sure about the Kees Cook comment. A simple netstat shows info about connections for an instant, but with the flag -c you get a snapshot of that state every second (see 'man netstat'). Maybe it does not have all the traffic, but is not a unique snapshot of the connections.
    – tremendows
    Dec 12, 2013 at 15:30
  • 3
    Well, I'm sure. This will not capture all network traffic of a process. Jun 10, 2016 at 9:31
  • This was useful for figuring out which port number was being blocked by an internal firewall. It showed the TCP SYNs (along with destination address and port number) being sent by the command I was running. Jan 15, 2019 at 12:48
16

I have come to a similar issue and I was able to sort it out based on this answer by ioerror, using NFLOG as described here:

# iptables -A OUTPUT -m owner --uid-owner 1000 -j CONNMARK --set-mark 1
# iptables -A INPUT -m connmark --mark 1 -j NFLOG --nflog-group 30 
# iptables -A OUTPUT -m connmark --mark 1 -j NFLOG --nflog-group 30 
# dumpcap -i nflog:30 -w uid-1000.pcap

Then you can create run the process in question from a user account that doesn't do anything else - and voila, you have just isolated and captured traffic from a single process.

Just wanted to post back in case it helps anyone.

15
+50

Just an idea: Is it possible to bind your application to a different IP address? If so, you can use the usual suspects (tcpdump, etc.)

Tools for applications which are not capable of binding to another IP address:

http://freshmeat.net/projects/fixsrcip

fixsrcip is a tool for binding outgoing TCP and UDP client sockets (IPv4) to specific source IP addresses on multi-homed hosts

http://freshmeat.net/projects/force_bind

force_bind allows you to force binding on a specific IP and/or port. It works with both IPv4 and IPv6.

2
  • Most applications don't support specifying their source IP, but this actually might be possible by using a container with CLONE_NEWNET but not CLONE_NEWNS.
    – Kees Cook
    Nov 11, 2010 at 6:14
  • Alternatively, you can create a network namespace and run your app inside it www.evolware.org/?p=293
    – Flint
    May 17, 2014 at 4:29
10

I wrote a C application that does what is described in the great answer above by felahdab!

See here: nsntrace github repo

2
  • That's cool, but I think it would be nice to include some more details for how to get it and use it :)
    – Zanna
    Jul 17, 2016 at 9:33
  • Thanks! I provided a link to the github repo which holds a README.md file which have usage and examples, and download instructions! Jul 18, 2016 at 9:49
6

You can try tracedump - http://mutrics.iitis.pl/tracedump

It does exactly what you want, you can either give it a process ID or a program to run.

1
  • 3
    Nice project, but... "Tracedump currently runs on 32-bit Linux hosts only" kills it for me, unfortunately.
    – gertvdijk
    Sep 30, 2014 at 13:29
5

This is a dirty hack but I'd suggest either a divert or a log target with iptables for a given UID. eg:

iptables -t nat -A OUTPUT -p tcp -m owner --uid-owner $USER -m tcp -j LOG 
iptables -t nat -A OUTPUT -p udp -m owner --uid-owner $USER -m udp -j LOG 

It might also be worth looking into something like '--log-tcp-sequence', '--log-tcp-options', '--log-ip-options', '--log-uid' for that log target. Though I suspect that will only help you post process a pcap that includes a ton of other data.

The NFLOG target might be useful if you want to flag packets and then certain tagged packets will be sent over a netlink socket to a process of your choosing. I wonder if that would be useful for hacking up something with wireshark and your specific application running as a specific user?

1
  • but this works only for outgoing, what about incomming ?
    – resultsway
    Oct 20, 2014 at 17:14
5

Try running the process you're interested in under strace:

strace ping www.askubuntu.com

It will give you some very detailed information about what your process is doing. As a process can open up any ports it wants to anywhere, using a predefined filter you may miss something.

Another approach would be to use a stripped-down virtual machine or a test machine on your network, and place your process on it in isolation on this. Then you can just use Wireshark to catch all from that machine. You'll be pretty sure that the traffic you capture will be relevant.

1
  • Adding in an "-e trace=network" prunes down some of the output, at the expense of losing the data which is actually written to the network. If you only care about "number of sockets opened" or similar, this makes things easier.
    – dannysauer
    Feb 25, 2015 at 20:20
4

Building on the answer by ioerror I suspect you can use iptables --uid-owner to set a marker on the traffic, and then you can ask wireshark to capture only traffic with that marker. You might be able to use a DSCP (differential services marker), flow id or a qos marker.

Or indeed you could use this to send those packets out a different interface, and then capture only on that interface.

2

wireshark bug #1184 is this feature. It was reported in year 2006 and is not implemented as of year 2019.

1

As a supplement to @felahdab's answer, I have written a shell script tool called runnet, which can easily create a network namespace to run specific programs. You can found it here: https://github.com/KB5201314/runnet.

just run

sudo runnet --internet <your cmd>

this will create a separate network namespace.

Then, you can see a interface like runnetxxxxx_vo on your wireshark. You canstart listening on it.

screenshot of wireshark interface list

You can even use the --internet option to control whether to forward traffic to the external network. You can also control which ports are mapped to the host, this is achieved with the power of socat.

0

You can use py_strace2pcap by following these steps:

  1. Clone the repository:

    git clone https://github.com/comboshreddies/py-strace2pcap.git
    
  2. Move inside the cloned directory:

    cd py-strace2pcap
    
  3. Install the required scapy module with:

    pip3 install scapy
    

    or

    pip3 install -r requirements.txt
    
  4. Then run strace with required set of parameters, e.g. (from the project's README):

    strace -f -s65535 -o /tmp/straceSample -ttt -T -yy -xx command
    

    and process it with this tool:

    py_strace2pcap.py file_to_store.pcap < /tmp/straceSample
    
  5. You will get a .pcap file that you can inspect with wireshark:

    wireshark file_to_store.pcap
    
New contributor
127.0 .0.1 is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
-1

maybe iptables and ulog can work? Not that I have an exact recipe, but I think iptables can match processes, once matched you could use ulog.

1
-2

I think you can create a shell script to loop through executing netstat and logging it to a text file. Something like (very rough steps):

echo "press q to quit"
while [ <q is not pressed>]
do
    `netstat -taucp | grep <pid or process name> 1>>logfile.txt`
done

I am not a programmer, so I can't refine this. But someone here can start from where I left off and create a working script for you.

1
  • 1
    Not good enough. The question is to capture all network traffic, not just whatever happens to be active at the times you happen to check. Jun 10, 2016 at 9:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .