19

Raised bug -> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1874257

SSH timeout issue, once connect to VPN.

Connecting via putty fine. No changes made before.

VPN established by open-connect. This is previously working. Reinstalled VPN packages and reboot still no luck

Environment

Dell XPS 9570
Ubuntu 16.04.6 Xenial Xerus)
kernel - 4.15.0-55-generic

$dpkg -l | grep -i openssh
ii openssh-client 1:7.2p2-4ubuntu2.8 -->
ii openssh-server 1:7.2p2-4ubuntu2.8
ii openssh-sftp-server 1:7.2p2-4ubuntu2.8

VPN tunnel info
====
vpn0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:IP P-t-P:xx Mask:255.255.252.0
          inet6 addr: fe80::b8e2:bea4:2e62:fe08/64 Scope:Link
          UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1406 Metric:1
          RX packets:962 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1029 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:87839 (87.8 KB) TX bytes:238740 (238.7 KB)

Issue

Unable to connect to any host via ssh or sftp after VPN connection

Tried

Reinstalled the openssh-client package and still no luck. May I know why the default cipher is not taking/hanging? Please let me know . There were no recent changes.

Workaround

Able to connect to ssh / sftp $ssh -c aes128-ctr user@IP

Below is the debug ssh client logs ===

$ssh -vvv user@ip
OpenSSH_7.2p2 Ubuntu-4ubuntu2.8, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "IP" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to IP [IP] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_rsa-cert type -1



debug2: compression stoc: none,[email protected]
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

<< Hangs here >>


Please shed some views

Thanks

7
  • 1
    Related: serverfault.com/questions/210408/…
    – Terrance
    Commented Apr 22, 2020 at 13:50
  • 1
    Tried with MTU lower to 1200, still not fixed. the above link is not useful.
    – Jay
    Commented Apr 25, 2020 at 5:28
  • There are several answers there in the link that you can try. Only the accepted one worked for the person who asked the question, but there's also trying MTU at 1400, or setting different certificate types, etc. Don't stop after only trying the one thing there. Look for what might fit yours the best. None of the answers there that have upvotes look destructive at all.
    – Terrance
    Commented Apr 25, 2020 at 14:15
  • 2
    Hey I tried the MTU to 1100 which fixed the problem. What is the reason behind this?
    – Jay
    Commented Apr 26, 2020 at 13:27
  • It could be a noisy or unreliable connection to the server. Probably that data corruption is happening to the packet when it is sent. See the trade offs in en.wikipedia.org/wiki/Maximum_transmission_unit Also, there is a possibility that forward error correction is not being used. See: en.wikipedia.org/wiki/Forward_error_correction These are only speculations that I have on it.
    – Terrance
    Commented Apr 26, 2020 at 14:41

6 Answers 6

12

As a temporary workaround, setting the KEX algorithm manually solves this problem for me.

Add KexAlgorithms ecdh-sha2-nistp521 to the corresponding SSH config, or add -oKexAlgorithms=ecdh-sha2-nistp521 to the command line args for one time use.

Reference (2019): https://superuser.com/a/1462220/555702

1
  • Thanks for sharing your answer, this was exactly what helped me with my problem.
    – Greg0ry
    Commented May 7, 2023 at 22:22
11

Setting MTU to 1200 solved it for me (Ubuntu 20.04).

with (replace {dev}):  
sudo ip li set mtu 1200 dev {dev}
2
  • Had same issue with git (via ssh) inside WSL2, sudo ip link set eth0 mtu 1300 also helped
    – user158037
    Commented Dec 6, 2021 at 14:12
  • 1
    Note that in some cases you will need to limit the MTU for the other direction too (on the SSH server). You can also limit MTU per route ip route ... mtu 1300. It is not always needed to set MTU for the whole interface. Commented Oct 30, 2022 at 18:01
4

The root cause of the issue is with the openconnect VPN client package, MTU handling fail to negotiate. Bug -> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1874257

1

A colleague of mine just experienced this hanging at "expecting SSH2_MSG_KEX_ECDH_REPLY" issue this morning. He couldn't ssh into an Ubuntu 20.04.2 LTS server via OpenVPN. At first I thought it was an MTU issue but it turns out that it was a file permissions issue. Our organization uses signed keys and the file containing the CA public key only had owner read/write permissions.

We fixed the problem by adding read permissions for all users:

chmod a+r ca.pub

The bizarre thing about this problem was that even with the highest level of debug enabled while trying to use the ssh client, there was no error reported relating to the inability to read the file. The only related debug message had to do with successfully finding the @cert-authority line in the known_hosts file.

0
0

I've experienced the problem while trying to connect to a GCP Ubuntu 22 VM which is a part of a ZeroTier network. A ZeroTier interface zt<xxxx> defaults MTU to 2800. Setting it to 1300 solved the issue.

0

Most of the time it happens because of PMTUD (Path MTU Discovery) does not work between client host and server host. Usually because (some) ICMP traffic is blocked on the path. Reasons might be:

  • hardened security
  • sometimes administrators being unaware of PMTUD use firewall rules without taking into account that the routers on the path can also participate in the connection, namely send ICMP messages
  • probably bugs in the software, for example I noticed that PMTUD stop working on MikroTik when you changing routing table in mangle table (though this could be expected behaviour)

Whatever the reason, the ICMP "unreachable" message is not sent to one or both participating hosts.

Changing MTU on the one or the both sides will work, but in most cases it is not desired and only can be recommended as a temporary workaround.

Instead, I recommend fixing PMTUD if possible. For example you have hosts A and B, routers R1 and R2 connected via the tunnel:

A ---- R1 ===== R2 ---- B

Tunnels usually has smaller MTU than standard 1500 LAN MTU.

  • If it is broken by some complex firewall setup on the routers, there could be a way to specify the rule that allows outgoing traffic from the router to the participating hosts
  • The other way is to use the workaround called MSS clamping. It can be done using rules in mangle table. Linux example: iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu. You can do this on any other capable router like Cisco, MikroTik, etc. It is better than reducing your MTU and therefore the network performance. Note that if there are some other routers between R1 and R2 which has lower MTU then R1 or R2 can not know which MSS to set. In this case you should set MSS explicitly like this iptables -t mangle -A POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o wireguard0 -j TCPMSS --set-mss 1380. Check here

Sometimes setting up MSS clamping for outgoing traffic is enough. Sometimes you need to set it up for both traffic coming from the tunnel and for traffic going to the tunnel.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .