Can't SSH without password to default port of 22 using public/private key authentication

Question

I have set up a private/public key authentication to target server correctly. It was running before I did the upgrade. The server is the GitLab CE server, running an omnibus distribution from Ubuntu 16.04 repository. The issue was found just after the upgrade process is done, when I unable to perform push/pull to the target server. It turn out the main problem might not because of the GitLab. The version of GitLab I'm using is GitLab CE 12.1.3

• I have check the permission of the ${HOME_USER}/.ssh/ directory both on the normal user as well as on the /var/opt/gitlab/.ssh/, the default home dir of the gitlab. It was configured as 700.
• The authorized_keys file also has been configured as 600
• both .ssh/ directory is perfectly owned by its user (user git for example)
• I have tried to re-generate the private/public key as well, using several type from ed25519 as well as rsa
• I did check the /etc/ssh/sshd_config as well as /etc/ssh/ssh_config, it just configured as usually done and it should be working as it is
• I did found it was strange behavior, the log didn't seems to trigger publickey authentication if I use the default port (22), but it works just fine with other port
• There is no ufw, firewall, fail2ban, selinux etc. settings applied

The following is the log I got while trying to connect through port 22

MacBook-Pro-2:.ssh urhen$ ssh -vT [email protected]
OpenSSH_7.8p1, LibreSSL 2.6.2
debug1: Reading configuration data /Users/urhen/.ssh/config
debug1: /Users/urhen/.ssh/config line 2: Applying options for gitlab-ce.example.com
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: /etc/ssh/ssh_config line 52: Applying options for *
debug1: Connecting to gitlab-ce.example.com port 22.
debug1: Connection established.
debug1: identity file /Users/urhen/.ssh/gitlab/id_rsa type 0
debug1: identity file /Users/urhen/.ssh/gitlab/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.8
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.8 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to gitlab-ce.example.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:mDG0M8BFZ4vMyAD1OKjgC1/RHORl1WtalFqfDMthWLA
debug1: Host 'gitlab-ce.example.com' is known and matches the ECDSA host key.
debug1: Found key in /Users/urhen/.ssh/known_hosts:47
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: password
debug1: Next authentication method: password
[email protected]'s password:

The custom port has the following log which make it able to be authenticated by public key

debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received

while the default port (22) is always ended up authenticated with password without having any other option. I have attached the complete log of both successful.txt (custom port) and unsucessful.txt (port 22) here

I welcome the possibility of "human error" and appreciate the attention and suggestion from you guys. Previously I post on the wrong forum , I believe this is the correct forum.

EDIT: I found the solution, actually the root cause, it turn out that there is a firewall/gateway enforcing the particular policy which only accept Password authentication for Port 22 in the front of the network. It has been applied on the company network which affect easier to reproduce here. It actually work as soon as the policy is taken out and by using the existing configuration.