SSH close connection after login

Discussion in 'Technical' started by Stelios, Dec 21, 2020.

  1. Stelios

    Stelios Member HowtoForge Supporter

    Hi all,
    This problem drives me nuts. I tried nearly everything I found on Internet.
    Only one specific ssh client running Linux Redhat ES 8 can't login to the Debian server via ssh. The rest of the clients are able to connect without a problem.

    This is the debug output from the client to the server:
    Code:
    [[email protected] ~]$ ssh -v ukserver
    OpenSSH_8.0p1, OpenSSL 1.1.1g FIPS  21 Apr 2020
    debug1: Reading configuration data /etc/ssh/ssh_config
    debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
    debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
    debug1: configuration requests final Match pass
    debug1: re-parsing configuration
    debug1: Reading configuration data /etc/ssh/ssh_config
    debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
    debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
    debug1: Connecting to ukserver [82.1.1.1] port 22.
    debug1: Connection established.
    debug1: identity file /home/ansible/.ssh/id_rsa type 0
    debug1: identity file /home/ansible/.ssh/id_rsa-cert type -1
    debug1: identity file /home/ansible/.ssh/id_dsa type -1
    debug1: identity file /home/ansible/.ssh/id_dsa-cert type -1
    debug1: identity file /home/ansible/.ssh/id_ecdsa type -1
    debug1: identity file /home/ansible/.ssh/id_ecdsa-cert type -1
    debug1: identity file /home/ansible/.ssh/id_ed25519 type -1
    debug1: identity file /home/ansible/.ssh/id_ed25519-cert type -1
    debug1: identity file /home/ansible/.ssh/id_xmss type -1
    debug1: identity file /home/ansible/.ssh/id_xmss-cert type -1
    debug1: Local version string SSH-2.0-OpenSSH_8.0
    debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9p1 Debian-10+deb10u2
    debug1: match: OpenSSH_7.9p1 Debian-10+deb10u2 pat OpenSSH* compat 0x04000000
    debug1: Authenticating to ukserver:22 as 'ansible'
    debug1: SSH2_MSG_KEXINIT sent
    debug1: SSH2_MSG_KEXINIT received
    debug1: kex: algorithm: curve25519-sha256
    debug1: kex: host key algorithm: ecdsa-sha2-nistp256
    debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
    debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
    debug1: kex: curve25519-sha256 need=32 dh_need=32
    debug1: kex: curve25519-sha256 need=32 dh_need=32
    debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
    debug1: Server host key: ecdsa-sha2-nistp256 SHA256:G6K0V8SI2U3+Ui4xceUrz2X7qLxWhbEvIhbsSD7rhPo
    debug1: Host 'ukserver' is known and matches the ECDSA host key.
    debug1: Found key in /home/ansible/.ssh/known_hosts:17
    debug1: rekey out after 4294967296 blocks
    debug1: SSH2_MSG_NEWKEYS sent
    debug1: expecting SSH2_MSG_NEWKEYS
    debug1: SSH2_MSG_NEWKEYS received
    debug1: rekey in after 4294967296 blocks
    debug1: Will attempt key: /home/ansible/.ssh/id_rsa RSA SHA256:i156I9WjRYRgZ6aQB+ZlimJPz5PWBmKEqyRpBftABhI
    debug1: Will attempt key: /home/ansible/.ssh/id_dsa
    debug1: Will attempt key: /home/ansible/.ssh/id_ecdsa
    debug1: Will attempt key: /home/ansible/.ssh/id_ed25519
    debug1: Will attempt key: /home/ansible/.ssh/id_xmss
    debug1: SSH2_MSG_EXT_INFO received
    debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
    debug1: SSH2_MSG_SERVICE_ACCEPT received
    debug1: Authentications that can continue: publickey,password
    debug1: Next authentication method: publickey
    debug1: Offering public key: /home/ansible/.ssh/id_rsa RSA SHA256:i156I9WjRYRgZ6aQB+ZlimJPz5PWBmKEqyRpBftABhI
    debug1: Server accepts key: /home/ansible/.ssh/id_rsa RSA SHA256:i156I9WjRYRgZ6aQB+ZlimJPz5PWBmKEqyRpBftABhI
    Connection closed by 82.1.1.1 port 22
    Permissions are fine as I can login from other clients without a problem:

    Code:
    [email protected]:~$ ls -la
    total 24
    drwxr-xr-x  4 ansible ansible 4096 Dec 21 07:23 .
    drwxr-xr-x 79 root    root    4096 Dec 20 20:20 ..
    -rw-------  1 ansible ansible  314 Dec 21 07:35 .bash_history
    drwx------  3 ansible ansible 4096 Dec 21 07:23 .gnupg
    drwx------  2 ansible ansible 4096 Dec 21 07:28 .ssh
    -rw-------  1 ansible ansible  884 Dec 20 20:27 .viminfo
    [email protected]:~$
    In my sshd_config I allow both public keys and passwords.

    From the client that I got the problem I cannot login at all no matter the user. I get all the time:
    Connection closed by 82.1.1.1 port 22

    The server who runs the ssh and want to connect to runs Debian 10 and the exact sshd_config that I got in many other servers where I can connect fine from the problematic host client.

    Any help is very much appreciated.
     
  2. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    The debug output seems A OK. But the connection gets closed?
    In what way is the debug output different to a successful connection from another host?
     
  3. Stelios

    Stelios Member HowtoForge Supporter

    Yes at the end is closing the connection straight away.

    This is the output from a client who connects fine:

    Code:
    MacbookPro:~ stelios$ ssh -v '[email protected]'
    OpenSSH_8.1p1, LibreSSL 2.7.3
    debug1: Reading configuration data /etc/ssh/ssh_config
    debug1: /etc/ssh/ssh_config line 47: Applying options for *
    debug1: Connecting to ukserver.mydomain.com port 22.
    debug1: Connection established.
    debug1: identity file /Users/stelios/.ssh/id_rsa type 0
    debug1: identity file /Users/stelios/.ssh/id_rsa-cert type -1
    debug1: identity file /Users/stelios/.ssh/id_dsa type -1
    debug1: identity file /Users/stelios/.ssh/id_dsa-cert type -1
    debug1: identity file /Users/stelios/.ssh/id_ecdsa type -1
    debug1: identity file /Users/stelios/.ssh/id_ecdsa-cert type -1
    debug1: identity file /Users/stelios/.ssh/id_ed25519 type -1
    debug1: identity file /Users/stelios/.ssh/id_ed25519-cert type -1
    debug1: identity file /Users/stelios/.ssh/id_xmss type -1
    debug1: identity file /Users/stelios/.ssh/id_xmss-cert type -1
    debug1: Local version string SSH-2.0-OpenSSH_8.1
    debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9p1 Debian-10+deb10u2
    debug1: match: OpenSSH_7.9p1 Debian-10+deb10u2 pat OpenSSH* compat 0x04000000
    debug1: Authenticating to ukserver.mydomain.com:22 as 'ansible'
    debug1: SSH2_MSG_KEXINIT sent
    debug1: SSH2_MSG_KEXINIT received
    debug1: kex: algorithm: curve25519-sha256
    debug1: kex: host key algorithm: ecdsa-sha2-nistp256
    debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
    debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
    debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
    debug1: Server host key: ecdsa-sha2-nistp256 SHA256:G6K0V8SI2U3+Ui4xceUrz2X7qLxWhbEvIhbsSD7rhPo
    debug1: Host 'ukserver.mydomain.com' is known and matches the ECDSA host key.
    debug1: Found key in /Users/stelios/.ssh/known_hosts:36
    debug1: rekey out after 134217728 blocks
    debug1: SSH2_MSG_NEWKEYS sent
    debug1: expecting SSH2_MSG_NEWKEYS
    debug1: SSH2_MSG_NEWKEYS received
    debug1: rekey in after 134217728 blocks
    debug1: Will attempt key: /Users/stelios/.ssh/id_rsa RSA SHA256:fs0QB6KurlvAubkgMUmPq1ieHm6oQRvQ0VxU3yMwjPk
    debug1: Will attempt key: /Users/stelios/.ssh/id_dsa
    debug1: Will attempt key: /Users/stelios/.ssh/id_ecdsa
    debug1: Will attempt key: /Users/stelios/.ssh/id_ed25519
    debug1: Will attempt key: /Users/stelios/.ssh/id_xmss
    debug1: SSH2_MSG_EXT_INFO received
    debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
    debug1: SSH2_MSG_SERVICE_ACCEPT received
    debug1: Authentications that can continue: publickey,password
    debug1: Next authentication method: publickey
    debug1: Offering public key: /Users/stelios/.ssh/id_rsa RSA SHA256:fs0QB6KurlvAubkgMUmPq1ieHm6oQRvQ0VxU3yMwjPk
    debug1: Server accepts key: /Users/stelios/.ssh/id_rsa RSA SHA256:fs0QB6KurlvAubkgMUmPq1ieHm6oQRvQ0VxU3yMwjPk
    debug1: Authentication succeeded (publickey).
    Authenticated to ukserver.mydomain.com ([82.1.1.1]:22).
    debug1: channel 0: new [client-session]
    debug1: Requesting [email protected]
    debug1: Entering interactive session.
    debug1: pledge: network
    debug1: client_input_global_request: rtype [email protected] want_reply 0
    debug1: Remote: /home/ansible/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
    debug1: Remote: /home/ansible/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
    debug1: Sending environment.
    debug1: Sending env LC_ALL = en_US.UTF-8
    debug1: Sending env LC_CTYPE = UTF-8
    Linux ukserver.mydomain.com 4.19.0-12-amd64 #1 SMP Debian 4.19.152-1 (2020-10-18) x86_64
    
    The programs included with the Debian GNU/Linux system are free software;
    the exact distribution terms for each program are described in the
    individual files in /usr/share/doc/*/copyright.
    
    Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
    permitted by applicable law.
    Last login: Mon Dec 21 13:02:29 2020 from 178.147.184.171
    [email protected]:~$ 
     
  4. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    Have you run tail -f /var/log/auth.log while trying that login that fails? Any further info there?
    Maybe someone familiar with SSH authentication knows why connection is closed right after "Server accepts key:"
    Is this the same key that works from other hosts?
     
  5. Stelios

    Stelios Member HowtoForge Supporter

    This is what auth.log shows, doesn't help either.
    Code:
    Dec 21 19:22:01 ukserver CRON[15365]: pam_unix(cron:session): session opened for user root by (uid=0)
    Dec 21 19:22:01 ukserver CRON[15365]: pam_unix(cron:session): session closed for user root
    Dec 21 19:22:05 ukserver sshd[15368]: Accepted publickey for ansible from 178.147.184.171 port 36136 ssh2: RSA SHA256:i156I9WjRYRgZ6aQB+ZlimJPz5PWBmKEqyRpBftABhI
    Each server has a different key. I tried generating a new key as well but same problem. It doesn't want to connect from this bloody client.
     
  6. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    Any security tools in play, eg. we used to use denyhosts which added hosts to /etc/hosts.deny (or hosts.allow or whatever you setup), and that behaved somewhat similarly, where you actually did get a connection but it was dropped right away (I don't know if it got to key exchange, offhand). Maybe search /etc/ for the client's ip address?

    Any network level firewall?

    You could try running strace (or ltrace) on the ssh server when you connect and see what you find out.
     
  7. Stelios

    Stelios Member HowtoForge Supporter

    @Jesse Norell from the same client I can login to many servers without a problem so there isn't any firewall or other rule that blocks the connection to the server. It reach the server, it authenticate and then drops the connection.
    On the server that rejects the connection I tried with firewall disabled too and same problem.
    There is no rejection at all from the client IP as from another client (VM) I can login fine with the same user. It is on a specific VM (client) that I can't login to the server. Both clients are under the same public IP as both are hosted in my computer. Client (VM) A gets the error and the other is working fine.
     
    Last edited: Dec 23, 2020
  8. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    I'd check my resources if I were facing this. Ram, its usages and having enough swap to compensate in its absence.
     
  9. Stelios

    Stelios Member HowtoForge Supporter

    There are 20GB left so there isn't any hardware issue.
     
  10. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

  11. Stelios

    Stelios Member HowtoForge Supporter

    If you read my post above I'm saying that I can login from another pc fine on the same account so there is no problem with the bash_profile.
     
  12. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Sorry, I must have missed that. So, nothing in the Redhat ES 8 that could possibly stop your ssh connection with the Debian server immediately after login?
     
  13. Stelios

    Stelios Member HowtoForge Supporter

    It is a plain installation of Redhat ES with nothing to drop connection as I can login from that server to many other servers without a problem.
     
  14. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Well now it just became another one of year 2020 mysteries that remain to be resolved.

    Happy new year 2021. ;D
     

Share This Page