SSH config option to use ProxyJump gateway's `/etc/hosts` lookup?

Question

I'm attempting a gateway hop through servers A -> B -> C, and would like to have this connection as an entry in my SSH config.

SSHing from A into B works. And then from B to C works. As does ssh B -t ssh C.

However, when I attempt use the following SSH config file, it fails.

Host B
    Hostname B

Host C
    Hostname C
    ProxyJump B

Host *
    User username
    ForwardAgent yes
    PKCS11Provider /usr/lib/ssh-keychain.dylib

When running this verbosely, I find I'm running into a problem with

debug1: getpeername failed: Bad file descriptor

This answer seems to suggest that problem arises from the lookup for C not being found (namely, inside the /etc/hosts file). When I look at the contents of /etc/hosts on B, all the final host locations I would like to connect to (including C) are listed. So I believe I want my connection to use B's /etc/hosts listing when making the final connection. Is there a way I can specify this in A's SSH config?

Note, I do not have root permissions on any of the machines (A, B, nor C).

Debug log:

username@A ~ % ssh C -v  
OpenSSH_7.9p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/username/.ssh/config
debug1: /Users/username/.ssh/config line 14: Applying options for C
debug1: /Users/username/.ssh/config line 19: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Setting implicit ProxyCommand from ProxyJump: ssh -v -W '[%h]:%p' B
debug1: Executing proxy command: exec ssh -v -W '[C]:22' B
OpenSSH_7.9p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/username/.ssh/config
debug1: /Users/username/.ssh/config line 19: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to B [111.111.111.111] port 22.
debug1: Connection established.
debug1: provider /usr/lib/ssh-keychain.dylib: manufacturerID <Apple, Inc.> cryptokiVersion 2.20 libraryDescription <Keychain emulation PKCS#11 API> libraryVersion 0.0
debug1: provider /usr/lib/ssh-keychain.dylib slot 0: label <Key For PIV Authentication (Use> manufacturerID <Apple, Inc.> model <Keychain> serial <000000> flags 0x404
debug1: have 1 keys
debug1: provider /usr/lib/ssh-keychain.dylib slot 1: label <Key For Digital Signature (User> manufacturerID <Apple, Inc.> model <Keychain> serial <000000> flags 0x404
debug1: have 2 keys
debug1: identity file /Users/username/.ssh/id_rsa type -1
debug1: identity file /Users/username/.ssh/id_rsa-cert type -1
debug1: identity file /Users/username/.ssh/id_dsa type -1
debug1: identity file /Users/username/.ssh/id_dsa-cert type -1
debug1: identity file /Users/username/.ssh/id_ecdsa type -1
debug1: identity file /Users/username/.ssh/id_ecdsa-cert type -1
debug1: identity file /Users/username/.ssh/id_ed25519 type -1
debug1: identity file /Users/username/.ssh/id_ed25519-cert type -1
debug1: identity file /Users/username/.ssh/id_xmss type -1
debug1: identity file /Users/username/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9
debug1: provider /usr/lib/ssh-keychain.dylib: manufacturerID <Apple, Inc.> cryptokiVersion 2.20 libraryDescription <Keychain emulation PKCS#11 API> libraryVersion 0.0
debug1: provider /usr/lib/ssh-keychain.dylib slot 0: label <Key For PIV Authentication (Use> manufacturerID <Apple, Inc.> model <Keychain> serial <000000> flags 0x404
debug1: have 1 keys
debug1: provider /usr/lib/ssh-keychain.dylib slot 1: label <Key For Digital Signature (User> manufacturerID <Apple, Inc.> model <Keychain> serial <000000> flags 0x404
debug1: have 2 keys
debug1: identity file /Users/username/.ssh/id_rsa type -1
debug1: identity file /Users/username/.ssh/id_rsa-cert type -1
debug1: identity file /Users/username/.ssh/id_dsa type -1
debug1: identity file /Users/username/.ssh/id_dsa-cert type -1
debug1: identity file /Users/username/.ssh/id_ecdsa type -1
debug1: identity file /Users/username/.ssh/id_ecdsa-cert type -1
debug1: identity file /Users/username/.ssh/id_ed25519 type -1
debug1: identity file /Users/username/.ssh/id_ed25519-cert type -1
debug1: identity file /Users/username/.ssh/id_xmss type -1
debug1: identity file /Users/username/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to B:22 as 'username'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes128-ctr MAC: [email protected] compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: [email protected] compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:XXXXXXXXXXXXXXXXXXXXXXX
debug1: Host 'B' is known and matches the ECDSA host key.
debug1: Found key in /Users/username/.ssh/known_hosts:5
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 4294967296 blocks
debug1: Will attempt key: /usr/lib/ssh-keychain.dylib RSA SHA256:YYYYYYYYYYYYYYYYYYYYYYYYY token agent
debug1: Will attempt key: /usr/lib/ssh-keychain.dylib RSA SHA256:ZZZZZZZZZZZZZZZZZZZZZZZZZ token agent
debug1: Will attempt key: /Users/username/.ssh/id_rsa 
debug1: Will attempt key: /Users/username/.ssh/id_dsa 
debug1: Will attempt key: /Users/username/.ssh/id_ecdsa 
debug1: Will attempt key: /Users/username/.ssh/id_ed25519 
debug1: Will attempt key: /Users/username/.ssh/id_xmss 
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering public key: /usr/lib/ssh-keychain.dylib RSA SHA256:YYYYYYYYYYYYYYYYYYYYYYYYY token agent
debug1: Server accepts key: /usr/lib/ssh-keychain.dylib RSA SHA256:YYYYYYYYYYYYYYYYYYYYYYYYY token agent
debug1: pkcs11_provider_unref: 0x111111111111 refcount 3
debug1: pkcs11_provider_unref: 0x111111111111 refcount 2
debug1: Authentication succeeded (publickey).
Authenticated to B ([111.111.111.111]:22).
debug1: channel_connect_stdio_fwd C:22
debug1: channel 0: new [stdio-forward]
debug1: getpeername failed: Bad file descriptor
debug1: Requesting [email protected]
debug1: Entering interactive session.
debug1: pledge: exec
debug1: client_input_global_request: rtype [email protected] want_reply 0
debug1: Remote: Agent forwarding disabled.
debug1: Remote: Port forwarding disabled.
debug1: Remote: User rc execution disabled.
debug1: Remote: Agent forwarding disabled.
debug1: Remote: Port forwarding disabled.
debug1: Remote: User rc execution disabled.
channel 0: open failed: administratively prohibited: open failed
stdio forwarding failed
ssh_exchange_identification: Connection closed by remote host

/etc/ssh/ssh_config

Host *
    SendEnv LANG LC_*
    ForwardX11 yes
    ForwardX11Trusted yes
    XAuthLocation /opt/X11/bin/xauth

Try using ProxyCommand instead of JumpProxy. See this article for more info. — harrymc, Commented Feb 10, 2020 at 7:46
@harrymc, I've attempted ProxyCommand ssh B -W %h:%p as an alternative to JumpProxy B, but this results in the same issue. Are you recommending any specific variation of the ProxyCommand parameters? — golmschenk, Commented Feb 10, 2020 at 16:50
More information may help. Try adding a debug log here using ssh -v. — harrymc, Commented Feb 10, 2020 at 16:56
Do you mean that you have tried the fixes in the post SSH tunneling error and they didn't help? Could you add to the post the files /etc/ssh/ssh_config and ~/.ssh/config (if it exists)? — harrymc, Commented Feb 12, 2020 at 7:17

Marco Garcês · Accepted Answer · 2020-02-12 15:56:27Z

1

When you connect to any of the hosts, the lookup is done on your localhost, not remote, so I would suggest to forget /etc/hosts and be sure to have HostName with the proper IP/DNS for host C.

Also, are you using for jump:

    ProxyCommand ssh -W %h:%p B

I noticed you were using it the other way around (in your comments)

answered Feb 12, 2020 at 15:56

Marco Garcês

112 bronze badges

Do you have any specific recommendations for how to obtain the "proper IP/DNS"? I have attempted the each of fully resolved names and the IP for C in B's /etc/hosts (by using them as the Hostname in A's config). As for the ProxyCommand argument order, I don't believe it should have an impact, but I tried the swapped ordering (as listed in your post) and received the same error.
– golmschenk
Commented Feb 12, 2020 at 19:24
my command is different, I'm using ProxyCommand which seems like the old way to do it; you show you want to use ProxyJump although you have JumpProxy (which is wrong). So either ProxyJump or ProxyCommand should work; also, let's says that: ` - HostA = 10.10.10.1 - HostB = 10.10.10.2 - HostC = 10.10.10.3 ` you want to reach HostC via Host B, just use ` Host B HostName 10.10.10.2 Host C HostName 10.10.10.3 ProxyJump HostB ` this should work; because the "proper IP" of C is being read on host A, and passed as an argument to host B, and then jump to host C.
– Marco Garcês
Commented Feb 17, 2020 at 17:08
Sorry, JumpProxy was just a typo in my example config file for the question. I have been using ProxyJump in the real case. I also attempted ProxyCommand as you noted (with both orderings of the parameters, and using both IP addresses or domain name host names). The issue given in the question occurs in all cases.
– golmschenk
Commented Feb 17, 2020 at 17:20
I have other hosts with intermediate gateway servers configured identically to the above, except with different targets and gateway servers (e.g., A -> D -> E). In these cases, the configuration successfully connects. In these cases, both the ProxyCommand and ProxyJump options work. It is due to the specific configuration of the host resolution with B and C that there is an issue, and this is what I'm looking for a solution to.
– golmschenk
Commented Feb 17, 2020 at 17:29
Just to note, we can see in the debug log that the ProxyCommand/ProxyJump is successfully connecting to the appropriate intermediate host. It is only during the resolution of C from B that there is an issue.
– golmschenk
Commented Feb 17, 2020 at 17:40

Add a comment |

harrymc · Accepted Answer · 2020-02-17 19:01:00Z

1

+50

In the lack of any other explanation, I believe that the problem is that you are missing a reverse dns entry, perhaps via a line in /etc/hosts.

I would suggest updating the /etc/hosts on all the three computers, so they all know about each other.

answered Feb 17, 2020 at 19:01

harrymc

1

Without permissions to /etc/hosts this may be difficult. However, with this method of specifying a user level hosts file it might be doable. Unfortunately, I won’t be back at my work station until after the bounty grace period. But as it’s most likely the correct solution, and the rest of your effort walking through the options (in the comments), I will be giving this answer the bounty.
– golmschenk
Commented Feb 18, 2020 at 1:02

Add a comment |

bey0nd · Accepted Answer · 2020-02-16 21:43:22Z

I have the following config running here:
~/.ssh/config (of my_host)

Host C
    HostName            192.168.3.1     # ip of Host C
    ProxyCommand        ssh -W %h:%p B
    ServerAliveInterval 60

Host B
    HostName            192.168.2.1     # ip of Host B
    ProxyCommand        ssh -W %h:%p A
    ServerAliveInterval 60

Host A
    HostName            192.168.1.1     # ip of Host A
    ServerAliveInterval 60

This provides for a connection like this:
'my_host' -> 'Host A' -> 'Host B' -> 'Host C'

If you use DNS-Names instead of the ip-addresses, then you need to make sure that each host can resolve the DNS-Name of the next host.
-> your host can resolve the DNS-Name of Host A
-> Host A can resolve the DNS-Name of Host B
-> Host B can resolve the DNS-Name of Host C

If you only need one hop and Host A of your example is your work station, then the following /ssh/config of Host A should do:

Host C
    HostName            192.168.3.1     # ip of Host C
    ProxyCommand        ssh -W %h:%p B
    ServerAliveInterval 60

Host B
    HostName            192.168.2.1     # ip of Host B
    ServerAliveInterval 60

A is the local machine. A can resolve B (during the first SSH command from A). B can resolve C (in the second SSH command from B). The problem arises when executing the single command with the Jump or ProxyCommand. Replacing the target the Hostname with the IP address results in the same issue. — golmschenk, Commented Feb 17, 2020 at 2:28
Can you pls. move PKCS11Provider /usr/... from Host * to Host B and try again? — bey0nd, Commented Feb 17, 2020 at 17:46

bey0nd · Accepted Answer · 2020-02-19 11:44:06Z

0

After some research, I've to change my answer.

I think your problem doesn't have anything to do with
debug1: getpeername failed: Bad file descriptor
as this line also appears in my working configuration.

debug1: channel_connect_stdio_fwd target:22
debug1: channel 0: new [stdio-forward]
debug1: getpeername failed: Bad file descriptor
debug1: Requesting [email protected]
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype [email protected] want_reply 0

I'm not sure yet of the 'pledge: network' vs. 'pledge: exec'

Looking at the messages to the end of log

debug1: Remote: Agent forwarding disabled.
debug1: Remote: Port forwarding disabled.
debug1: Remote: User rc execution disabled.
debug1: Remote: Agent forwarding disabled.
debug1: Remote: Port forwarding disabled.
debug1: Remote: User rc execution disabled.

What happens if you comment out ForwardAgent yes in ~/.ssh/config and ForwardX11 yes, ForwardX11Trusted yes as well as XAuthLocation /opt/X11/bin/xauth in _/etc/ssh/ssh_config_?

edited Feb 19, 2020 at 11:44

answered Feb 19, 2020 at 8:31

bey0nd

4552 silver badges8 bronze badges

I do not have root access, and so cannot edit /etc/ssh/ssh_config. However, by overriding these settings in my own config file using ForwardAgent no, ForwardX11 no, ForwardX11Trusted no, and XAuthLocation /usr/local/bin/xauth, then I get the same result, but with pledge: network instead of pledge: exec.
– golmschenk
Commented Feb 19, 2020 at 21:53
1

"SSHing from B into A works." Is that really from B to A? "And then from B to C works." After you SSHed from B to A? That sounds like B -> A -> C Also it would be interesting to know what configuration your user on B uses to connect to C. If I remember correctly then on a forwarded connection all config is done on the first system. Therefore if B can connect to C and A can not connect to C via B, then either B doesn't allow a forwarded connection or there is a difference between the config on A and B.
– bey0nd
Commented Feb 20, 2020 at 7:00
Ah, sorry! That was a typo (I'm slightly amazed at how many typos I had in this question). This should say "SSHing from A into B works". I have corrected it now. Thank you for pointing this out.
– golmschenk
Commented Feb 20, 2020 at 16:09
Then, if it works from A to B and from B to C but not fromA to C via B, we should compare the ssh config and parameters used by A and B. Maybe that shed a light on the problem.
– bey0nd
Commented Feb 20, 2020 at 16:59

Add a comment |

Kevin Keane · Accepted Answer · 2020-11-10 17:42:18Z

0

I had this exact problem, as well.

In my case, name resolution worked fine exactly as expected.

The cause turned out to be that TCP port forwarding was disabled on the intermediate hosts. The solution thus was to change AllowTcpForwarding to yes in the intermediate host's /etc/ssh/sshd_config

In my case, I had several Match clauses in sshd_config file, so I had to change the AllowTcpForwarding in multiple places.

answered Nov 10, 2020 at 17:42

Kevin Keane

9703 gold badges9 silver badges19 bronze badges

Add a comment |

Tomek · Accepted Answer · 2020-11-10 20:14:42Z

Here is the problem:

debug1: Remote: Port forwarding disabled.

According to ssh man page -J / ProxyJump works in the following way:

Connect to the target host by first making a ssh connection to
the jump host described by destination and then establishing a
TCP forwarding to the ultimate destination from there.

This basically means that the jump server must allow TCP forwarding for this option to work which is not the case here.

One possible workaround would be to use socat or netcat on the jump host to forward the connection to target host. Something along the lines:

Host C
    Hostname C
    ProxyCommand ssh B socat STDIO TCP:%h:%p

Enderman · Accepted Answer · 2023-11-15 20:40:27Z

0

In lack of root access, you can use only ssh config. Why not set C's IP in the ssh config on A?

Host B
    Hostname B

Host C
    Hostname <ip of C>
    JumpProxy B

Host *
    User username
    ForwardAgent yes
    PKCS11Provider /usr/lib/ssh-keychain.dylib

edited Nov 15, 2023 at 20:40

Enderman

1138 bronze badges

answered Feb 16, 2020 at 2:39

Mikhail Moskalev

2,09616 silver badges14 bronze badges

I have attempted using the IP of C in place of the name of C, but this results in the same issue.
– golmschenk
Commented Feb 17, 2020 at 2:22

Add a comment |

Stack Exchange Network

SSH config option to use ProxyJump gateway's `/etc/hosts` lookup?

7 Answers 7

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged
ssh
.

Linked

Hot Network Questions

SSH config option to use ProxyJump gateway's `/etc/hosts` lookup?

7 Answers 7

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged ssh.

Linked

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
ssh
.