1

Trying to connect to Windows Server 2012 VPN over L2TP.

Server configured to support L2TP using "custom IPSec policy" PSK. All authentication methods are checked off.

Clients are configured to connect with L2TP and is using the PSK for authentication. Tested with settings [Encryption is optional accepting PAP, CHAP, or MS-CHAPv2] and [Encryption is option accepting EAP-MSCHAPv2].

The server has a public IP and all traffic is being routed to it for testing purposes, nothing is being filtered out. I am using two clients, one using it's public IP from another network and one on the same subnet as my server trying to use its private IP. All clients and the server have their firewalls completely disabled. Neither client can use the VPN and both are Windows 10.

I have verified all L2TP miniports are working in Device Manager.

Connection attempt errors:

Client error: Error 789: The L2TP connection attempt failed because the security layer encountered a processing error during the initial negotiations with the remote computers.

Errors in RRAS logs: None(RRAS is configured to log all events)

Improve this question
5
  • How do you connect to the Internet? You say you have a public IP directly connected but what hardware sits between the server and your ISP connection?
    – Kinnectus
    Commented Sep 5, 2016 at 10:57
  • Additionally, have you tried using the Server 2010 VPN wizard? It should do everything for you... Finally, I would recommend you choosing one tunnel type and sticking to it... SSTP if all your clients are Windows-based because this just works (and works across problematic scenarios such as cafes, airports etc.).
    – Kinnectus
    Commented Sep 5, 2016 at 11:07
  • I've updated the post with more relevant information. Apologies for the confusion.
    – Needs More Documentation
    Commented Sep 6, 2016 at 6:29
  • vpnranks.com/how-to-fix-vpn-connection-error-789 - see the last "try this" on this link. About enabling and starting a couple of services on your clients.
    – Kinnectus
    Commented Sep 6, 2016 at 6:59
  • I restarted the RRAS service and verified the other services in the article were running. I then tried restarting those as well. Then I just rebooted the whole server. No luck yet. FYI, based on the settings I don't think I need them but in case I am wrong I do not have any certificates to identify my clients or server.
    – Needs More Documentation
    Commented Sep 6, 2016 at 7:41

1 Answer 1

Reset to default
0

Perhaps this blog entry could help. #3 below fixed this issue for me:

  • Incorrect pre-shared key: Solution: Ensure that the pre-shared secret is configured correctly on the client machine. It must match between the MX and the client.

  • Firewall blocking VPN traffic to MX: Solution: Ensure UDP ports 500 (IKE) and 4500 (IPsec NAT-T) are being forwarded to the MX and not blocked. If traffic cannot reach the MX on these ports, the connection will timeout and fail.

  • IKE and AuthIP IPsec Keying Modules disabled: Solution: This occurs most often when 3rd party VPN software has been installed and disables the IKEEXT service. This can be re-enabled by navigating in Windows to Control Panel > Administrative Tools > Services. Find the service named “IKE and AuthIP IPsec Keying Modules” and open it. Change the Startup type to “Automatic”. it may be necessary to remove the 3rd party VPN software.

In my case, I didn't have to uninstall any 3rd party VPN software.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .