Setting up IPsec with Kerberos on Server 2012 R2

Ok, so not strictly an SCCM related post but it might be helpful to many people struggling to setup IPsec.

We have recently been asked to ensure that all our SCCM 2012 servers communicate with each other using IPsec. Now, anyone who has tried to do this, having never done it before, will know there isn’t that much online on how to do this. At first hand it seems quite complex but actually its pretty easy to do.

I have only ever done this on Server 2012 R2, so I’ve no idea if this same process will work on earlier versions (give it a go and let me know!). Also, this is using Kerberos as the authentication method only (so obviously can only be done in a domain environment) and requesting IPsec, rather than forcing it (meaning non-IPsec computers can still communicate with the servers, albeit without encryption). Here is how we implemented it:

  1. Login with an admin account, open up Services and ensure the IKE and AuthIP IPsec Keying Modules service is running and set to start automatically.
  2. Open Windows Firewall with Advanced Security, right-click over Connection Security Rules node and select New Rule….
  3. In the New Connection Security Rule Wizard, on the Rule Type page, select the Isolation radio button and click Next.
  4. On the Requirements page, select the Request authentication for inbound and outbound connections radio button and click Next.
  5. On the Authentication Method page, select the Computer (Kerberos V5) radio button and click Next.
  6. On the Profile page, leave as default and click Next.
  7. On the Name page, give the rule a sensible name and click Finish.

That’s it! Now obviously you’ll want to test its working and I’m sure you can use some sort of network monitor to do this but the easiest way is to use the tools in the firewall itself.

Simply set this up on 2 servers, have them ping each other (this is important as its starts the IPsec connection) and then go into Windows Firewall with Advanced Security again, expand the Monitoring node, then expand the Security Associations node and click on the Main Mode node (you may need to hit F5 to refresh the node). In the right window you should see any IPsec connections, along with the authentication method and encryption type:

IPsec away!