Everyone wants their server to be as secure as possible, but there are many different approaches that can be taken to accomplish this end. Now Microsoft didn't invent IP Security (or IPSec for short), it was developed by them in conjunction with Cisco and the IETF, but Windows 2000 has a very robust implementation of it built in. The intent of IPSec is to help in creating secure connections between different machines, even when the software that's communicating has no knowledge of the encryption. IPSec can also be used to apply rules as to what kind of IP traffic a machine will accept, akin to a limited firewall - that part of IPSec is what this article deals with. I'm going to assume that if you're reading this, you already have Win2k installed and know how to get to the IPSec administration portion - if you don't, please check the additional resources section.

    Let me start out by saying that IPSec isn't what I consider to be firewall, but it can do many of the same operations that make up the core of what a firewall does. In a perfect world your server would be located behind a firewall AND have IPSec installed - the more layers of protection you can have on your network, the better.
    At its simplest, a firewall simply allows or denies traffic on a specific port or from a specific IP. In this way you can ensure that only traffic you have intentionally allowed will connect, this is important because in many cases servers are running several services that will accept connections (such as Windows Networking) that probably have no business running on the server. If the firewall is configured to only allow HTTP (port 80) traffic through, then it doesn't matter if you're sharing all your drives without even a password (although I would highly recommend against it! :) ), the Windows Networking server is essentially invisible.
    IPSec offers the same level of control - you can specify a specific port and/or IP address and then which rule should apply to any traffic that matches (permit, deny, permit but only on secured connection, etc).

    As many of you may be aware, each network interface also has TCP/IP filtering as an advanced option. TCP/IP filtering is similar to a stripped down version IPSec, but fails to be terribly useful for a couple of reasons. First it only views packets from the server's point of view, in most cases this would be fine but it fails miserably in others. For example, if you choose to disable all UDP connections but allow port 53 (used for DNS), people will still be able to connect to your server and make DNS queries (if you're running a DNS server), but you will be unable to make any DNS queries outside of your machine! Why? Simple, you've only allowed port 53 traffic, which would be fine if you were using only that port, but most client network applications use a randomly assigned network port which, guess what, is blocked in your configuration. This doesn't affect HTTP because it uses TCP as its transport, but with IPSec you have the flexibility to say "allow traffic with a source port OR destination port of 53". Another major disadvantage of TCP/IP filtering is that it requires a reboot each time it's changed, where IPSec never requires a reboot whether you're turning on or off a specific filter or even IPSec altogether.

    Rules in IPSec are handled in a very simple approach, the closest match to the incoming packet determines how it is handled overriding everything above it. So if you have two rules, one that denies all TCP traffic and another that permits HTTP traffic (port 80 TCP), then your server is effectively blocking everything that doesn't connect on port 80. Running an FTP server on the same machine? Not anymore... If you want it accessible, then you need to add another rule permitting FTP traffic (port 20/21 TCP).

    I've taken the liberty of creating an IPSec template which should do everything most people will want to and can easily be customized for anything else that might be needed. All of the filters have both client and server configuration and are pretty self-explanatory, if you put a check next to "APS - HTTP Server" it will allow people to connect to a web server running on your machine. If you put a check next to "APS - HTTP Client" it will allow you to connect to other web servers on the network - check them both and it will allow both, very simple. Here's a list of what is available by default (everything is turned off, except the Terminal Services Server and DNS Client):

Application Abbrev Port Protocol Configuration Domain Name File Transfer Web Secure Web Receiving Email Sending Email Network Management Network Time News Database Telnet Terminal Services Windows Networking DHCP DNS FTP HTTP HTTPS POP3/IMAP SMTP SNMP NTP NNTP SQL Telnet TermServ WinNet 67/68 53 20/21 80 443 110/143 25 161/162 123 119 1433 23 3389 137-139/445 TCP/UDP TCP/UDP TCP TCP TCP TCP TCP UDP UDP TCP TCP TCP TCP TCP/UDP

    Before you go hog-wild and start rampaging through your registry, I HIGHLY recommend that you backup your current IPSec configuration. That's not to say that I've even run into any problems with installing this into any machines, but it's better to be safe than sorry plus it's very easy to do. To backup your current IPSec configuration, open up regedit (Start Menu->Run then type in regedit) and navigate to the following location:

HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\IPSec

Now make sure IPSec is selected, then export all the keys (from the menu select Registry->Export Registry File...); this will create a .reg file with everything you had configured prior to this (it will probably be around 200-300k). Stash the export file you've just made someplace safe in case you decide at some point you'd like to restore back to the previous configuration. Keep in mind that in order to do this, you must delete everything from the key above, and then import the original .reg file - otherwise nothing will change.

    If you're followed the steps in "Playing it safe", then you probably already have a good idea of what's coming. Click on the following link and download my IPSec configuration:

AnalogX Public Server IPSec Configuration v1.00 (29k zip file)
Once you've downloaded it and unzipped it, you'll have a file called "APS-IPSec.reg", just double-click on the file from the Windows Explorer, it will prompt you to verify whether or not you want to install the registry keys, click Yes and then you're done - it's installed! Of course, it's not actually running yet, you'll need Assign it from the IPSec Policy manager to start it up, but you should edit it first to turn on or off any rules that are appropriate for your server
    If everything went smoothly installing then you should have "AnalogX Public Server" in your IPSec Policies, doubleclick on it to bring up the listing of defined/applied rules. To keep things clean and simple to distinguish, all rules and filters I created start with "APS - ", by default all the rules are configured but not enabled - to enable a rule simply check the box next to its name. For example, if you run an SQL server on your system, make sure the box next to "APS - SQL Server" is checked, and if you make SQL queries from this machine to some other server make sure to check the box next to "APS - SQL Client". Although all the rules have Kerberos listed next to them, the Filter Actions "APS - Permit" and "APS - Deny" are set to not use any form of encryption, so it's unused.

    All of the rules in my configuration rely on three overall rules, which IPSec uses when it doesn't have any better match. These rules deal with things on the protocol level, specifically with TCP, UDP, and ICMP. In practice, you'll probably never want to turn off the global denial of TCP or UDP packets that you receive - this ensures that you'll need to explicitly list every type of communications you wish to allow on your machine. The only protocol that you may or may not want to allow is ICMP. For those of you unfamiliar with ICMP, at its most basic it's the protocol that is used by Ping and traceroute; if you choose to disable ICMP, then no one will get a ping or traceroute response from your machine, but also you will be unable to ping or traceroute FROM your machine (since neither the outgoing request nor the incoming response will be allowed).

    Another advantage of IPSec is that you can modify a rule to only allow a specific IP address. Why would you want to do that? A simple example would be for something like an FTP server - say you need to run an FTP server on your server but don't want anyone else to access it, or to even know that it's running. Select the "APS - FTP Server" rule and click Edit, then select the same from the Filter List and choose Edit again, now you're in its criteria for a match. Select "FTP Listen" and choose Edit; now change the Destination Address from "Any IP Address" to "A specific IP Address", type in your IP address and voila - you're done! Now your IP is the only one that can connect to the FTP server, or even tell that there's one listening. Of course this is only applicable when you've got a static IP, but you can easily use "A specific IP Subnet" instead and select the IP range your ISP has; while not as good as binding to a specific IP, it's better than open to the world.

    There is a decent amount of information around the net concerning IPSec, both for the beginner and the advanced. Here's a couple of the resources that I found particularly useful:

Secure IIS 5 Checklist

This should be your FIRST stop when installing a machine with IIS - most of what it contains is pretty common sense, but it's always good to have a detailed list to go through to make sure you don't miss anything. The security template they list is also a must.

How to enable IPSec on a machine

So you've follow the checklist and now you're want to know HOW to actually get IPSec running - here's a quick, clear page showing you exactly what you need to do (including pictures).

How to manage IPSec policies

IPSec's running fine on your system, but you're feeling the need to start doing some more trick things, here's another quick page showing you how to add IPSec to your management console to help keep things under control.

Using IPSec to secure communications between two machines

Maximum security - engaged! A relatively straight-forward description of how to configure IPSec to encrypt communications between two computers, regardless of whether the software applications you're using support encryption.

IPSec resources for Windows 2000

The most comprehensive list of IPSec resources I've found on the net - links to Microsoft knowledgebase topics, magazine articles, websites, and just about anything else related to IPSec.

    There you have it - you're now an expert in IPSec! Well, not really, but you at least hopefully understand the framework that IPSec is built around. At the very least you got a cool IPSec configuration that you can have up and running on your machine in just a few minutes, giving you significantly more security and control then you would have had without. If you have any questions or comments, please don't hesitate to send them to me.