Internet Protocol Security (IPSec)

  • Checklist: Defining IPSec policies
  • Best practices
  • How to
  • Concepts
  • Troubleshooting

Note

  • IPSec, Internet Key Exchange (IKE), and related services for Windows 2000 are jointly developed by Microsoft Corporation and Cisco Systems, Inc.


Checklist: Defining IPSec policies

Step | Reference
Establish a security plan. | Establishing an IPSec security plan
Review IP Security policy concepts. | IPSec policy properties
Start IP Security Policy Management. | To start IP Security Policy Management
Define an IPSec policy for each scenario in your enterprise security plan. | To define IPSec policies
Assign IPSec policies. | To assign IPSec policy


Best practices

Planning an effective implementation

  • Evaluate the type of information being sent over your network. Is it sensitive financial data, proprietary information, or electronic mail? Because of their function, some departments may require a higher level of security than the majority of the enterprise.
  • Determine where your information is stored, how it is routed through the network, and from what computers it will be accessed. This provides information about the speed, capacity, and utilization of the network prior to implementation, which is helpful for performance optimization.
  • Evaluate your vulnerability to network attacks.
  • Design and document an enterprise-wide network security plan. Take into account the security framework of Windows, including the Active Directory model, and how security is applied to Group Policy. For more information, see Active Directory overview.

Consider:

  • What you should secure? Should you secure traffic between some computers or all computers; only certain protocols or ports?
  • How should you secure it? Should you secure the traffic with integrity only or confidentiality also, and at what strength?
  • Where should you secure it? Should you secure it just over remote access connections, or also through the local area network?
  • Who will manage policy? Should domain administrators, server administrators, or local computer administrators?
  • Will encryption settings work with all pertinent computers? Will data be accessed by computers using strong cryptography (3DES encryption) and those with standard cryptography?

In addition:

  • Design, create, and test the IPSec policies, to clarify and refine what policies and policy structures are truly necessary. During testing of your deployment scenarios, run normal workloads on applications to gain realistic feedback. During initial tests, if you want to view the packet contents with Network Monitor or a sniffer, use the Medium security method level or a custom security method set to AH, since using High or ESP will prevent viewing of the packet.
  • Reduce administrative overhead spent on policy by using the predefined policies, rules, and filter actions whenever possible. They can be activated, modified, or used as a template for defining your own.

For information on security planning, see Establishing an IPSec security plan. For information on special requirements for deploying IPSec, see Special IPSec considerations.


How to ...

  • Start IP Security Policy Management
  • Define IPSec policies
  • Assign IPSec policy
  • Manage IPSec policy


Start IP Security Policy Management

To start IP Security Policy Management

  1. Click Start, click Run, type MMC, and then click OK.
  2. In MMC, click Console, click Add/Remove Snap-in, and then click Add.
  3. Click IP Security Policy Management, and then click Add.
  4. Select the computer for which you want to manage IPSec policies:
    5. To | Do this
Only manage the computer on which this console is running | Click Local Computer.
Manage IPSec policies for any domain members | Click Manage domain policy for this computer's domain.
Manage IPSec policies for a domain of which the computer that is running this console is not a member | Click Manage domain policy for another domain.
Manage a remote computer | Click Another computer.
  5. Click Finish, click Close, click OK, and then click Console and then Save to save the console settings.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To view the saved console, click Start, point to Documents, and then click My Documents.
  • If you need to revise your initial choice, you must start the Microsoft Management Console, add this snap-in again, and save the console again. (If you require multiple configurations, you should save it under another name.)

Related Topics


Define IPSec policies

  • Add or edit IPSec policies
  • Define rules
  • Define advanced IPSec settings


Add or edit IPSec policies

To add or edit IPSec policies

  1. In IP Security Policy Management, choose whether to define a new policy or edit a current one:
    2. To | Do this
Create a new policy | In the console tree, click IP Security Policies on Description, then on the Action menu, click Create IP Security Policy. Complete the instructions in the IP Security Policy wizard until the Properties dialog box for your new policy appears.
Edit an existing policy | Right-click the policy, and then click Properties.
  2. Click the General tab, then in Name, enter a unique name.
  3. In Description, type a description of the security policy, such as which groups or domains it affects.
  4. If this computer is part of a domain, to specify how often the policy agent will check Group Policy for updates, type a value in Check for policy changes every number minute(s).
  5. If you have special requirements for the security on the key exchange, click Advanced.
  6. Click the Rules tab and create any necessary rules for the policy.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • To configure advanced settings, see Related Topics.
  • To configure Rules, see Related Topics.
  • If you are creating a new policy, and the new policy is not displayed in the console tree, right-click the IP Security Policies on Description folder and then click Refresh.

Related Topics


Define rules

  • Add or edit rules
  • Activate rules
  • Define IPSec authentication methods
  • Specify IPSec connection types
  • Specify an IPSec tunnel
  • Define IP filter lists
  • Define filter actions


Add or edit rules

To add or edit rules

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Decide whether or not you want to use the Add wizard:
    • To use the Security Rule wizard to add a rule, confirm that the Use Add Wizard check box is selected, click Next, and then follow the instructions.
    • To add or edit a rule manually, confirm that the Use Add Wizard check box is cleared, and then click Add or Edit and continue with this procedure.
  3. Define the IP Filter List, Filter Action, Connection Type, Authentication Methods, and Tunnel Setting properties.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • New rules are automatically applied to the policy being edited or created.
  • The Default Response rule is automatically added (and activated if you so choose) in every new IPSec policy. If you do not want this rule to be part of your policy, deactivate it. (Predefined rules cannot be removed.)

Related Topics


Activate rules

To activate rules

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Select the check box next to any rules you want to activate, or clear the check box to deactivate a rule.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Define IPSec authentication methods

To define IPSec authentication methods

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the rule you want to modify, and then click Edit.
  3. On the Authentication Methods tab, click Add. Or, if you are reconfiguring an existing method, click the authentication method, and then click Edit.
  4. Define authentication methods:
    • Click Windows 2000 default to use the Kerberos V5 security protocol for authentication services, if this rule applies to computers that are validated by a trusted Windows 2000 domain.
    • To use a public key certificate for authentication services, click Use a Certificate from this certificate authority (CA).
    • If you choose to use a certificate, click Browse to select a root CA.
    • To specify your own key that will be used for authentication, click Use this string to protect the key exchange (preshared key).

Important

  • The authentication key value is stored inside the IPSec policy in a readable format. By default, IPSec policy stored in Active Directory can be read by any authenticated user, but changed only by directory administrators. Therefore, specifying an authentication key value in directory-based policy is not a good security practice. Use authentication keys only when IPSec policy is stored and managed locally on the computer to which it applies. The IPSec Policy Management snap-in must be run on a domain controller to securely manage IPSec policy stored in Active Directory.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • If you choose to use a certificate for authentication, you must select a CA. You cannot leave this field blank.
  • Each peer in the communication must use the same key for authentication, or authentication will fail.
  • To delete the selected security method, click Remove.
  • To move the selected security method up one level, click Move Up. Repeat until the security method is at the desired preference level.
  • To move the selected security method down one level, click Move Down. Repeat until the security method is at the desired preference level.
  • To open IP Security Policy Management, see Related Topics.
  • If filters in more than one rule apply to traffic between the same two IP addresses, then the authentication methods list should be identical in both rules. If not, then security requests initiated by one rule may match the filter of the other rule, yet have a different authentication method. In this case, the negotiation will fail. Once a rule has initiated a successful Internet Key Exchange (IKE) security association between two IP addresses, other rules that are filtering different traffic will not renegotiate the authentication and master key (the IKE security association). Instead, these other rules will use the same IKE security association to negotiate security (algorithms and session keys) for traffic that matches their particular filter.

Related Topics


Specify IPSec connection types

To specify IPSec connection types

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the rule you want to modify, and then click Edit.
  3. On the Connection Type tab, select the type of network connections to which this rule will apply:
    • To apply this rule to all network connections you have created on that computer, click All network connections.
    • To apply this rule to all LAN connections you have created on that computer, click Local area network (LAN).
    • To apply this rule to any remote or dial-up connections you have created on that computer, click Remote access.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Specify an IPSec tunnel

To specify an IPSec tunnel

  1. In the details pane of IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Select the rule you want to modify, and then click Edit.
  3. On the Tunnel Setting tab, specify the computer that will be the tunnel endpoint:
    4. To | Do this
Disable tunneling for this rule | Click This rule does not specify an IPSEC tunnel.
Use tunneled communications to a specific computer | Click The tunnel endpoint is specified by this IP Address.

Notes

  • To define IPSec policy for a computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • The tunnel endpoint is a static IP address at the other computer to which you want to tunnel the packets, for example, a gateway to a corporate network.
  • You need two rules, one inbound and one outbound, for IPSec tunneling to function. Tunnel filters should not be mirrored. Two filter lists, each containing a one-direction filter, are required to specify a tunnel. Each filter list specifies a direction of traffic that corresponds to each tunnel end point. Example: From subnet A to subnet B, tunnel to the external gateway address for B. From subnet B to subnet A, tunnel to the external gateway address for A.
  • IPSec tunnels should only be used for interoperability with other routers, gateways or end-systems which do not support L2TP/IPSec or PPTP Virtual Private Network (VPN) tunneling technology. IPSec tunnel mode is supported as an advanced feature, only in gateway-to-gateway tunneling scenarios and for server-to-server or server-to-gateway configurations. IPSec tunnels are not supported for client remote access VPN scenarios. L2TP/IPSec or PPTP should be used for client remote access VPN.

Related Topics


Define IP filter lists

  • Add or edit IP filter lists
  • Add or edit IPSec filters
  • Configure advanced IPSec filter settings
  • Activate an IP filter list
  • Manage multiple IP filter lists


Add or edit IP filter lists

To add or edit IP filter lists

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. In the IP Security Policy Properties dialog box, click the rule to which you want to add the new IP filter list, and then click Edit.
  3. If you are adding a new IP filter list, on the IP Filter List tab, click Add. If you are reconfiguring an existing filter list, click the filter list, and then click Edit.
  4. Type a unique name for the list.
  5. Type a description, for example, what traffic types this list represents.
  6. Create the filters.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • For procedures on creating filters and advanced filter settings, see Related Topics.

Related Topics


Add or edit IPSec filters

To add or edit IPSec filters

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the rule that contains the IP filter list you want to modify, and then click Edit.
  3. Do one of the following:
    • If you are adding an IPSec filter, on the IP Filter List tab, click Add.
    • If you are reconfiguring an existing IP filter list, click the IP filter, and then click Edit.
  4. In IP Filter List, do one of the following:
    To | Do this
    Use the IP Filter wizard to create a filter | Confirm that the Use Add Wizard check box is selected.
    Create a filter manually | Clear the Use Add Wizard check box, and then click Add.
    Reconfigure an existing filter | Clear the Use Add Wizard check box, and then click Edit.
  5. On the Addressing tab, select the Source Address:
    Select | To secure packets from
    My IP Address | All IP addresses on the computer for which you are configuring this filter.
    Any IP Address | Any computer.
    A Specific IP Address | The IP address you specify in IP Address.
    A Specific IP Subnet | The IP address you specify in Subnet Mask.
  6. Click the Destination Address and repeat step 5 for the destination address.
  7. Under Mirrored, select the appropriate setting:
    To | Do this
    Create a filter to secure traffic between the source and destination computers. (Filtering to secure traffic in one direction is not supported.) | Select the Mirrored check box
    Create a filter that will permit or drop traffic | You can select the Mirrored check box, but it is not required.
    Create a filter for an IPSec tunnel | Clear the Mirrored check box and create two rules, each with a different filter list.
  8. In Description, type a description for this filter. (For example, specify to what hosts and traffic types it applies.)
  9. (Optional) If you require additional IP filtering by a specific protocol or port number, on the Protocol tab, configure advanced filter settings.

Caution

  • Filters are the most important part of IPSec policy for a host which is protected by IPSec. If you do not specify the proper filters in either client or server policies, or if the IP addresses change before the policy's filters are updated, security may not be provided.
    • Never use DHCP-assigned IP addresses in an IPSec filter. Use My IP Address as a source or destination in the policy that is used by a host that uses a DHCP address. Changes in IP addresses will then automatically update a My IP Address filter.
    • In the policy of a host that will be requesting IPSec protection for its traffic, make sure the filter destination address is a static IP address for the destination host. All IP addresses for the destination must be included in the filter list.
    • When adding a new static IP address to a protected host:
      • Modify the IPSec policy filters on all clients and hosts that make security requests to the protected host. Ensure that those clients updated their policy before adding the new address.
      • Inspect the policy being used on the protected host. If the filters specify static IP addresses for local connections, after adding the new IP address to the interface, edit and save the new filter list to include the new static IP address or restart the Policy Agent service. My IP Address filters will be updated when the new static IP address is added.
    • If the protected host is a web server and your clients use a proxy server, make sure that communications over all network paths are secured by IPSec:
      • between the host and all clients that directly connect
      • between the host and the proxy server
      • between the proxy server and all clients of the proxy server
    • The following filters are internally defined to permit (not secure) this traffic:
      • Internet Key Exchange (IKE): Source Address = Any, Destination Address = Any, Protocol = UDP, Source port = 500, Destination port = 500
      • Kerberos V5: Source Addr =Any, Destination Addr=Any, Protocol=UDP or TCP, Src port = 88, Dst port=88
      • Resource Reservation Protocol (RSVP): Source=Any, Destination=Any, Protocol=46
      • IP Multicast traffic
      • IP Broadcast traffic

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To configure Protocol settings for a filter, see Related Topics.
  • To open IP Security Policy Management, see Related Topics.
  • For IPSec tunnel rules, only address-based filters are supported. Protocol-specific and port-specific filters are not supported. Tunnel filters should not be mirrored. Two filter lists, each containing one direction filer, are required to specify a tunnel. Each filter list specifies a direction of traffic that corresponds to each tunnel end point. Example: From subnet A to subnet B, tunnel to the external gateway address for B. From subnet B to subnet A, tunnel to the external gateway address for A.
  • Filters are applied in the order of most-specific filters first. Filters are not applied in the order in which they appear in the list.
  • If an outbound packet does not match any filter, then it is sent unsecured.
  • If an IKE security request is received , the source IP address of the request is used to find a matching filter. The security action and tunnel setting that is associated with that filter determines the IKE response.
  • All filters used in tunnel rules are matched first before end-to-end transport filters are matched.

Related Topics


Configure advanced IPSec filter settings

To configure advanced IPSec filter settings

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the rule that contains the IP filter list you want to modify, and then click Edit.
  3. On the IP Filter list tab, click the filter you want to modify and then click Edit.
  4. Click the Protocol tab.
  5. Click a setting for Select a protocol type:
    6. To filter | Do this
Any packets sent or received on any protocol in the TCP/IP protocol suite | Click Any.
Traffic sent on a specific protocol | Click the protocol.
Packets sent on a custom protocol | Click Other, and then type the protocol number.
  6. (Optional) If you chose TCP or UDP, you can also filter packets by the protocol source port.
    • To attempt to secure any packets sent on any port used by the protocol type you selected, click From any port.
    • To attempt to secure any packets sent on a specific port used by the protocol type you selected, click From this port, and then type a port number.
  7. Configure the destination port:
    • To filter packets received on any port used by the protocol type you selected, click To any port.
    • To only filter packets received on the port number you specify, click To this port.

Important

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • If you are unsure of these values, contact your system or network administrator to obtain this information. An erroneous value may cause communication to be blocked or unsecured.
  • Configuring advanced filter settings for tunnel mode rules is not recommended or supported.

Note

  • To open IP Security Policy Management, see Related Topics.

Related Topics


Activate an IP filter list

To activate an IP filter list

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the rule you want to modify, and then click Edit.
  3. On the IP Filter List tab, click the IP filter list that you want to designate as the active list for this rule.

    4. The IP filter list you selected is now the active list for the rule.

Note

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Manage multiple IP filter lists

To manage multiple IP filter lists

  1. In IP Security Policy Management, in the console tree, click IP Security Policies.
  2. On the Action menu, click Manage IP Filter Lists and Filter Actions.
  3. Click Add, or if you are reconfiguring an existing list, select that list and click Edit.
  4. Define filters.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Define filter actions

  • Add or edit filter actions
  • Add or edit IPSec security methods
  • Configure advanced IPSec security methods
  • Activate a filter action
  • Manage multiple filter actions


Add or edit filter actions

To add or edit filter actions

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the rule you want to modify, click Edit, and then click the Filter Action tab.
  3. Choose whether to use the Add wizard or not:
    • To use the Add Filter Action wizard to guide you through the process, confirm that the Use Add Wizard check box is selected, and then follow the instructions that are given.
    • To add or edit the filter actions manually, confirm that the Use Add Wizard check box is cleared, and then click Add to create a filter action, or click Edit to reconfigure a filter action.
  4. Choose a filter action:
    • Click Permit to allow receiving or sending of packets in cleartext. Security will not be requested for these packets.
    • Click Block to force packets that match the filter to be discarded immediately. Security will not be requested for these packets.
    • Click Negotiate security to use the list of security methods in Security Method preference order to provide security for packets that match the filter. Security requests will be accepted for these packets.
  5. If you do not want to block incoming, unsecured communications, but you want to ensure that all outgoing communications and subsequent two-way communications are secured, select the Accept unsecured communication, but always respond using IPSec check box.
  6. If you want to enable communication with other computers that do not support IPSec and ensure that communication continues if there is no response to a request for IPSec negotiation, select the check box Allow unsecured communication with non-IPSec aware computers. After the initial IPSec negotiation has failed to a particular peer, this option will disable IPSec to that peer for a period of time.
  7. Select the Session key Perfect Forward Secrecy check box to guarantee that no master keys or master keying material will be re-used to generate the session key.
  8. On the General tab, type a unique name.
  9. Type a description. For example, you might type what security levels this filter action represents.
  10. If you chose Negotiate security, add new security methods or edit the existing ones for the filter action.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • Allow unsecured communication with non-IPSec-aware computers and Accept unsecured communication, but always respond using IPSec are not available in the Default Response rule.
  • Session Key Perfect Forward Secrecy will create a new master key during every session rekey operation. This is the most secure setting. However, it adds considerable overhead to servers and will increase the time required to complete a rekey operation. Both initiator and responder must be configured with the same setting for successful negotiation. Also, interoperability with other products may be affected when using this feature.

Related Topics


Add or edit IPSec security methods

To add or edit IPSec security methods

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the rule you want to modify, and then click Edit.
  3. On the Filter Action tab, click the filter action you want to modify, and then click Edit.
  4. Choose whether to add a security method or to edit an existing one:
    • To add a new security method, on the Security Methods tab, click Add.
    • If you are reconfiguring an existing method, click the security method and then click Edit.
  5. Select the security level:
    6. To | Do this
Use the ESP protocol to encrypt the data (provides confidentiality) using 56-bit DES as the algorithm, 128-bit MD5 for the integrity algorithm, and default key lifetimes (100mb, 1 hour). | Click High (ESP).
Use the AH protocol to sign (provide integrity for) both the packet's addressing information (IP header), and the data, using 128-bit MD5 as the integrity algorithm, and default key lifetimes (100mb, 1 hour). | Click Medium (AH).
Use custom security levels | Click Custom, click Settings, and then change the desired settings.
  6. Repeat this procedure to configure additional security methods for the filter action.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • In Filter Action, click Remove to delete the selected security method.
  • In Filter Action, click Move Up to move the selected security method up one level. Repeat until the security method is at the desired preference level.
  • In Filter Action, click Move Down to move the selected security method down one level. Repeat until the security method is at the desired preference level.
  • For information about how key lifetimes affect security, see the Windows 2000 Resource Kit.
  • When negotiating security, the initiator proposes this list of security methods to the responder. The responder agrees to the first method (from the top down) that is found anywhere in its own security methods list. If none of the proposed security methods match an entry in the responder’s list, then the responder fails the negotiation.
  • IPSec policies allow the choice of a strong encryption algorithm, 3DES, that uses a longer key length than DES for higher security. This policy will be received by all computers to which the policy is assigned. However, Windows 2000-based computers must have the High Encryption Pack installed in order to perform the 3DES algorithm. If a Windows 2000-based computer receives a 3DES setting, but does not have the High Encryption Pack installed, then the 3DES setting in the security method is set to the weaker DES. If this system receives a true 3DES security method proposal, it will fail to find any matching security method and will continue to evaluate the next proposal in the initiator’s security method list. If only 3DES is contained in the initiator’s list, then this system, as responder, will fail the negotiation.

Related Topics


Configure advanced IPSec security methods

To configure advanced IPSec security methods

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the rule you want to modify and click Edit.
  3. On the Filter Action tab, click the filter action you want to modify, and then click Edit.
  4. Choose whether to add a new security method or edit an existing one.
  5. To add a new security method, on the Security Methods tab, click Add.
  6. To reconfigure an existing method, click the security method, and then click Edit.
  7. Click Custom, and then click Settings.
  8. To provide integrity for both the packet's addressing information (IP header) and the data, click Data and address integrity without encryption (AH).
  9. If you chose AH, select an algorithm for integrity:
    • Click MD5 to use a 128-bit key value.
    • Click SHA1 to use a 160-bit value.
  10. To provide both integrity and encryption (confidentiality) for the data, click Data integrity and encryption (ESP)
  11. If you chose ESP, click the algorithm you want to use for data integrity. If you have enabled AH, you can click None for the ESP integrity algorithm to improve performance. If AH is not selected, ESP must have an integrity algorithm selected. See step 9 for possible algorithms.
  12. If you chose ESP, select an algorithm for confidentiality:
    13. To | Do this
Use the algorithm that provides the highest security | Click 3DES.
Encrypt data for privacy, but not use the higher security and overhead of 3DES | Click DES.
Not use encryption for confidentiality | Click None.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • For more information on cryptographic settings see Special IPSec considerations.
  • Use caution when specifying lifetimes. The default internal lifetimes should work in most environments and be interoperable with other products. If your security requirements demand shorter lifetime values, you can explicitly set appropriate lifetime values in each security method. In general, the lowest values that are explicitly set on initiator and responder will be used.
  • Using shorter lifetime values does not increase the security with which data is protected. Short key lifetimes will only decrease the amount of data that is revealed if an attacker discovers one encryption key. Instead of adjusting the lifetime values lower, consider using a stronger encryption algorithm to protect data, such as 3DES.
  • Before changing the default lifetimes, to understand how lifetimes and rekeying work in detail, see the Windows 2000 Resource Kit.

Related Topics


Activate a filter action

To activate a filter action

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Select the rule you want to modify, and then click Edit.
  3. On the Filter Action tab, click the filter action that you want to activate for this rule.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • You can always activate one of the two default filter actions, Request Security (Optional) and Require Security, provided you have not removed them.

Related Topics


Manage multiple filter actions

To manage multiple filter actions

  1. In IP Security Policy Management, click the IP Security Policies folder.
  2. Click Action, and then click Manage IP Filter Lists and Filter Actions.
  3. Click Add, or if you are reconfiguring an existing filter action, click that Action, and then click Edit.
  4. Configure the filter action.
  5. Define security methods for the filter action, if appropriate.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Define advanced IPSec settings

  • Configure key exchange
  • Create key exchange methods


Configure key exchange

To configure key exchange

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the General tab, and then click Advanced.
  3. To force a master key to be rekeyed for every session key, click Master key Perfect Forward Secrecy.
  4. If you require a different setting, enter a value in Authenticate and generate a new key after every (minutes), which will cause a re-authentication and new key generation at that interval.
  5. If you require a different setting, enter a value in Authenticate and generate a new key after every (sessions) to set a maximum limit on the number of times a master key or its base keying material may be reused to generate the session key. This limit, when reached, will force a re-authentication and new key generation.
  6. If you have special requirements for security methods on the key exchange, click Methods.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • IPSec, Internet Key Exchange (IKE), and related services for Windows 2000 are jointly developed by Microsoft Corporation and Cisco Systems, Inc.
  • Use Master key Perfect Forward Secrecy where it is required for interoperability. The default setting, Off, should work in most environments and be interoperable with most other products.
  • A session limit of zero (0) will cause rekeys to only be determined by the time setting.

Related Topics


Create key exchange methods

To create key exchange methods

  1. In IP Security Policy Management, right-click the policy you want to modify, and then click Properties.
  2. Click the General tab, click Advanced, and then click Methods.
  3. Click Add, or if you are reconfiguring an existing method, click the security method, and then click Edit.
  4. Select an Integrity Algorithm:
    • Click MD5 to use a 128-bit value.
    • Click SHA to use a 160-bit value (stronger).
  5. Select a Confidentiality Algorithm:
    • Click 3DES to use the highest security algorithm.
    • Click DES if you are required to connect to computers that do not have 3DES or if you do not need the higher security and overhead of 3DES. For more information on cryptographic settings, see Special Considerations.
  6. Select a Diffie-Hellman Group to set the length of base keying material used to generate the actual keys:
    • Click Low (1) to use 768 bits as a basis.
    • Click Medium (2) to use 1024 bits as a basis (stronger).

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • IPSec, Internet Key Exchange (IKE), and related services for Windows 2000 are jointly developed by Microsoft Corporation and Cisco Systems, Inc.
  • A method is a combination of three settings. The initiator and the responder must have a method in common, one that uses the same settings, for negotiations to succeed.
  • In Key Exchange Security Methods, click Remove to delete the selected security method.
  • In Key Exchange Security Methods, click Move Up to move the selected security method up one level. Repeat until the security method is at the desired preference level.
  • In Key Exchange Security Methods, click Move Down to move the selected security method down one level. Repeat until the security method is at the desired preference level.
  • For more information on cryptographic settings see Special IPSec considerations.

Related Topics


Assign IPSec policy

  • Assign IPSec policy to Group Policy
  • Activate IPSec policy on a computer


Assign IPSec policy to Group Policy

To assign IPSec policy to Group Policy

  1. In the console from which you manage Group Policy, click the Group Policy object to which you want to assign an IPSec policy.
  2. Expand Computer Configuration, then Windows Settings, and then Security Settings.
  3. Click the IP Security Policies folder.
  4. Right-click the policy you want to assign, and then click Assign.

Notes

  • To define IPSec policy for a computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • IPSec policy will remain active even after the Group Policy object to which it is assigned has been deleted. You must unassign the IPSec policy before you delete the policy object. If you delete the policy objects and keep the policy assigned, the IPSec Policy Agent will assume it simply cannot find the policy and use a cached copy.
  • The settings will take effect the next time Group Policy is refreshed. For information on how to force an immediate refresh, see Group policy overview.
  • To remove the IPSec policy assignment from the Group Policy, right-click the policy and click Unassign. If you need to disable IPSec only for a specific computer, you can disable the IPSec Policy Agent Service on that computer.

Related Topics


Activate IPSec policy on a computer

To activate IPSec policy on a computer

  1. In IP Security Policy Management, click the IP Security Policies folder.
  2. Right-click the policy you want to assign, and then click Assign.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • To remove the IPSec policy assignment from the computer, right-click the policy and click Unassign. If you need to disable IP security, you can disable the IPSec Policy Agent Service on that computer.

Related Topics


Manage IPSec policy

  • Test IPSec policy integrity
  • Restore predefined IPSec policies
  • Export an IPSec policy
  • Import an IPSec policy
  • Refresh the list of IPSec policies
  • Delete an IPSec policy
  • Rename an IPSec policy
  • Export the list of IPSec policies


Test IPSec policy integrity

To test IPSec policy integrity

  1. In IP Security Policy Management, click the IP Security Policies on folder.
  2. Click Action, point to All Tasks, and then click Check Policy Integrity.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Restore predefined IPSec policies

To restore predefined IPSec policies

  1. In IP Security Policy Management, click the IP Security Policies folder.
  2. Click Action, point to All Tasks, and then click Restore Default Policies.
  3. Click Yes.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Export an IPSec policy

To export an IPSec policy

  1. In IP Security Policy Management, click the IP Security Policies folder.
  2. Click Action, point to All Tasks, and then click Export policies.
  3. Type the path and the name of the file to which you want the policy information saved.
  4. Click Save.

Notes

  • To define IPSec policy for a computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Import an IPSec policy

To import an IPSec policy

  1. In IP Security Policy Management, click IP Security Policies.
  2. Click Action, point to All Tasks, and then click Import policies.
  3. Type the path and the name of the file from which you are importing the IPSec policy information.
  4. If you want to override all existing policies with the imported information, select the Delete all existing policy information check box.
  5. Click Open.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Refresh the list of IPSec policies

  1. In IP Security Policy Management, right-click the IP Security Policies on folder.
  2. Click Refresh.

Notes

  • To open IP Security Policy Management, see Related Topics.
  • This updates the list to reflect any additions or deletions.
  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.

Related Topics


Delete an IPSec policy

To delete an IPSec policy

  1. In IP Security Policy Management, right-click the policy you want to remove.
  2. Click Delete.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.

Related Topics


Rename an IPSec policy

To rename an IPSec policy

  1. To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  2. In IP Security Policy Management, right-click the policy you want to rename, and then click Rename.
  3. To save the change, type the new name and press Enter.

Note

  • To open IP Security Policy Management, see Related Topics.

Related Topics


Export the list of IPSec policies

To export the list of IPSec policies

  1. In IP Security Policy Management, right-click IP Security Policies.
  2. Click Export List.
  3. Enter the file name to which you want to save the displayed list of IPSec policies.
  4. Click Save.

Notes

  • To define IPSec policy for computers, you must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group.
  • To open IP Security Policy Management, see Related Topics.
  • This does not export the IPSec policies, only a text list of the policies displayed in the console.

Related Topics


Concepts

This section covers:

  • Internet Protocol Security overview
  • Understanding Internet Protocol security
  • Using Internet Protocol security
  • Resources


Internet Protocol security overview

This section covers:

  • Internet Protocol security: What it is
  • Basis for using Internet Protocol security


Internet Protocol Security: What it is

Internet Protocol security (IPSec) is:
  • The long term direction for secure networking.
  • Agressive protection against private network and Internet attacks, while retaining ease of use.
  • A suite of cryptography-based protection services, and security protocols.
  • End-to-end security. The only computers which must know about IPSec protection are the sender and receiver in the communication.
  • The ability to protect communication between workgroups, local area network computers, domain clients and servers, branch offices which may be physically remote, extranets, roving clients, and remote administration of computers.

The Windows 2000 implementation of IPSec is based on industry standards in development by the Internet Engineering Task Force (IETF) IPSec working group.


Basis for Using Internet Protocol Security

  • Policy-based security
  • In-depth defense
  • Network security
  • Cryptography-based protection


Policy-based security

Stronger security methods that are based on cryptography have the potential to greatly increase administrative overhead. Windows 2000 avoids this pitfall by implementing policy-based administration for Internet Protocol security (IPSec).

Instead of applications or operations systems, you use policies to configure IPSec. Network security administrators can configure IPSec policies ranging from those appropriate for a single computer to an Active Directory domain, site, or organizational unit. Windows 2000 provides a central management console, IP Security Policy Management, for defining and managing IPSec policies. The policies can be configured to provide variable levels of protection for most traffic types, in most existing networks.


In-depth defense

Because network attacks can result in system downtime, lost productivity, and public exposure of sensitive information, safeguarding that information from interpretation or modification by unauthorized parties is a high priority.

Network protection strategies generally focus on perimeter security, preventing attacks from outside the private network by using firewalls, security gateways, and user authentication of dial-up access. However, this does not protect against attacks from within the network.

Focusing only on access control security (such as using smart cards and Kerberos) may not result in comprehensive protection, because these methods rely on user names and passwords. Many computers are shared by multiple users, resulting in an unsecured computer, since it is often left in a logged-on state. In addition, if a user name and password have been hijacked by an attacker, security based on access control alone will not stop the illegal access of network resources.

Physical-level protection strategies, which are not commonly used, protect the physical network wires and access points from being used. However, it is unlikely this can guarantee protection of the entire network path, since the data will have to travel from the source to a destination.

A good security plan combines several security strategies, to achieve in-depth protection. Any of these strategies may be combined with IPSec. This provides yet another layer of security, by ensuring that the sending computer secures each IP data packet prior to transmission, before it ever reaches the network wires, and the receiving computer unsecures the data only after it has been received and verified.


Network security

The implementation of IPSec at the IP transport level (Network Layer 3) enables a high level of protection with little overhead. Deploying IPSec requires no changes to existing applications or operating systems. It can be deployed for your existing enterprise scenarios such as:

  • Workgroups
  • Local Area Network (LAN): client/server, peer to peer
  • Remote access: roving clients; Internet access; extranets; remote offices.

Other security mechanisms which operate above Network Layer 3, like Secure Sockets Layer (SSL), only provide security to applications which know how to use SSL, such as Web browsers. You must modify all other applications to protect communications with SSL. Security mechanisms which operate below Network Layer 3, such as link layer encryption, only protect the link, but not necessarily all links along the data path. This makes link layer encryption unsuitable for end-to-end data protection on Internet or routed intranet scenarios.

The implementation of IPSec at Network Layer 3 provides protection for all IP and upper-layer protocols in the TCP/IP protocol suite, such as TCP, UDP, ICMP, Raw (protocol 255), and even custom protocols which send traffic at the IP layer. The primary benefit of securing information at this layer is that all applications and services that use IP for transport of data can be protected with IPSec, without any modification to those applications or services. (To secure protocols other than IP, the packets must be encapsulated by IP.)


Cryptography-based protection

IPSec protects the data so that an attacker will find it extremely difficult or impossible to interpret. A combination of an algorithm and a key is used to secure information. A high level of security is achieved by using cryptography-based algorithms and keys. An algorithm is the mathematical process by which the information is secured; the key is the secret code or number required to read, modify, or verify secured data.

IPSec significantly prevents and reduces network attacks with features which include:

Automatic key management

Key generation

To enable secure communication, two computers must be able to establish the same, shared key, without sending that key across a network to each other. IPSec uses the Diffie-Hellman algorithm to enable this key exchange, and provide the keying material for all other encryption keys.

The two computers start the Diffie-Hellman computation, then exchange an intermediate result publicly and securely (using authentication). Neither computer ever sends the actual key. Using the shared information from the exchange, each computer generates the identical secret key. Expert users can change the default key exchange and data encryption key settings.

Key lengths

Every time the length of a key is increased by one bit, the number of possible keys doubles, making it exponentially more difficult to break the key. IPSec security negotiation between two computers generates two types of shared secret keys: master keys and session keys. Master keys are long, either 768 bits or 1024 bits. The master keys are used as a source from which session keys are derived. Session keys are derived from the master key in a standard way, one session key for each encryption and integrity algorithm required. For detailed information on how keys are managed, see Key Exchange.

Dynamic rekeying

IPSec can automatically generate new keys during a communication. This prevents an attacker from obtaining the entire communication with a single compromised key. Expert users can change the default keying intervals.

Security services

Integrity

Integrity protects information from unauthorized modification in transit, ensuring that the information that is received is exactly the same as the information that is sent. Mathematical hash functions are used to uniquely mark, or "sign," each packet. The receiving computer checks the signature before opening the packet. If the signature (and therefore, packet) has changed, the packet is discarded to prevent a possible network attack.

Authentication

Authentication verifies the origin and the integrity of a message by assuring the genuine identity of each computer. Without strong authentication, an unknown computer is suspect, with any information it sends. Multiple methods of authentication may be listed in each policy, to ensure that a common authentication method may be found for Windows 2000 domain members, computers not running Windows 2000, and remote computers.

Confidentiality (data encryption)

Confidentiality ensures that data is only disclosed to intended recipients. When it is selected, the Encapsulating Security Payload (ESP) format of IPSec packets is used. Packet data is encrypted before transmission, ensuring that the data cannot be read during transmission, even if the packet is monitored or intercepted by an attacker. Only the computer with the shared, secret key is able to interpret or modify the data. The United States Data Encryption Standard (DES) algorithms, DES and 3DES, are used to provide confidentiality of both the security negotiation and the application data exchange. Cipher Block Chaining (CBC) is used to hide patterns of identical blocks of data within a packet without increasing the size of the data after encryption. Repeated encryption patterns can compromise security by providing a clue which an attacker can use to attempt to discover the encryption key. An Initialization Vector (an initial random number) is used as the first random block to encrypt and decrypt a block of data. Different random blocks are used in conjunction with the secret key to encrypt each block. This ensures that identical sets of unsecured data are transformed into unique sets of encrypted data. For more information on cryptographic settings see Special IPSec considerations.

Nonrepudiation

Ensures the sender of a message is the only person who could have sent the message; the sender cannot deny having sent the message.

Anti-replay

Also called replay prevention, it ensures the uniqueness of each IP packet. Messages captured by an attacker cannot be re-used or replayed to establish a session or gain information illegally.


Understanding Internet Protocol Security

This section introduces the key concepts and terminology you should know before you deploy IPSec.

For information about specific topics, see:

  • IPSec security protocols
  • IPSec policy agent
  • IPSec security negotiation
  • IPSec packet handling
  • How IPSec works
  • Virtual Private Networking with IPSec


IPSec security protocols

Security protocols provide data and identity protection for each IP packet. Windows 2000 IPSec uses the Authentication Header and Encapsulating Security Payload to provide these services.

  • Authentication Header (AH)

    • AH provides authentication, integrity, and anti-replay for the entire packet (both the IP header and the data carried in the packet); AH signs the entire packet. It does not encrypt the data, so it does not provide confidentiality. The data is readable, but protected from modification. AH uses HMAC algorithms to sign the packet.

    For example, Alice on Computer A sends data to Bob on Computer B. The IP header, the AH header, and the data are protected from modification by the signature. This means Alice can be certain it was really Bob who sent the data and that the data was unmodified.

    Authentication Header

    Integrity and authentication are provided by the placement of the AH header between the IP header and the transport protocol header (TCP or UDP).

  • Encapsulating Security Payload (ESP)

    • ESP provides confidentiality, in addition to authentication, integrity, and anti-replay.

    ESP does not normally sign the entire packet unless it is being tunneled. Ordinarily, only the data is protected, not the IP header.

    For example, Alice on Computer A sends data to Bob on Computer B. The data is encrypted because ESP provides confidentiality. Upon receipt, after the verification process is complete, the data portion of the packet is decrypted. Alice can be certain it was really Bob who sent the data, that the data is unmodified, and that no one else was able to read it.

    Encapsulating Security Payload

    Security is provided by by the placement of the ESP header between the IP header and the transport protocol header (TCP or UDP).

Expert users can select which protocol will be used for a communication by configuring security methods in the IPSec policy. For information on how to configure security methods, see To add or edit security methods.


IPSec Policy Agent

The IPSec Policy Agent is a mechanism that resides on each Windows 2000 computer, that appears in the list of system services. The Policy Agent retrieves the active IPSec policy information, and passes it to the other IPSec mechanisms which require that information to perform security services.

IPSec Policy Agent

The Policy Agent starts automatically at system start time. If there are no active IPSec policies, or if the Policy Agent cannot connect to Active Directory for some reason, the Policy Agent will continue to poll Active Directory for an assigned policy, or check the registry for a locally-assigned policy.


IPSec security negotiation

Before secured data can be exchanged, a contract between the two computers must be established. This contract is called a security association (SA). In an SA, both computers agree on how to exchange and protect information, as shown in the following figure.

ISAKMP/Oakley Service

Security Associations

An security association (SA) is a combination of a policy and keys which defines the common security services, mechanisms, and keys used to protect the communication from end to end. The security parameters index (SPI) is a unique, identifying value in the SA used to distinguish among multiple security associations that exist at the receiving computer. For example, multiple associations may exist if a computer securely communicates with multiple computers simultaneously. This situation occurs mostly when the computer is a file server or a remote access server which serves multiple clients. However, one computer can have multiple SAs with a single computer. In these situations, the receiving computer uses the SPI to determine which SA will be used to process the incoming packets.

To build this contract between the two computers, the IETF has established a standard method of security association and key exchange resolution, which combines the Internet Security Association and Key Management Protocol (ISAKMP) and the Oakley key generation protocol. ISAKMP centralizes security association management, reducing connection time. Oakley generates and manages the authenticated keys used to secure the information.

This process protects not only computer-to-computer communications, but also protects remote computers that are requesting secure access to a corporate network or any situation in which the negotiation for the final destination computer (or endpoint) is actually performed by a security router or other proxy server. In the latter situation, referred to as ISAKMP client mode, the identities of the endpoints are hidden to further protect the communication.

To ensure successful, secure communication, ISAKMP/Oakley performs a two-phase operation. It assures confidentiality and authentication during each phase by the use of negotiated encryption and authentication algorithms that are agreed on by the two computers. With the duties split between two phases, keying can be accomplished with great speed when required.

Key Exchange

During this initial phase, the two computers establish the first SA, called the ISAKMP SA. (This SA is named in order to differentiate between the SAs established in each of the two phases.) Oakley provides identity protection during this exchange, enabling total privacy. This can help prevent common types of network attacks which focus on hijacking identities.

The security negotiation process during this phase includes:

  1. Policy Negotiation

    2. This determines:

    • The encryption algorithm: DES, 3DES, 40bitDES, or none.
    • The integrity algorithm: MD5 or SHA.
    • The authentication method: Public Key Certificate, preshared key, or Kerberos V5 (the Windows 2000 default).
    • The Diffie-Hellman group.

    If certificates or preshared keys are being used for authentication, the computer identity is protected. However, if the default Windows 2000 method is used, the computer identity is unencrypted until the entire identity payload is encrypted during Step 3.

  2. Key Information Exchange

    3. This results in each computer having the necessary information to generate the shared, secret key--the master key--for the ISAKMP SA. Actual keys are never exchanged; only the public information needed by Diffie-Hellman to generate the shared, secret key. The Oakley service on each computer generates the master key used to protect authentication.

  3. Authentication

    4. The computers attempt to authenticate the key information exchange. Without successful authentication, communication cannot proceed. The master key is used, in conjunction with the algorithms negotiated in Step 1. Regardless of which authentication method is used, the identity payload is protected from both modification and interpretation.

The initiating peer sends an offer list of potential security levels to the responding peers. The responder cannot modify the offer. Should the offer be modified, the initiator will reject the responder's message. The responder sends a reply either accepting the offer, or it simply discards the offers and sends a message back indicating no offer was chosen. The process then begins again.

The negotiation messages that are exchanged are automatically resent five times. After three seconds, a soft SA will be established, if the active policies allow unsecured communications with non-IPSec capable computers. If an ISAKMP response is received before the cycle times out, the soft SA will be deleted and standard SA negotiation begins. The potential number of SAs that are formed is only limited by system resources.

The ISAKMP SA is used to initiate the second phase of security negotiations.

Data Protection

A pair of SAs are negotiated on behalf of the IPSec service, and are referred to as IPSec SAs. Keying material is refreshed or new keys are generated if PFS or key lifetimes have been enabled, the ISAKMP SA has expired.

The security negotiation process during this second phase includes:

  1. Policy negotiation

    2. This determines:

    • The IPSec protocol: AH, ESP.
    • The integrity algorithm: MD5, SHA.
    • The encryption algorithm: DES, 3DES, 40bitDES, or none.

    A common agreement is reached, and two SAs are established: one for inbound communications and one for outbound communications.

    Oakley refreshes the keying material and new, shared, secret keys are generated for authenticating and, possibly, encrypting the packets. If a new key is required, a second Diffie-Hellman exchange takes place. Unless a key or SA has expired, Oakley refreshes the keying material from the Diffie-Hellman exchange that was performed during the key exchange.

  2. The SAs and keys are passed to the IPSec driver, along with the SPI.

The entire negotiation is protected by the ISAKMP SA. Except for the ISAKMP header on the packets, all of the message packets are encrypted, and an integrity signature following the ISAKMP header authenticates the message. Oakley prevents replays of negotiation messages, providing anti-replay protection.

The automatic message retry process is almost identical to the process discussed in the key exchange negotiation, with one exception: if this process reaches a time-out for any reason during the second or greater negotiation of the same ISAKMP SA, a renegotiation of the ISAKMP SA is attempted. If a message for the data protection phase is received without establishing a ISAKMP SA, it is rejected.

The ability to use a single ISAKMP SA for multiple IPSec SA negotiations makes the security negotiation process extremely fast. As long as the ISAKMP SA does not expire, renegotiation and re-authentication are not necessary. The active IPSec policy determines the number of times this can occur .

SA Lifetimes

When a key lifetime is reached for the master or session key, the associated SA is renegotiated. An Oakley "delete" message is sent to the responding peer which requests the responder to mark the ISAKMP SA as "expired." This prevents bogus IPSec SAs from being formed from an old SA. Oakley expires ISAKMP SAs, and the IPSec driver expires IPSec SAs.

When the IPSec Policy Agent retrieves policy updates, it will update the IP filter list stored in the IPSec driver. Any SAs associated with old filters which no longer exist in the policy will be deleted, along with the old filters in the IPSec driver cache.

If the power save features in Windows 2000 cause the computer to hibernate or sleep and the SA expires during this period, the SA will be automatically renegotiated when the peer is re-activated. If Windows 2000 is shut down on one of the peers, the Oakley "delete"message cleans up any remaining SAs, and new SAs are renegotiated when the peers attempt communication again. If Windows 2000 terminates abnormally, the Oakley "delete" message will not be sent. In this case, the old SAs might still be in place until the default time-out period is reached. If this occurs, the SAs must be manually deleted.


IPSec packet handling

The IPSec driver receives the active IP filter list from the IPSec Policy Agent, and then matches every inbound and outbound packet against the filters in the list. When a packet matches a filter, it applies the filter action.

IPSec driver

If the filter action permits transmission, then the packet is received or sent with no modifications. If the action blocks transmission, then the packet is discarded.

Outbound and inbound processing uses the negotiated security association (SA) and keys. The IPSec driver stores all current SAs in an internal database. If multiple SAs are present, the driver uses the Security Parameters Index (SPI) to match the correct SA with the correct packet.

When an outbound IP packet matches the IP filter list with an action to negotiate security, the IPSec driver queues the packet, then notifies Internet Key Exchange (IKE), which begins security negotiations with the destination IP address of that packet. If several outbound packets going to the same destination match the same filter before IKE has finished the negotiation, then only the last packet sent is saved.

Once the negotiations have successfully completed, IKE provides the IPSec driver with the parameters for the SA, including the session keys. The IPSec driver secures the queued outbound IP packet and sends it to the network card for transmission. If the negotiation failed, the IPSec driver discards the packet.

When an IPSec-secured inbound packet matches the IP filter list, the IPSec driver verifies the integrity of the packet, decrypts the packet if necessary, and translates it back into a normal IP format. The IPSec driver matches the IP packet again against the filter to make sure no traffic is received other than what was agreed upon during the negotiation. If the packet matches the filter, then the IPSec driver sends it back to TCP/IP for delivery up to the application.

When an unsecured, normal IP packet is received, the IPSec driver matches the packet against all filters in the filter list. If a match occurs and the filter action for that filter requires IP security, or blocks the packet, then the packet is discarded.

The IPSec driver matches all inbound unsecured packets against the list of filters that specify IPSec tunnels first, then matches the packet against all filters that specify end-to-end (transport) filters. The IPSec driver does not filter certain types of IP packets. For more information, see Filter actions.


How IPSec works

For simplicity, this example illustrates IPSec from a domain computer to a domain computer. Alice, using an application on Computer A, sends a message to Bob.

Overview of the IPSec process
  1. The IPSec driver on Computer A checks the IP Filter List in the active policy for a match with the address or traffic type of the outbound packets.
  2. The IPSec driver notifies ISAKMP to begin security negotiations with Computer B.
  3. The ISAKMP service on Computer B receives a request for security negotiations.
  4. The two computers perform a key exchange, establish an ISAKMP SA and a shared, secret key.
  5. The two computers negotiate the level of security for the data transmission, establishing a pair of IPSec SAs and keys for securing the IP packets.
  6. Using the outbound IPSec SA and key, the IPSec driver on Computer A signs the packets for integrity, and encrypts the packets if confidentiality has been negotiated.
  7. The IPSec driver on Computer A transfers the packets to the appropriate connection type for transmission to Computer B.
  8. Computer B receives the secured packets and transfers them to the IPSec driver.
  9. Using the inbound SA and key, the IPSec driver on Computer B checks the integrity signature and decrypts the packets, if necessary.
  10. The IPSec driver on Computer B transfers the decrypted packets to the TCP/IP driver, which transfers them to the receiving application.

Alice and Bob never see any of the process. The standard routers or switches in the data path between the peers do not require IPSec. They will automatically forward the encrypted IP packets to the destination. However, if a router is functioning as a firewall, security gateway, or proxy server, you must enable special filtering to enable the secured IP packets to pass through. For more information, see Special IPSec considerations.


Virtual Private Networking with IPSec

The entire process of encapsulation, routing, and de-encapsulation is called tunneling. Tunneling hides, or encapsulates, the original packet inside a new packet. This new packet may have new addressing and routing information, which enables the new packet to travel through networks. When tunneling is combined with privacy, the original packet data (as well as the original source and destination) is not revealed to those listening to traffic in the network. The network could be any internetwork: a private intranet, or the Internet. Once the encapsulated packets reach their destination, the encapsulation header is removed and the original packet header is used to route the packet to its final destination.

The tunnel itself is the logical data path through which the encapsulated packets travel. To the original source and destination peer, the tunnel is usually transparent and appears as just another point-to-point connection in the network path. The peers are unaware of any routers, switches, proxy servers, or other security gateways between the tunnel’s beginning point and the tunnel’s end point. When tunneling is combined with privacy, it can be used to provide Virtual Private Networks (VPN).

In Windows 2000, two types of tunneling are provided that use IPSec:

  1. Layer 2 Tunneling Protocol (L2TP/IPSec), in which L2TP provides encapsulation and tunnel management for any type of network traffic and IPSec in transport mode provides the security for the L2TP tunnel packets.
  2. IPSec in tunnel mode, in which IPSec itself does the encapsulation for IP traffic only.

Before using either type of tunneling, a complete understanding of the functionality should be obtained. For more information, see "Virtual Private Networking and IPSec" in the Windows 2000 Resource Kit.

The encapsulated packets travel through the network inside the tunnel. (In this example, the network is the Internet.) The gateway may be an edge gateway which stands between the outside Internet world and the private network--a router, firewall, proxy server, or other security gateway. Also, two gateways may be used inside the private network to protect traffic across less trusted parts of the network.

L2TP and IPSec

IPSec and L2TP are combined to provide both tunneling and security for IP, IPX and other protocol packets across any IP network. IPSec can also perform tunneling without L2TP, but it is only recommended for interoperability, when one of the gateways does not support L2TP or PPTP.

L2TP encapsulates original packets inside a PPP frame, performing compression when possible and then inside a UDP-type packet assigned to port 1701. Since the UDP packet format is an IP packet, L2TP automatically uses IPSec to secure the tunnel, based on the security settings in the user configuration of the L2TP tunnel. The IPSec Internet Key Exchange (IKE) protocol negotiates security for the L2TP tunnel using certificate-based authentication by default. This authentication uses computer certificates, not user certificates, to verify that both source and destination computers trust each other. If IPSec transport security is successfully established, then L2TP negotiates the tunnel, including compression and user authentication options and performs access control based on the user identity. Thus, L2TP/IPSec is the easiest, most flexible, most interoperable and more secure tunneling option for both client remote access VPN and gateway-to-gateway VPN tunnels.

Configuration for L2TP/IPSec VPN remote access clients is performed using Network and Dial-up Connections. Configuration for the VPN remote access server and for gateway-to-gateway tunnels is performed using the Routing and Remote Access console.

Tunnel packet using L2TP and IPSec

The original packet header, shown here as the IP or IPX header, carries the original and ultimate source and destination addresses (addresses used on the private network), and the outer IP header, shown as New IP Header contains the source and destination addresses of the tunnel end points (addresses used in the public network). The L2TP header carries tunnel control information. The PPP header identifies the protocol of the original packet, for example IP or IPX. For more information on L2TP/IPSec, see Network and Dial-up Connections in Windows 2000 Server Help.

IPSec Tunneling

The primary reason for using IPSec tunnel mode is for interoperability with other routers, gateways, or end-systems which do not support L2TP/IPSec or PPTP VPN tunneling technology. IPSec tunnel mode is supported only in gateway-to-gateway tunneling scenarios and for certain server-to-server or server-to-gateway configurations as an advanced feature. The Windows 2000 Resource Kit chapter on IPSec describes these scenarios and configurations in more detail and should be understood before using IPSec tunnel mode. IPSec tunnel mode is not supported for client remote access VPN scenarios. L2TP/IPSec or PPTP should be used for client remote access VPN.

The two formats of IPSec packets can be used also in tunnel mode:

  • ESP Tunnel Mode

    • The original IP header (which is the original packet header) usually carries the ultimate source and destination addresses, while the outer IP header usually contains the source and destination address of security gateways. The ESP tunnel format always provides strong integrity and authenticity for traffic carried inside the tunnel. The ESP tunnel is used mainly to provide privacy for the tunneled packets using DES or 3DES encryption. The level of encryption is specified in the Filter Action of the tunnel rule, and thus could also be configured for no encryption if the contents of the tunnel traffic does not require privacy.

    ESP tunnel mode

    In the preceding illustration, the original packet between the ultimate source and destination is encapsulated by the new IP and ESP headers. The Signed area indicates where the packet has been protected with integrity. The Encrypted area indicates that the entire original packet may be encrypted.

    The information in the new IP header is used to route the packet from origin to the tunnel destination end point; usually a security gateway. The new IP ESP header is not protected by the integrity hash. This is the IETF RFC design to allow the packet header to be modified by network components as necessary to provide additional services, such as changing the source or destination IP address, or giving it higher priority over other packets.

  • AH Tunnel Mode

    • AH tunnel mode does not provide encryption privacy for the contents of the tunnel, only strong integrity and authenticity.

    AH tunnel mode

    The entire packet is signed for integrity, including the new tunnel header. Thus, no change in the source or destination address can be made once the packet is sent by the source of the tunnel. The IETF RFC design still allows for a few fields in the new IP header to be modified by network components to provide priority for certain packets, and to delete stray or old packets. ESP and AH can be combined to provide tunneling, which includes both integrity for the entire packet and confidentiality for the original IP packet.

IPSec tunnels provide security for "IP only" traffic. The tunnel is configured to protect traffic either between two IP addresses or between two IP subnets. If the tunnel is used between two hosts instead of between two gateways, the outer IP address is the same as the inner IP address. In Windows 2000, IPSec does not support protocol-specific, port-specific, or application-specific tunnels. Configuration is performed using the IPSec Policy console by specifying a security rule containing a filter to describe the traffic that goes into the tunnel, a filter action for securing the tunnel and an authentication method to be used by the tunnel end points. Three types of authentication are supported: certificates, pre-shared key, and Kerberos.

For conceptual information on IPSec policy tunnel settings, see IPSec tunneling. For information on configuring an IPSec tunnel, see To specify an IPSec tunnel.


Using Internet Protocol Security

You must have appropriate administrator rights to Group Policy or be a member of the local system Administrators group in order to define IPSec policy for a computer.

  • IPSec policy planning
  • Description of an IPSec policy
  • IPSec scenarios


IPSec policy planning

This section suggests ways to plan for and deploy IPSec for your enterprise. You should also review Best Practices for general information on IPSec deployment.

For information on specific topics, see:

  • Establishing an IPSec security plan
  • Predefined IPSec policies
  • Assign IPSec policy
  • Special IPSec considerations
  • Managing multiple IP Filter Lists and Filter Actions


Establishing an IPSec security plan

Whether for a large domain or a small workgroup, implementing IPSec means finding a balance between making information easily available to the largest number of users, and protecting sensitive information from unauthorized access.

Finding the proper balance requires:

  • Assessing the risk and determining the appropriate level of security for your organization.
  • Identifying valuable information.
  • Defining security policies that use your risk management criteria and protect the identified information.
  • Determining how the policies can best be implemented within the existing organization.
  • Ensuring that management and technology requirements are in place.
  • Providing all users with both secure and efficient access to the appropriate resources, according to their needs.

Security considerations are also influenced by the way the computer will be used. For example, the required security may differ, depending on whether the computer is a domain controller, Web server, remote access server, file server, database server, intranet or remote client. The Windows 2000 security framework is designed to fulfill the most stringent security requirements. However, software alone may be less effective without careful planning and assessment, effective security guidelines, enforcement, auditing, and sensible security policy design and assignment.

There is no exact definition of the measures that define standard security. These can vary widely, depending on an organization's policies and infrastructures. The following security levels can be considered as a general basis for planning your IPSec deployment:

  • Minimal security

    • Computers do not exchange sensitive data. IPSec is not active by default. No administrative action to disable IPSec would be required.

  • Standard security

    • Computers, especially file servers, are used to store valuable data. Security must be balanced so it does not become a barrier to legitimate users trying to perform their tasks. Windows 2000 provides predefined IPSec policies which secure data, but do not necessarily require the highest level of security: Client (Respond Only) and Server (Request Security). These, or similar custom policies, will optimize efficiency without compromising security.

  • High security

    • Computers that contain highly sensitive data are at risk for data theft, accidental or malicious disruption of the system (especially in remote dial-up scenarios), or any public network communications. Secure Server (Require Security), a predefined policy, requires IPSec protection for all traffic being sent or received. Secure Server (Require Security) includes strong confidentiality and integrity algorithms, Perfect Forward Secrecy, key lifetimes and limits, strong Diffie-Hellman Groups. Unsecured communication due to non-IPSec aware computers, or failed security negotiation is blocked.


Predefined IPSec policies

Windows 2000 provides a set of predefined IPSec policies. By default, all predefined policies are designed for computers which are members of a Windows 2000 domain. They may be assigned without further action, modified, or used as a template for defining custom policies.

Predefined policies

Client (Respond Only)

This is used for computers which should not secure communications most of the time. For example, intranet clients may not require IPSec, except when requested by another computer. This policy enables the computer on which it is active to respond appropriately to requests for secured communications. The policy contains a default response rule, which enables negotiation with computers requesting IPSec. Only the requested protocol and port traffic for the communication is secured.

Server (Request Security)

This is used for computers which should secure communications most of the time. An example would be servers which transmit sensitive data. In this policy, the computer accepts unsecured traffic, but always attempts to secure additional communications by requesting security from the original sender. This policy allows the entire communication to be unsecured if the other computer is not IPSec-enabled.

Secure Server (Require Security)

This is used for computers which always require secure communications. An example would be a server which transmits highly sensitive data, or a security gateway which protects the intranet from the outside. This policy rejects unsecured incoming communications, and outgoing traffic is always secured. Unsecured communication will not be allowed, even if a peer is not IPSec-enabled.

Predefined rules

Like the predefined policies, the default response rule is provided for activation without further action or it may be modified to fit specific needs. It is added to each new policy you create, but not automatically activated. It is for any computers which do not require security, but must be able to appropriately respond when another computer requests secured communications.

Predefined filter actions

Like the predefined rules, these are provided for activation without further action, modification, or as a template for defining custom filter actions. They are available for activation in any new or existing rule:

  • Require Security. High security. Unsecured communication will not be allowed.
  • Request Security (Optional). Medium to low security. Unsecured communication is allowed, to enable communication with computers that do not or can not negotiate IPSec.


Assigning IPSec policy

Internet Protocol Security Policy Management

Internet Protocol Security Policy Management is used to create and configure IPSec policies through the Microsoft Management Console (MMC). It can manage policy centrally (for Active Directory clients), manage policy locally (the computer on which you are running the snap-in), or manage policy remotely for a computer or domain.

You must add the snap-in to the MMC. A wizard guides you through the correct snap-in configuration. The customized console can then be saved so that it is available to you again at any time.

Policies stored in Active Directory

A Group Policy object defines access, configuration and usage settings for accounts and resources. IPSec policies can be assigned to the Group Policy object of a computer account, site, domains, or organizational unit. When the IPSec policy is applied to the Group Policy object for the Active Directory object, the IPSec policy is propagated to any computer accounts affected by that Group Policy object. For more information, see Active Directory overview.

There are some things to keep in mind when assigning an IPSec policy.

  • IPSec policies that are applied to domain policy will override the local, active IPSec policy when that computer is a member of the domain.
  • IPSec policies that are assigned to organizational units in Active Directory will override domain level policy for any members of that organization unit, and the lowest-level organizational unit IPSec policy will override IPSec policy for higher-level organizational units for any members of that organizational unit, not merge with them.
  • Assigning policies at the highest possible level provides the greatest breadth of effect with the least amount of administrative effort.
  • IPSec policy will remain active even after the Group Policy object to which it is assigned has been deleted. You must unassign the IPSec policy before you delete the policy object. If you delete the policy objects and keep the policy assigned, the IPSec Policy Agent will assume it simply cannot find the policy and use a cached copy.
  • Backup and restore of Group Policy in Active Directory must also include IPSec policies to ensure consistency.

The IPSec Policy Agent only checks Active Directory for updates to the active or assigned IPSec policy. If new IPSec policies have been created in Active Directory, or an IPSec policy has been changed and assigned to a client computer, the Winlogon service will discover these changes during its next polling cycle for Group Policy changes, it will notify the IPSec Policy Agent, and then the changes will be applied to the client computer.

For information on how to assign IPSec policy to objects in Active Directory, see To assign IPSec policy to Group Policy overview.

Local Computer Policy

Each computer running Windows 2000 has exactly one local Group Policy object, often called the Local Computer Policy. Using this local Group Policy object, Group Policy settings can be stored on individual computers whether or not they are part of an Active Directory environment or a networked environment. Because its settings can be overwritten by Group Policy objects associated with sites, domains, or organizational units, the local Group Policy object is the least influential one in an Active Directory environment. In a non-networked environment (or in a networked environment lacking a Windows 2000 domain controller), the local Group Policy object's settings are more important because they are not overwritten by other Group Policy objects.

For information on how to assign IPSec policy locally, see To activate IPSec policy on a computer.

For more information on Group Policy, see Group Policy overview.


Special IPSec considerations

The following considerations will help simplify administration of IPSec policies:

Windows 2000 Encryption Requirements

The availability of IPSec encryption features in Windows 2000 is subject to United States export regulations, and may also be subject to your local and national regulations. The Internet Key Exchange (IKE) uses encryption to protect the security negotiation, and the IPSec component of TCP/IP uses encryption to protect application data packets. IKE must be able to use at least DES encryption, as provided by the Cryptographic API (CAPI), to protect the negotiation. Windows 2000-based computers must be able to perform DES in CAPI in order to secure traffic with any type of IPSec packet format.

IPSec policies allow the choice of a strong encryption algorithm, 3DES, that uses a longer key length than DES for higher security. This policy will be received by all computers to which the policy is assigned. However, Windows 2000-based computers must have the High Encryption Pack installed in order to perform the 3DES algorithm. If a computer receives a 3DES setting, but does not have the High Encryption Pack installed, then the 3DES setting in encryption policy is set to the weaker DES. For more information on US export regulations, see the Microsoft Web site (http://www.microsoft.com/).

Authentication

If the computers in your enterprise are part of a Windows 2000 domain, IPSec authentication can be accomplished with the default Windows 2000 authentication protocol (Kerberos V5). You do not have to deploy public key certificates for intranet communication.

IP Filter Lists

  • Try to use general filters, covering a group of computers with only one filter. For example, try to use Any IP address or an IP subnet address, rather than specifying a specific computer's source and destination IP address.
  • Define filters which allow you to group and secure traffic from segments of your network that are related.

Filter Actions

  • If you need to prevent communication with unauthorized computers, ensure that security is not negotiated for non-essential data or when peers are not IPSec-enabled, and make use of filter actions as blocking (Block) or pass-through policies (Permit).
  • When configuring custom security methods, set only the ESP confidentiality selection to None when a higher-layer protocol will provide data encryption.
  • For remote communication scenarios (including IPSec tunneling), consider a list of security methods which specifies high levels of security, such as 3DES only, short key lifetimes (less than 50Mb), and Perfect Forward Secrecy for the master and session keys. This will help protect against known key attacks.

Remote Access Communications

  • If you will be using L2TP, the list of authentication methods must include certificates, and at least one computer-level public key certificate must be configured on each peer (remote client or remote access server). Windows 2000 domain controllers can be configured to auto-enroll domain members in a certificate authority. For more information on certificates, see Overview of public key policies.
  • To help protect against network attacks on a remote access server, the predefined filter action, Require Security, should be active (or a similar custom filter action). In addition, take caution when enabling Allow unsecured communication with non-IPSec aware computers and Accept unsecured communication but always respond using IPSec. Allowing unsecured communications may cause security problems.
  • When configuring the IP Filer List for a remote access server (which usually has multiple interfaces and addresses), do not set the filters to My IP Address. Otherwise, IPSec will attempt to enforce that filter on all interfaces on the server. Instead, configure the filter to the IP address of the interface which sends and receives the traffic you want to secure.
  • If you administer computers in your enterprise remotely, you must add a rule to your active IPSec policy to prevent RPC TCP traffic from being blocked when it comes from the internal network. (This type of traffic is used by the remote access configuration tools in Windows 2000). For example:
    1. The IP filter list in the rule should specify an outgoing address of the corporate subnet (which is the location of your administrative console), and an incoming address of the managed computer's internal IP address. The protocol type should be set to TCP.
    2. The filter action in the rule should enable Accept unsecured communication and Allow communication with non-IPSec-enabled computers.

SNMP

If a computer will be running an SNMP service, you must add a rule to prevent SNMP messages from being blocked:

  1. The IP filter list should specify the source and destination addresses of the SNMP management systems and agents. The Protocol type should be set to UDP, to and from ports 161 and 162. This will require two filters: one for UDP, to and from port 161, and the other for UDP, to and from port 162.
  2. The filter action should be set to Permit, which blocks negotiation for security and passes through any traffic which matches the IP filter list.

Security Gateways

For a security gateway, firewall, proxy server, router or any server which is an access point from the intranet to the outside world, special filtering must be enabled on that computer to ensure that packets secured with IPSec are not rejected. At a minimum, the following input and output filters must be defined for Internet interface on the computer:

    Input Filters

    1. IP Protocol ID of 51 (0x33) for inbound IPSec Authentication Header traffic.
    2. IP Protocol ID of 50 (0x32) for inbound IPSec Encapsulating Security Protocol traffic. UDP port 500 (0x1F4) for inbound ISAKMP/Oakley negotiation traffic.

    Output Filters

    1. IP Protocol ID of 51 (0x33) for outbound IPSec Authentication Header traffic.
    2. IP Protocol ID of 50 (0x32) for outbound IPSec Encapsulating Security Protocol traffic. UDP port 500 (0x1F4) for outbound ISAKMP/Oakley negotiation traffic.

DHCP, DNS, and WINS services; Domain Controllers

Before enabling IPSec for servers functioning in any of the above capacities, determine if all the clients are also IPSec-capable. Otherwise, secure negotiation might erroneously fail, and access to network resources may be blocked.

DNS

If you want to specify a DNS name in an IP filter list, but your DNS servers are not IPSec-enabled, a special policy setting is required. Otherwise, IPSec and other services on the server may not be able to successfully resolve the DNS computer name to a valid IP address.

  1. Set the IP filter list to exempt traffic between the computer and the DNS server from requiring IPSec:
    • Set the Source address to My IP address.
    • Set the Destination address to the IP address of your DNS server.
    • Enable Mirrored to automatically create the inbound filter.
    • Configure the protocol settings, From this Port and To this Port, to the port which your DNS server has been configured to use for traffic. It is generally port 53.
  2. Set the filter action to Permit, to ensure that DNS traffic passes through and security is never negotiated for traffic which matches this IP filter list.

Caution

  • Using a DNS name in the IPSec policy instead of an IP address allows easier administration, however, it should only be utilized when the computer has a static IP address (one not obtained automatically from DHCP) to which the DNS name will be resolved. For example, if an attacker breaks into the DNS server and changes the IP addresses associated with DNS names, IPSec will resolve the name to a different IP address, resulting in unsecured traffic.


Managing multiple IP filter lists and filter actions

You can define a global pool of IP filter lists and filter actions, which are then available in any new or existing rule. This method can be used as an alternative to configuring the IP filter list or filter action as you define each rule. In this way, all the IP filter lists and filter actions necessary for your security plan may be defined in one place.

For information on how to centrally configure IP filter lists and filter actions, see To manage multiple IP filter lists or To manage multiple filter actions.

The IP filter list and filter action procedures are the same procedures which appear when you are defining these within a rule. For help with defining an IP Filter List, see To add or edit IP filter lists. For help with defining a filter action, see To add or edit filter actions.


Description of an IPSec policy

  • IPSec policy properties
  • Rules
  • IP filter lists
  • Filter actions
  • IPSec security methods
  • IPSec authentication
  • IPSec tunneling
  • IPSec connection types
  • Advanced IPSec settings


IPSec policy properties

IPSec policies can be applied to local computers, domain members, domains, organizational units, or any Group Policy object in Active Directory. Your organization's IPSec policies should be based on your organization's written guidelines for secure operations. Policies may store multiple security actions, called rules, so that one policy may be applied to multiple computers.

There are two storage locations for IPSec policies:

  1. Active Directory
  2. Locally defined in the registry for stand-alone computers and computers which are not always part of a trusted Windows 2000 domain. If the computer is temporarily unconnected to a trusted Windows 2000 domain, the policy information is cached in the local registry. For more information, see Active Directory overview.

Windows includes predefined policies which can be activated, modified to meet your needs, or used as a template for your own custom policies. Each policy you define should apply to a scenario in your security plan. Special configuration settings may apply if you are assigning policies to a DHCP server, Domain Name System (DNS), WINS, Simple Network Management Protocol (SNMP), or remote access server.

Group Policy

IPSec policies which are assigned to a Group Policy object in Active Directory become part of Group Policy, and are transferred to the member computers each time Group Policy propagates. For more information on Group Policy, see Group Policy overview.

When assigning an IPSec policy in Active Directory, consider the following:

  • IPSec policies assigned to a domain policy will override any active, local IPSec policy only when that computer is connected to the domain.
  • IPSec policies assigned to an organizational unit will override an IPSec policy assigned to the domain policy, for any member computers of that organizational unit. The IPSec policy assigned to the lowest-level organizational unit will override an IPSec policy assigned to a higher-level organizational unit, for any member computers of that organizational unit.

For information on how to define IPSec policies, see To add or edit IPSec policies.


Rules

IPSec rules govern how and when communication is secured. Each rule contains a list of filters, and a collection of security actions that will take place upon a match with that list.

  • IP filter lists
  • Filter actions
  • IPSec security methods
  • IPSec authentication
  • IPSec tunneling
  • IPSec connection types
  • Advanced IPSec settings


IP filter lists

An IP filter list triggers security negotiations based on a match with the source, destination, and type of IP traffic. This type of IP packet filtering enables a network administrator to precisely define what IP traffic will be secured. Each IP filter list contains one or multiple filters, which define IP addresses and traffic types. One IP filter list can be used for multiple communication scenarios.

IPSec requires both an inbound and outbound filter between the computers specified in the filter list. Inbound filters apply to incoming traffic, enabling the receiving computer to respond to requests for secure communication; or match traffic against the IP filter list. Outbound filters apply to traffic leaving a computer towards a destination, triggering a security negotiation which takes place before traffic is sent. For example, if Computer A wants to securely exchange data with Computer B:

  • The active IPSec policy on Computer A must have a filter for any outbound packets to Computer B. Source=A and Destination=B.
  • The active IPSec policy on Computer A must have a filter for any inbound packets from Computer B. Source=B and Destination=A.

Each peer must also have the reverse filter:

  • The active IPSec policy on Computer B must have a filter for any inbound packets coming from Computer A. Source=A and Destination=B.
  • The active IPSec policy on Computer B must have a filter for any outbound packets to Computer A. Source=B and Destination=A.

Filter settings

Each filter defines a particular subset of inbound or outbound network traffic which should be secured. You must have a filter to cover any traffic to which the associated rule applies. A filter contains the following settings:

  • The source and destination address of the IP packet. This may be configured at a granular level, using a single IP address or DNS name, or groups of addresses, subnets, or networks.
  • The protocol over which the packet is being transferred. This automatically defaults to cover all protocols in the TCP/IP protocol suite. However, it can be configured to an individual protocol level to meet special requirements, including custom protocols.
  • The source and destination port of the protocol for TCP and UDP. By default, all ports are covered, but this can be configured to apply to only a specific port.

For information on how to configure IP filter lists, see To add or edit IP filter lists.

Special filters may be required for certain types of traffic, such as SNMP traffic or security gateways. For more information, see Special IPSec considerations.


Filter actions

A filter action defines the security requirements for the data transmission. A filter action can be configured to:

  • Function as a passthrough policy ("Permit"). This does not allow negotiation for secure communication. IPSec simply ignores traffic in this case. This is appropriate for traffic from computers such as Web TV servers, which are solely broadcasting communication which should never be secured. Be sure to limit the IP filter list to a minimal scope when using this type of filter action, so you don't let traffic through which should be secured.
  • Block traffic ("Block"). This stops communication from a unauthorized computer. Be sure to use the smallest IP filter list you can when using a blocking filter action so that you do not block valid computers from communicating.
  • Enable communication with non-IPSec enabled computers ("Accept unsecured communication, but always respond using IPSec" and "Allow unsecured communication with non-IPSec aware computers"). This is a filter action which falls back to unsecured communication, if necessary. Again, you may need to limit the IP filter list to a minimal scope. Otherwise, any communications affected by the rule in which this filter action resides could result in data being sent without security if negotiation fails for any reason. If you are concerned about denial of service attacks, you might consider disabling these settings. However, communication with computers which cannot initiate IPSec, such as legacy systems, may be blocked.
  • "Enable Session Key Perfect Forward Secrecy (PFS)." This determines how a new key is generated. Enabling PFS ensures that a key used to protect a transmission cannot be used to generate any additional keys. In addition, the keying material for that key cannot be used to generate any new keys. Session Key PFS does not require a re-authentication, and is therefore uses less resources than Master Key PFS. When Session Key PFS is enabled, a new key exchange is performed to accumulate new key material before the new session key is generated.
  • Specify security requirements ("Use these security settings"). The list of security requirements which must be negotiated is called security methods. Each one defines which algorithms, security protocols, and key lifetimes should be negotiated.

For information on how to configure filter actions, see To add or edit filter actions.


IPSec security methods

Each security method defines the security requirements of any communications to which the associated rule applies. Creating multiple security methods increases the chance that a common method can be found between two computers.

The ISAKMP/Oakley service reads the list of security methods in descending order, and exchanges negotiation messages with the other peer until a common method is found.

Predefined security methods

  • High. Uses the Encapsulating Security Payload (ESP), to provide confidentiality (data encryption), authentication, anti-replay, and integrity, making this appropriate when high security levels are required. ESP does not provide integrity for the IP header (addressing). If you require both data and addressing protection, can create a custom security method. If you do not require encryption, use Medium.
  • Medium. Uses the Authentication Header (AH) protocol to provide integrity, anti-replay, and authentication. This is appropriate when your security plan calls for standard levels of security. AH provides integrity for both the IP header and data, but does not encrypt the data.

Custom security methods

Expert users can specify custom security methods, if both encryption and addressing integrity, stronger algorithms, or key lifetimes are required:

  • Security Protocols

    • Both AH and ESP may be enabled in a custom security method, if you require IP header integrity and data encryption. If you chose to enable both, you do not need to specify a second integrity algorithm for ESP; the algorithm you select for AH will provide integrity.

  • Integrity
    • Message Digest 5 (MD5), which produces a 128-bit key.
    • Secure Hash Algorithm (SHA), which results in a 160-bit key. Longer key lengths provide greater security, so SHA would be considered stronger.
  • Confidentiality
    • 3DES is the most secure of the DES combinations, and somewhat slower in performance. 3DES processes each block three times, using a unique key each time.
    • DES is to be used when the high security and overhead of 3DES is not necessary, or for interoperability. DES uses only 56 bits of keying material.
  • Key Lifetimes

    • Key lifetimes determines when a new key is generated, rather than how it is generated. You can specify a lifetime in kilobytes, seconds, or both. For example, if the communication takes 100,000 seconds and you specify the key lifetime as 1000 seconds, 10 keys will be generated to complete the transfer. This ensures that, even if attackers gain part of a communication, they will not be able to gain the entire communication. Automatic key regeneration is provided; configuration is optional. Note that any time a key lifetime is reached, the SA is also renegotiated in addition to the key refresh or regeneration.

Interoperability of encryption settings

When IPSec provides confidentiality of data between a Windows 2000-based computer that has the High Encryption Pack installed and another computer that does not use 3DES, your list of security methods must include DES. Otherwise, the computer that you are trying to communicate with may not be able to reach a common security agreement with your computer. If confidentiality of the data is not needed, you can select ESP format with an integrity algorithm only, and set the encryption to None. Alternately, you can select an AH format with an integrity algorithm.

For information on how to configure security methods, see To add or edit security methods.


IPSec authentication

Each rule defines a list of authentication methods. Each authentication method defines the requirements of how identities will be verified in communications to which the associated rule applies. The two peers must have at least one common authentication method or communication will fail. Creating multiple authentication methods increases the chance a common method between two computers can be found.

Only one authentication method may be specified between a pair of computers, regardless of how many are configured. If you have multiple rules which apply to the same pair of computers, you must configure the authentication methods list in those rules to enable the pair to use the same authentication method. For example, if a rule between a pair of computers specifies only Kerberos for authentication and filters only TCP data and, in another rule, specifies only Certificates for authentication and filters only UDP data, authentication will fail.

  • The Kerberos V5 security protocol is the default authentication technology. This method can be used for any clients running the Kerberos V5 protocol (whether or not they are Windows 2000-based clients) which are members of a trusted domain.
  • A public key certificate should be used in situations which include Internet access, remote access to corporate resources, external business partner communications, or computers which do not run the Kerberos V5 security protocol. This requires that at least one trusted certificate authority (CA) has been configured. Windows 2000 supports X.509 Version 3 certificates, including CA certificates generated by commercial certifying authorities.
  • A preshared key may be specified. This is a shared, secret key which is previously agreed upon by two users. It is quick to use and does not require the client to run the Kerberos V5 protocol or have a public key certificate. Both parties must manually configure IPSec to use this preshared key. This is a simple method for authenticating computers that are not based on Windows 2000, stand-alone computers, or any clients not running the Kerberos V5 protocol. Note that this key is for authentication protection only.

For information on how to configure authentication methods, see To define authentication methods.


IPSec tunneling

As discussed in Basics of IPSec tunneling, IPSec can perform Layer 3 tunneling for scenarios in which L2TP cannot be used.

Remote communications using tunneling

The receiving computer is the IPSec tunnel endpoint to the sender. In this example, the Remote Gateway is the tunnel endpoint in the Remote Client IPSec policy, and vice-versa.

If you are using the Layer 2 Tunneling Protocol (L2TP) for remote communications, the tunnel setting should be set to No IPSec tunneling, which is the default.

If you are specifying IPSec tunneling for interoperability, when one of the peers is not L2TP-capable, a tunnel setting can be configured for each rule. These settings are:

  • This rule does not specify an IPSec tunnel (default). Disables IPSec tunneling for communications covered by the rule.
  • The tunnel endpoint is specified by this IP Address.

For information on how to configure an IPSec tunnel, see To specify an IPSec tunnel.


IPSec connection types

For each IPSec rule, you must define which connection types on your computer will be affected. The connection types apply to all connections in Network and Dial-up Connections on the computer for which you are configuring IPSec policy.

Each rule has one connection type setting:

  • All Network Connections. The rule will apply to communications sent via any of the network connections you have configured on the computer.
  • Local Area Network (LAN). The rule will only apply to communications sent via LAN connections you have configured on the computer.
  • Remote Access. The rule will only apply to communications sent via any remote access or dial-up connections you have configured on the computer.

For information on how to configure connection types, see To specify connection types.

For information on configuring network connections on a computer, see Configuring a network or dial-up connection.


Advanced IPSec settings

The following topics review the concepts necessary for advanced policy settings, in case your deployment scenarios have special requirements which are not met by the default settings.

  • Key exchange
  • Key exchange methods


Key exchange

The strength of the keys which protect the key exchange phase is enhanced by the following features:

Key Lifetimes

Lifetime settings determine when a new key is generated. Any time a key lifetime is reached, the associated SA is also renegotiated. The process of generating new keys at intervals is called dynamic rekeying or key regeneration. Lifetimes allow you to force a the generation of a new key (regeneration) after a specific interval. For example, if the communication takes 100 minutes and you specify the key lifetime as 10 minutes, 10 keys will be generated, one every 10 minutes, during the exchange. Using multiple keys ensures that if attacker manages to gain the key to one part of a communication, the entire communication is not compromised. Automatic key regeneration is provided by default. Expert users may override the default and specify a master key lifetime in minutes, by session key, or Perfect Forward Secrecy.

Caution should be taken when setting very disparate key lifetimes, since they also determine the lifetime of the SA. For example, setting a master key lifetime of 8 hours (480 minutes), and a session key lifetime (set within a Filter Action) of 2 hours may result in an IPSec SA in place for almost 2 hours after the ISAKMP SA expired. This has the potential to occur if the new IPSec SA is generated right before the ISAKMP SA expires.

  • Session key limit

    • Repeated rekeying off the same master key may eventually compromise the key. For example, if Bob on computer A sends a message to Alice on computer B, and then a few minutes later sends another message to Alice, the same master key material can be re-used since a security association was recently established with that computer. If you want to limit the number of times this re-use occurs, expert users can specify a session key limit.

    Note that if you decide to enable Perfect Forward Secrecy for the master key, the Session key limit will be ignored; PFS forces key regeneration each time. For example, enabling Master key Perfect Forward Secrecy is the same as specifying a Session key limit of one.

    Note that, if you specify both a master key lifetime in minutes, and a session key limit, whichever interval is reached first will trigger a new key.

  • Master key Perfect Forward Secrecy (PFS)

    • Determines how a new key is generated. Enabling PFS ensures that a key used to protect a transmission, in whichever phase, cannot be used to generate any additional keys. In addition, the keying material for that key cannot be used to generate any new keys.

    Master key PFS should be used with caution as it requires re-authentication. This may cause additional overhead for any domain controllers in your network. It is not required to be enabled on both peers.

For information on how to configure key exchange settings and methods, see To configure key exchange and To create key exchange methods.


Key exchange methods

In addition to key properties, advanced users can set security methods for Phase I (Main Mode) of IKE negotiation. It is generally recommended that all the IKE settings (PFS, key lifetime) and security methods remain at the defaults, to avoid unnecessary overhead. This will provide a standard (medium) level of security. If your security plan calls for a high level of security, you may consider modifying the default security methods.

You can specify which algorithms will be used for integrity and confidentiality (optional). The same algorithms available for Phase II security methods are available here: MD5 and SHA for integrity; DES and 3DES for confidentiality.

Diffie-Hellman Groups

Diffie-Hellman groups are used to determine the length of the base prime numbers used during the key exchange. The strength of any key derived depends in part on the strength of the Diffie-Hellman group the prime numbers are based on:

Group 2 (medium) is stronger than Group 1 (low). Group 1 will provide 768 bits of keying material, while Group 2 will provide 1,024 bits. If mismatched groups specified on each peer, negotiation will fail. The group cannot be switched during the negotiation.

A larger group results in more entropy and therefore a key which is harder to break.

The Diffie-Hellman group is configured as part of the Phase I (Main Mode) key exchange settings and is considered a master key. New keys generated during the data protection Phase II (Quick Mode) are derived from the Diffie-Hellman Phase I master key material, unless Phase II Perfect Forward Secrecy is being used.

For information on how to configure key exchange settings and methods, see To configure key exchange and To create key exchange methods.


IPSec scenarios

This section describes how to deploy common IPSec configurations. Each scenario reviews the IPSec policy settings that are necessary for successful, secured communication. While your network configuration may be different than those discussed here, the basic concepts apply.

This section discusses the following IPSec scenarios:

  • Domain, Peer to Peer: IPSec
  • Remote communications: IPSec


Domain, Peer to peer: IPSec

In order to provide security for groups which normally exchange highly sensitive information, you often had to segment the intranet. Groups of computers on different, physical segments prevented security violations. IPSec provides protection while still allowing groups of secure computers to reside within the same physical intranet.

Domain security with IPSec

This figure represents a domain comprised of computers in the finance department of an enterprise. Most intranet clients do not need to communicate securely. However, a group of servers in the network store highly sensitive information which some intranet clients will need to access. All computers have computer accounts in an Active Directory domain or object. The computers are grouped into Active Directory organizational units for security reasons. This enables the appropriate assignment of IPSec policies, based on the function of the computers:

  • Servers which store and exchange highly sensitive information belong to the Highest Security Servers organizational unit.
  • Other servers which may use unsecured communication to enable data exchange with non-IPSec capable computers in the domain belong to the Secure Servers organizational unit.
  • Clients which require the ability to appropriately respond when secure communications are required belong to the Secure Computers group.
  • Clients which do not require the ability to talk to the secure servers belong to the default Computers group.

When you group computers into organizational units, you can assign IPSec policies to only those computers which require IPSec. It also allows the appropriate level of security to be assigned, avoiding excessive security overhead. In this scenario, Active Directory stores the IPSec policies for all computers.

High security between the clients and the domain controller is unnecessary: Kerberos-related exchanges between the clients and the domain controller are already encrypted, and the IPSec policy transmission from Active Directory to the member computers is protected by Windows LDAP security.

IPSec should be combined here with access control security. User permissions are still a necessary part of using security to protect access to the file shares available on any of the Highest Security of Very Secure Servers. IPSec secures the network level traffic, so that attackers can not interpret or modify the data.

Predefined Configurations

Windows 2000 provides a set of predefined IPSec configurations. By default, all predefined policies are designed for computers that are members of a Windows 2000 domain. Following are descriptions of Windows 2000 predefined policies. The pre-defined policies, filter lists, and filter actions provided in both the local computer and Active Directory are not intended for immediate use. They are intended to show differences in behavior that are possible with different policy settings.

Computers: Client (Respond Only)

This policy is for computers that do not secure communications for the majority of the time. For example, intranet clients may not require IPSec except when it is requested by another computer. This policy enables the computer on which it is active to respond appropriately to requests for secured communications. It contains a Default Response rule, which enables negotiation with computers requesting IPSec. Only the requested protocol and port traffic for the communication is secured.

Secure Servers: Server (Request Security)

This policy is for computers that secure communications for the majority of the time, such as servers that transmit sensitive data. This policy enables the computer to accept unsecured traffic, but always attempt to secure additional communications by requesting security from the original sender. This policy allows the entire communication to be unsecured if the other computer is not IPSec-enabled.

Highest Security Servers: Secure Server (Require Security)

This policy is for computers that always require secure communications, such as a server that transmits highly-sensitive data. This policy accepts unsecured, incoming communications, and always secures outgoing traffic.


Remote communications: IPSec

Secure, remote communication is achieved by combining the Layer 2 Tunneling Protocol (L2TP) and IPSec. L2TP is used to build the tunnel through which the data will travel, and IPSec secures the data.

Roving clients

A common requirement is securing communications between remote clients and the enterprise network. This may be a sales consultant who spends most of the time on the road, or an employee working from a home office.

Remote access clients using L2TP and IPSec

In this figure, the remote gateway is a server which provides edge security for the enterprise's intranet. The remote client represents a roving user who needs regular access to network resources and information. The client may or may not be going through an Internet service provider (ISP); it is shown here to represent the path of communication if the client is using an ISP to access the remote gateway. L2TP is combined with IPSec to provide a simple, efficient way to build the tunnel and protect the data across the Internet. The tunnel itself is between the Remote Client and the remote gateway, since it is used only to protect communication through the Internet.

Branch offices

A large enterprise will often have multiple sites which need to communicate, for example, a corporate office in New York, and a sales office in Washington. As in the previous scenario, L2TP is combined with IPSec to provide the tunnel and protect the data between the sites.

L2TP/IPSec Router-to-Router Tunneling

In this figure, the Windows 2000 routers provide edge security and a communication route between the two sites. The routers may have a lease line, demand dial, or other type of connection. The tunnel runs between the routers only, since it is being used to protect communication through the Internet. However, the virtual private network (VPN) extends between the intranet computers at each site that are exchanging data.

Securing remote communication

Instead of configuring IPSec policies, these remote communication scenarios require configuration of the L2TP security properties in Windows 2000.

When L2TP is configured to use IPSec for security:

  • The required IP filter and filter action lists are dynamically set in the IPSec Policy Agent for the duration of the connection.
  • Authentication is determined by L2TP, which requires a computer public key certificate and its associated private key.
  • Default key exchange settings are in effect.

The level of Internet Protocol security that is used for the duration of the connection is dependent upon the L2TP security configuration:

  • No encryption. IPSec still requires a certificate and will negotiate AH.
  • Optional encryption. If the other computer requests or requires secured communication, IPSec can offer security levels ranging from ESP/3DES to AH/MD5.

    • Session key Perfect Forward Secrecy is not enabled unless the other computer requests it.

    You can fall back to unsecured communication.

  • Required encryption. The computer will require secured communication. The security that is offered is identical to that under optional encryption, except that you cannot fall back to unsecured communication.

For information on configuring L2TP encryption settings, see Dial-up remote access security.


Resources

The Microsoft Windows 2000 Resource Kit contains advanced information on how to configure, administer, optimize, and troubleshoot IPSec.

For other sources of additional information for IPSec, see these sources:

  • Internet drafts contain detailed, technical information on IPSec architecture, including the algorithms and protocols used. The Internet drafts directories provide authors with the ability to distribute and solicit comments on documents that they may eventually submit for publication as an RFC. To see the most current Internet drafts on IPSec architecture and mechanisms, see the IETF Web site.
  • RFCs are an evolving series of reports, proposals for protocols, and protocol standards used by the Internet community. IPSec standards are defined in RFCs published by the Internet Engineering Task Force (IETF) and other working groups. To see the most current Internet Drafts on IPSec architecture and mechanisms, see the IETF Web site.
  • For the latest information on Windows 2000 Security, see the Microsoft Security Advisor page on the Microsoft Web site (http://www.microsoft.com/).
  • For information on the standards and technology for IPSec, as well as other computer security topics, see the Computer Security Division of the National Institute of Standards and Technology Information Technology Laboratory.

Note


Troubleshooting

What problem are you having?

Verifying the network connection between two computers.

Cause: Separate network problems may be causing what appears to be communication failures related to Internet Protocol security (IPSec).

Solution: The TCP/IP utility, ping, can be used to determine if IPSec-secured communication can take place when a predefined policy is assigned to a computer. This will allow you to separate network problems from IPSec issues. When verifying network connectivity, make sure the computer has no IPSec policy assigned. Stopping the policy agent is a good way to make sure there is no policy.

See also: To verify valid IPSec connections.

Verifying the IPSec policy on a computer is active and in effect.

Cause: IPSec-secured communication may not be responding as expected.

Solution: You can use several tools to view which IPSec policy is in effect on the computer.

See also: Verifying IPSec policy assignment.

Verifying the active IPSec policy is responding properly.

Cause: Separate network problems may be causing what appears to be IPSec-related communication failures.

Solution: The TCP/IP utility, ping, can be used to determine if IPSec-secured communication can take place when a predefined policy is assigned to a computer. This will allow you to separate network problems from IPSec issues. If the ping is trying to negotiate security, then the computer has policy to protect that traffic (which uses ICMP to the destination) and is trying to negotiate a security association with the peer computer.

See also: Testing IPSec policy assignment.

Basic IPSec troubleshooting.

Cause: IPSec-secured communication is failing. This may be due other network problems, IPSec policy consistency or assignment issues, or because the computer has been removed from the domain and is now using local policy. Note that failure in core networking services, such as DHCP, DNS, and WINS, can cause unpredictable IPSec failures.

Solution: The TCP/IP utility, ping, can be used to separate other network issues from IPSec-related problems. Restarting the policy agent may be necessary to force a policy update. You can use several tools to view which IPSec policy is in effect on the computer. Make sure the traffic that is supposed to be protected has an appropriate filter. Make sure both sides have a matching filter action. Also, make sure both sides have matching authentication methods.

See also: To restart the IPSec Policy Agent; To verify valid IPSec connections; Verifying IPSec policy assignment; Testing IPSec policy assignment.

Errors in Event Viewer related to IPSec.

Cause: There may be a problem with the IPSec Policy Agent, or the Oakley service.

Solution: To determine why IPSec is not functioning correctly, you can view the Security Audit log for messages which may help pinpoint the origin of the problem.

See also: IPSec: Security Audit Log.

Verifying successful, IPSec-secured communications and statistics.

Cause: A pattern of failures when attempting IPSec-secured communications.

Solution:Use IPSec Monitor to confirm whether your secured communications are successful. It provides statistics that aid in tuning IPSec performance and troubleshooting. It can also be used in conjunction with Network Monitor. Use Network Monitor to view security protocol transmissions that have been transferred over the network. Network Monitor displays IPSec packets as ISAKMP, AH, or ESP.

See also: Using IPSec Monitor; Using Network Monitor with IPSec.

Policy location error.

Cause: This error may occur if the policy agent is retrieving policy from the incorrect storage location.

Solution: Examine the policy agent logging events in Event Viewer, and the policy agent log itself for messages which indicate where policy is being retrieved from: the local registry or the domain. If the HKLM\SOFTWARE\Policies\Microsoft\Windows\IPSec\GPTIPSECPolicy is set, then policy will be downloaded from the directory service, otherwise, it will be downloaded from the registry. The local computer will also warn the user that the domain policy is in effect when it starts. If the policy is supposed to be domain based policy, but it is not being downloaded, then try deleting HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Group Policy\History and then restarting the computer.

Caution

  • Incorrectly editing the registry may severely damage your system. Before making changes to the registry, you should back up any valued data on the computer.

See also: Wrong IPSec policy location error.

Policy mismatch error.

Cause: If negotiations are failing, it may be due to incompatible IPSec policy settings.

Solution: You can use several tools to view which IPSec policy is in effect on the computer.

See also: IPSec policy mismatch error.

Bad SPI error in Event Viewer.

Cause: This error may occur if key lifetime values are not set properly or the security association has expired.

Solution: To determine and correct the problem, use IPSec Monitor to examine the number of rekeys. This event is usually logged when a security association (SA) has been deleted on one side, but the other side is still sending traffic on this security association. It is usually a temporary situation that fixes itself and occurs on systems that are busy.

See also: IPSec: Bad SPI error in Event Viewer.

IPSec Monitor shows no security associations established.

Cause: This may be due to soft security associations between the computers, which prevent hard (secured) security associations from being established.

Solution: You may have to restart the IPSec Policy Agent. Confirm that you need the Allow unsecured communications with non IPSec-aware computers filter action option, which causes soft security associations to show up with computers that do not respond to IPSec requests.

See also: Soft SAs are preventing hard SAs.

Restarting the IPSec Policy Agent.

Cause: This may be necessary as part of another troubleshooting step. It can be used to clear old or soft security associations or to force a policy download from the domain or local policy store.

Solution: Restart the IPSec Policy Agent.

See also: To restart the IPSec Policy Agent.

IPSec component files are missing.

Cause: The files necessary for IPSec components such as ISAKMP/Oakley, the IPSec Policy Agent, or the IPSec driver have been removed or deleted.

Solution: Reinstall TCP/IP.

See also: Reinstalling IPSec components.


IPSec: Bad SPI Messages in Event Viewer

This error may occur if a key lifetime value is set too low, or if the SA has expired but the sender continues to transmit data to the receiver. To determine and correct the problem, use IPSec Monitor to examine the number of rekeys. For more information, Using IPSec Monitor.

If the number of rekeys is very large compared to the amount of time the connections have been active, set the key lifetimes in the policy to be longer. Good values for high-traffic Ethernet connections are greater than 50 megabytes and greater than five minutes.

This may not entirely eliminate bad SPI's, but should significantly reduce the occurrences.


IPSec: Security Audit Log

An event is any significant occurrence in the system that requires notification. Event logging starts automatically each time you start Windows 2000. Use Event Viewer to view the System Log for troubleshooting problems with the policy agent. You should also check the Security Log for Oakley informational messages and messages regarding failed IPSec communications. For more information Event Viewer, see View Event Logs.

For example, to determine why one of the services is not running, you can filter the System Log to only display messages pertaining to the Policy Agent. You can also display a particular type of Policy Agent event message. You can use the same filtering on the Security Log for Oakley messages.

  1. Expand the Event Viewer folder to display the System and Security logs.
  2. Use the Action menu to apply filtering.

System Log messages in the Event Log

System Log Informational Event 279

  • Source: PolicyAgent
  • Category: None

Indicates whether an IPSec policy is in effect on the computer, the source of the IPSec policy (local, domain), and Active Directory polling interval, if policy source is the domain. Also indicates that a change to an IPSec policy has been detected in the policy source. In this case, the text will show as Updating IPSec Policy.

System Log Error Event 284

  • Source: PolicyAgent
  • Category: None

Indicates the Policy Agent was unable to contact Active Directory for the domain in which the computer is a member.

Security Log messages in the Event Log

This log contains messages from ISAKMP/Oakley.

Security Log Success Audit event 541

  • Source: Security
  • Category: Logon/Logoff

Indicates an IPSec hard security association has been established. (Soft security associations are not audited.)

Security Log Success Audit event 542

  • Source: Security
  • Category: Logon/Logoff

Indicates an IPSec hard or soft security association has ended.

Application Log messages in the Event Log

This log contains messages from ISAKMP/Oakley. The following messages indicate that a domestic version of Windows 2000 has attempted to negotiate higher security than an export client can support.

Application Warning Event 541

  • Source: Oakley
  • Category: None

Indicates the export client cannot generate domestic-strength key material; the resulting negotiation agrees only on export-strength key material.

Application Warning Event 542

  • Source: Oakley
  • Category: None

Indicates the export client can not perform encryption stronger than DES; the resulting negotiation agrees only on DES (provided the other computer can do DES).


Soft SAs are preventing hard SAs

If IPSec Monitor reveals that secured (or hard) security associations (SAs) are not being established, it may be due to soft SAs between the peers. This could prevent hard SAs from being established. To determine if this is the cause, run IPSec Monitor on one of the peers. If an SA exists between the two computers, and the security setting is "None", this indicates a soft SA. This soft SA will remain on that computer as long as traffic is being regularly sent between the two computers. For more information, Using IPSec Monitor.

To prevent this, stop all traffic until the SA times out (usually 5 minutes), verify the SA is gone in IPSec Monitor, then start traffic again. A hard SA should automatically be established if policies are compatible. Then, restart the policy agent to delete all soft SAs you must restart the policy agent. For more information, Restarting the IPSec Policy Agent.


IPSec Internet drafts

Internet drafts contain detailed, technical information on IPSec architecture, including the algorithms and protocols used. The Internet draft directories provide authors with the ability to distribute and solicit comments on documents that they may eventually submit for publication as an RFC.

To see the most current Internet drafts on IPSec architecture and mechanisms, see IPSec-related Web sites.

IPSec Internet Draft References

The IPSec implementation in Windows 2000 adheres to, but does not fully implement the following IETF IPSec architecture document:

  • PKI Requirements for IP Security (draft-ietf-ipsec-pki-req-01.txt)

The IPSec implementation of the Encapsulating Security Payload protocol in Windows 2000 is based on the following IETF IPSec drafts:

  • draft-simpson-cbc-01.txt
  • draft-ietf-ipsec-ciph-des3-00.txt
  • draft-hoffman-des40-03.txt


Using IPSec Monitor

Use IPSec Monitor to confirm whether your secured communications are successful. It displays the active security associations on local or remote computers. For example, you can use IPSec Monitor to determine whether there has been a pattern of authentication or security association failures, possibly indicating incompatible security policy settings. The statistics showing the authentication failures may indicate negotiation failed because of mismatched authentication methods.

IPSec Monitor can be run on the local computer, or it can be run remotely if you have a network connection to the remote computer.

To start IPSec Monitor

  1. Click Start, then click Run.
  2. Type ipsecmon ComputerName, where ComputerName is the computer that you want to monitor.
  3. Click Options to set the refresh rate.

An entry will be displayed for each active security association. The information contained in each entry includes the name of the active IPSec policy, the active filter action and IP filter list (including details of the active filter), and the tunnel endpoint, if one was specified.

It can also provide a multitude of statistics that aid in tuning IPSec performance and troubleshooting, including:

  • The number and type of active security associations.
  • The total number of Master and Session keys in (Main and Quick modes). Successful IPSec security associations will initially create one master key and one session key. (Subsequent key regenerations are shown as additional session keys.).
  • The total number of confidential (ESP) or authenticated (ESP or AH) bytes sent or received. Since ESP provide authenticity and confidentiality, both counters are incremented.
  • The total number of soft associations.

The refresh rate is the only configurable option. By default, the statistics will update every fifteen seconds. The statistics are accumulated with each communication which uses IPSec.


IPSec policy mismatch error

If negotiations are failing, it may be due to incompatible IPSec policy settings. Follow these steps to correct the problem:

  1. Run the Event Viewer and examine the Security Log. Recent events will include attempts at Oakley negotiation, with a description of their success or failure. Do this on each computer attempting to participate in the communication. For more information on Event Viewer, see View Event Logs.
  2. Try a policy integrity check on each computer. For more information, see To test policy integrity.
  3. Determine the cause of policy mismatch:
    • Verify that authentication methods are compatible.
    • Verify that there is at least one compatible security method.
    • If you are using IPSec tunneling, verify the tunnel endpoint settings (the IP Address or DNS name) are correct, and that the endpoint computers are operational and functioning correctly.


Using Network Monitor with IPSec

Use Network Monitor to view the IPSec security protocol transmissions. Network Monitor captures information that has been transferred over a network at any given time. For information on installing Network Monitor, see To install Network Monitor.

  • Viewing Authentication Header Packets

    • Network Monitor can display packets that are secured with Authentication Header (AH) as a TCP, ICMP or UDP packet with an AH header. In the Next Header field, AH is displayed as IP protocol number 51 (in decimal).

  • Viewing Encapsulating Security Payload Packets

    • Since the Encapsulating Security Payload (ESP) encrypts the original header, Network Monitor displays them as ESP packets (as opposed to TCP, ICMP or UDP). The data payload in the ESP packet will not be readable since it is encrypted. In the Next Header field, ESP is displayed as IP protocol number 50 (in decimal). To see that the data has been encrypted, view the Hex pane. IPSec has created a hash of the ICMP and Data fields of the frame, and encrypted the data fields of the frame.

  • Viewing ISAKMP or Oakley Packets

    • In the Next Header field, ISAKMP/Oakley is displayed as UDP port number 500 (in decimal).

To view IPSec packets, you will need to configure Network Monitor with the addresses of the source and destination computers for which you want to communicate. For example, to filter IP secured traffic between Computer A with IP address 172.16.0.10 and Computer B with IP address 172.16.0.5, you must specify that address pair in a capture filter. For information on configuring capture filters, see Using capture filters and To capture network frames.

Notes

  • Note that, while you may choose to use your own network sniffer instead of Network Monitor, it may not be able to interpret the IPSec-specific portions of the IP packets.
  • You can assign the predefined IPSec policies and use ping to generate traffic to view with Network Monitor when you are doing your deployment testing. You may have to type it more than once, because the ping command has a very short time out, and there is some delay in establishing the IPSec association between the two computers.


Restart the IPSec Policy Agent

To restart the IPSec Policy Agent

  1. In Services, double-click the IPSec Policy Agent.
  2. Click Restart.

Note

  • For information on starting Services, see Related Topics.
  • This may be necessary to clear up old security associations (SAs), soft SAs, or to force a policy download from Active Directory to domain clients.
  • Restarting the IPSec Policy Agent will also force the restart of the IPSec driver.
  • Use Event Viewer to determine possible causes of failure if the Policy Agent will not start.

Related Topics


Verify valid IPSec connections

To verify valid IPSec connections

  1. Open Command Prompt.
  2. Type ping IP_Address

    3. where IP_Address is the address of the computer with which you are trying to communicate.

Notes

  • This procedure determines if standard, unsecured communication can take place. This will allow you to separate network problems from IPSec issues.
  • You should receive four replies to the ping. This verifies that you can communicate with your partner. If you see a "Destination host unreachable" message, it indicates the other computer cannot be contacted at all. If you see a "Request timed out" message, it indicates the computer can be contacted, but has some policy applied which is blocking communication.


Test IPSec policy assignment

When using the Server (Request Security) or Secure Server (Require Security) policies, the ICMP packet sent by the ping command will match the IP Filter List in these policies, and IPSec will attempt to apply security to the ping command. However, this process takes longer than the ping command normally waits for a response. Because of this, you may temporarily receive erroneous "Destination unreachable" messages. You may see an "Unknown host" message if you ping by the computer's DNS name and have an active IPSec policy that blocks DNS traffic. If you are trying to contact another computer which is not IPSec-capable, you may receive the message "Request timed out". To avoid these issues, exempt ICMP traffic in all your IPSec policies.

IPSec Monitor can be also used to verify the results of a ping command. A security association will be listed whenever a ping command is successful, regardless of whether it was a hard or soft security association.

The following results are expected when you carry out a ping command on a computer with these predefined IPSec policies:

Client (Respond Only)

Action | Result
Ping a non-IPSec enabled computer | Normal results. Client (Respond Only) only negotiates security in response to security requests.
Ping an IPSec-enabled computer | Dependent on the other computer's IPSec policy. If the other computer is using Server (Request Security), then the computer will try to respond securely to the ping. If the security negotiation can be done in time, the ping will be successful.

Server (Request Security)

Action | Result
Ping a non-IPSec enabled computer | Initially, "Destination Unreachable " appears while negotiation for security is attempted. However, after two seconds, a soft security association (SA) will be established and ping traffic will succeed. (The actual time may be longer if you used a DNS name for the ping command, since the DNS resolution requires a soft association to be established with the DNS servers first.) The default settings of this policy will allow communication to fall back to clear if the other computer does not respond, and the ping command will succeed, once a soft association is created.
Ping an IPSec-enabled computer | Initially, a "Destination Unreachable" message is displayed while negotiation for security is attempted. However, after two seconds, a soft security association will generally be established and ping traffic will succeed. (The actual time may be longer if you used a DNS name for the ping command since the DNS resolution requires a soft association to be established with the DNS servers first.) If negotiations fail, IPSec may install a blocking filter for that destination IP address and all traffic will be blocked for five minutes, during which the ping will return time outs. However, the default settings of this policy will allow communication to fall back to clear if the other computer does not respond, and the ping will succeed once a soft association is created.

Secure Server (Require Security)

In both instances that are discussed for Secure Server (Require Security), the ping will not be able to resolve the DNS name and it will fail, unless the DNS servers are IPSec-enabled or there are exemptions in IPSec policy for DNS traffic.

Action | Result
Ping a non-IPSec enabled computer | Ping may not be able to resolve the DNS name and "Unknown computer [computer]" appears. If you activated the Secure Server (Require Security) policy immediately prior to the ping, and the DNS name in the local DNS cache, then the name will resolve and the ping will immediately display four "Destination Computer Unreachable" messages.
Ping an IPSec-enabled computer | Ping may not be able to resolve the DNS name. If it succeeds, secure ping traffic will succeed after 2-4 seconds. If it fails, then IPSec installs a blocking filter for that destination IP address and all traffic will be blocked for five minutes, during which a ping command will return time outs.


Reinstall IPSec components

If the files necessary for IPSec components such as ISAKMP/Oakley, the IPSec Policy Agent, or the IPSec driver have been removed or deleted, you can reinstall the IPSec components by removing and reinstalling TCP/IP. The IPSec components will be reinstalled as part of the Internet Protocol installation. For procedural information on how to remove and reinstall the Internet Protocol, see To remove TCP/IP.


IPSec RFCs

RFCs are an evolving series of reports, proposals for protocols, and protocol standards used by the Internet community. IPSec standards are defined in RFCs published by the Internet Engineering Task Force (IETF) and other working groups.

IPSec RFC References

  • RFC 1828: IP Authentication using Keyed MD5
  • RFC 1829: The ESP DES-CBC Transform
  • RFC 2085: HMAC-MD5 IP Authentication with Replay Prevention
  • RFC 2104: HMAC: Keyed-Hashing for Message Authentication
  • RFC 2401: Security Architecture for the Internet Protocol
  • RFC 2402: IP Authentication Header
  • RFC 2403: The Use of HMAC-MD5-96 within ESP and AH
  • RFC 2404: The Use of HMAC-SHA-1-96 within ESP and AH
  • RFC 2406: IP Encapsulating Security Payload (ESP)
  • RFC 2407: The Internet IP Security Domain of Interpretation for ISAKMP
  • RFC 2411: IP Security Document Roadmap

To see the most current Internet drafts on IPSec architecture and mechanisms, see IPSec-related Web sites.


IPSec policy utilities

Several utilities are provided for general, IPSec policy administrative tasks. All of the following tasks are accessed through the IP Security Policy Management console.

Restore Default Policies

This utility is provided in order to restore the original, predefined configurations, should you modify or delete them beyond regression. Any current modifications you have made to any of the default policies and rules will be lost. For more information, see To restore predefined configurations.

Import/Export Policies

This utility allows you to easily transfer policies from one computer to another, in situations where you do not have a domain; for example, a workgroup. This saves the administrative task of manually recreating the policy settings on each workgroup member. It also allows backup of current policies, in case they need to be restored later. For more information, see To export an IPSec policy or To import an IPSec policy.

Policy Integrity Check

This utility verifies that changes made to any policy settings are properly reflected. It is applicable to IPSec policies you have assigned to Group Policy, which are then propagated by Group Policy to the computer accounts in the Group Policy object. If you find that the IPSec policy you assigned is not being propagated to the computer accounts in the Group Policy object, this task is a good troubleshooting step, especially if multiple administrators are editing and assigning IPSec policy. For more information, see To test policy integrity.


Verify IPSec policy assignment

Verifying IPSec policy assignment

You can use the following tools to verify that the assigned policy is active.

IPSec Monitor

  1. Click Start, then Run.
  2. Type ipsecmon ComputerName

When IPSec Monitor opens, you will see a message in the lower, right corner indicating whether IPSec is enabled on the computer. For IPSec to be enabled, a policy must have been assigned. However, no policies will be listed in the IPSec Monitor Security Association list unless a security association (SA) with another computer is currently active.

Security Settings

The local policies in effect can be viewed by opening a console with Group Policy and displaying the security settings and policies currently in effect on the computer. For more information, see View effective security settings. For information on assigning IPSec policy, see To assign IPSec policy to Group Policy overview or To activate IPSec policy on a computer.

Event Viewer and the Security Audit Log

The IPSec Policy Agent makes entries to the System Log to indicate the source of its policy. It also indicates the polling interval as specified by the active policy for checking for policy changes in Active Directory. Administrators who edit the active IPSec policy on the local computer cause the changes to take effect immediately.

You can also see whether the computer is using local policy or policy from Active Directory by viewing the Event Log. Specifically, the System Log informational entry by the IPSec Policy Agent.

For information, see Event Viewer and the Security Audit Log.

TCP/IP Properties

By displaying the properties for the Internet Protocol (TCP/IP), you can see the active IPSec policy. If the computer is running local IPSec policy, the name will be displayed in an editable form. If the computer is running policy assigned through Group Policy, the name will be unavailable, and cannot be edited.


IPSec-related Web sites

The following Web sites contain links which relate to IPSec:

  • The International Engineering Task Force Web site. The IPSec Working Group page contains links to RFCs, Internet Drafts, and other IPSec-related links at the IETF Web site.
  • For information and the latest recommendations on Windows 2000 security deployment, see the Microsoft Security Advisory Web pages on the Microsoft Web site. (http://www.microsoft.com/)
  • For information on the standards and technology for IPSec, as well as other computer security topics, see the Computer Security Division of the National Institute of Standards and Technology Information Technology Laboratory.

Note


Wrong IPSec policy location error

This error may occur if the policy agent is retrieving policy from the incorrect storage location. To determine from where policy is being retrieved, and correct the problem:

  1. Using Event Viewer, examine the System Log. (For more information about Event Viewer, see View Event Logs.) If you are using policies stored in Active Directory, polling events should be displayed.
  2. If you are using domain policy but want to use local policy:
    • Leave the domain and reboot.
    • Move the applicable Group Policy object in the directory so you do not receive domain policy.
  3. You can also examine the policy agent log. This may require turning on the registry settings for the log.
  4. View the Systemroot\IPSecPA.log file in a text editor. If the last few lines do not indicate the policy source, proceed to the next step.
  5. At a command prompt, type the following exactly as it appears. These commands are case-sensitive and will fail if case is incorrect:

    findstr Using systemroot\ipsecpa.log

    6. The log will indicate whether you are using Group Policy or local computer policy.