↓Skip to main content

IPSec Mode

·1289 words·7 mins
lab1918
Author
lab1918

IPsec, or Internet Protocol Security, operates primarily in two modes: Transport Mode and Tunnel Mode. These modes determine how the IPsec protocol secures data traffic.

  1. Transport Mode:

    • In this mode, IPsec encrypts only the payload of the IP packet, not the IP header.
    • It’s commonly used for end-to-end communication between two hosts.
    • The original IP header remains intact, and a new header is inserted after the IP header.
    • Ideal for client-to-server communication and when both ends have fixed IP addresses.

  2. Tunnel Mode:

    • IPsec in tunnel mode encrypts the entire IP packet.
    • A new IP header is added to the encrypted packet, making it appear to the network as a new packet.
    • This mode is used in virtual private networks (VPNs) where one or both ends can be a network gateway, like a router or firewall.
    • Ideal for network-to-network communications, especially when the end points may not have fixed IP addresses or are behind NAT (Network Address Translation).

Both modes use IPsec’s security protocols - Authentication Header (AH) and Encapsulating Security Payload (ESP) - to provide data confidentiality, data integrity, and authentication. The choice between transport and tunnel mode depends on the specific requirements of the network configuration and the security objectives.

To illustrate how packets are structured in IPsec’s Transport and Tunnel modes, let’s consider a simple example. Assume we have an original IP packet that needs to be secured, with the following basic structure:

  • IP Header (Source IP, Destination IP, etc.)
  • Data Payload (The actual data being transmitted)

IPsec Transport Mode Packet Structure
#

In Transport Mode, IPsec encrypts only the Data Payload of the packet, leaving the original IP Header unmodified. The packet structure after applying IPsec in Transport Mode would look something like this:

  • Original IP Header (Unchanged)
  • IPsec Header (Added by IPsec for security, could be AH or ESP)
  • Encrypted Data Payload (The original data payload, now encrypted)

Here, the IPsec Header is inserted after the original IP Header, and only the Data Payload is encrypted. This mode is commonly used for end-to-end communication between two hosts.

+-------------------------+
| Original IP Header      |
+-------------------------+
| IPsec Header (AH/ESP)   |
+-------------------------+
| Encrypted Data Payload  |
+-------------------------+

Explanation:

  • Original IP Header: Remains as it is, carrying the source and destination IP addresses.
  • IPsec Header (AH/ESP): Inserted after the original IP header, providing security functions.
  • Encrypted Data Payload: The original data payload, now encrypted by IPsec.

In Transport Mode, only the payload is encrypted, and the original IP header is left intact.

IPsec Tunnel Mode Packet Structure
# 

+---------------------------------+
| New IP Header                   |
+---------------------------------+
| IPsec Header (AH/ESP)           |
+---------------------------------+
| Encrypted Packet                |
| +-------------------------+     |
| | Original IP Header      |     |
| +-------------------------+     |
| | Data Payload            |     |
| +-------------------------+     |
+---------------------------------+

Explanation:

  • New IP Header: Created by IPsec, with potentially different source and destination IPs (usually representing the gateways).
  • IPsec Header (AH/ESP): Provides security functions to the encapsulated packet.
  • Encrypted Packet: This includes the entire original packet (Original IP Header and Data Payload) encrypted as a single unit.

In Tunnel Mode, IPsec encrypts the entire original IP packet (both the IP Header and the Data Payload) and then adds a new IP Header on top. The packet structure in Tunnel Mode would be:

  • New IP Header (Added by IPsec, with new source and destination IPs typically representing the gateways)
  • IPsec Header (Could be AH or ESP, providing security to the encapsulated packet)
  • Encrypted Original IP Packet (This includes both the Original IP Header and the Data Payload, all encrypted as a single unit)

In this mode, the entire original packet becomes the payload of a new IP packet. Tunnel Mode is often used in site-to-site VPN scenarios where entire networks are connected over the internet, and the internal structure of each network is hidden.

IPsec AH Packet Format
#

The IPsec Authentication Header (AH) provides authentication, integrity, and optionally anti-replay protection for IP packets. It doesn’t encrypt the payload but ensures that the packet has not been tampered with in transit. The AH is inserted into the IP packet in either transport or tunnel mode. Here’s the format of an IPsec AH packet:

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Next Header   |  Payload Len  |          Reserved             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                 Security Parameters Index (SPI)               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Sequence Number Field                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
+                Authentication Data (variable)                 +
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Explanation:

  • Bits 0-7 (Next Header): Identifies the type of the next header.
  • Bits 8-15 (Payload Len): Length of AH in 4-byte units, minus 2.
  • Bits 16-31 (Reserved): Reserved for future use, set to zero.
  • Bits 32-63 (SPI, Security Parameters Index): A value that, in combination with the destination IP address and security protocol (AH), uniquely identifies the security association.
  • Bits 64-95 (Sequence Number Field): A counter field that increases by one for each packet sent. Used to prevent replay attacks.
  • Bits 96+ (Authentication Data): Contains the integrity check value. The length of this field is variable, depending on the authentication algorithm used.

This format is aligned with the typical structure used in RFCs for defining protocol packet formats, with each line representing 32 bits (4 bytes) of the header. The AH header provides integrity and authentication, but does not encrypt the payload data. The RFC format is designed to be precise and unambiguous, which is essential for protocol implementation and interoperability.

IPsec ESP Packet Format
#

The Encapsulating Security Payload (ESP) in IPsec is designed to provide confidentiality, data origin authentication, integrity, and anti-replay protection. The ESP packet has a specific format, different from the Authentication Header (AH). Here’s the format of an IPsec ESP packet:

0                   1                   2                   3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                Security Parameters Index (SPI)                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Sequence Number Field                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Payload Data (variable)                    |
~                                                               ~
|                                                               |
+               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|               |     Padding (0-255 bytes)                     |
+-+-+-+-+-+-+-+-+               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                               |  Pad Length   | Next Header   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                  Authentication Data (variable)               |
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Details of ESP Header Components:

  • Security Parameters Index (SPI): A value that, in combination with the destination IP address and security protocol (ESP), uniquely identifies the security association.
  • Sequence Number Field: A counter field that increases by one for each packet sent. Used to prevent replay attacks.
  • Payload Data: The data being transferred, which can be encrypted.
  • Padding: Used to ensure that the data (Payload + Padding) is a multiple of the cipher block size, or to align the next header.
  • Pad Length: The length of the padding in bytes.
  • Next Header: Identifies the type of the next header after the ESP trailer.
  • Authentication Data: Contains the integrity check value and optional authentication. This field is variable in length and may not be present if authentication is not used.

Notes:
#

  • Encryption: The Payload Data, Padding, Pad Length, and Next Header are all encrypted.
  • Authentication: The Authentication Data field provides integrity and authenticity for the entire packet, including the ESP header, but excluding the Authentication Data itself.
  • Use in Modes: Like AH, ESP can be used in either transport or tunnel mode, affecting where it is placed within the packet.
  • Confidentiality: ESP provides confidentiality for the payload data, unlike AH which only provides integrity and authentication.

ESP is widely used in IPsec VPNs due to its ability to encrypt the payload, thus ensuring both the confidentiality and integrity of the data being transmitted.