IPsec - Internet Protocol Security

What is IPsec?

IPsec (Internet Protocol Security) is a suite of protocols that secures IP communications by encrypting and authenticating each IP packet. When used outside of VPN, IPsec operates directly between endpoints (e.g., host-to-host or network-to-network) without creating a virtual tunnel. Think of it as a secure envelope for each individual IP packet, without the need for a full VPN tunnel.

Why is IPsec important outside VPN?

  • Data Confidentiality: Encrypts individual IP packets to prevent unauthorized access.

  • Data Integrity: Ensures packets haven’t been altered during transit.

  • Authentication: Verifies the identity of communicating devices.

  • Granular Security: Secures specific traffic flows without tunneling all traffic.

  • Low Overhead: No need for virtual interfaces or VPN clients.

How IPsec works (in simple steps):

  • Negotiation: Devices agree on security parameters using IKE (Internet Key Exchange).

  • Authentication: Devices authenticate using pre-shared keys or certificates.

  • Encryption & Integrity: Each IP packet is encrypted and signed.

  • Transmission: Secure packets are sent directly between endpoints.

  • Decryption: Receiving device decrypts and verifies each packet.

Where is IPsec used?

  • Host-to-Host Security: Direct secure communication between two servers or endpoints.

  • Intranet Protection: Secures internal traffic within a corporate LAN or WAN.

  • IoT Security: Protects data exchanged between IoT devices and control systems.

  • Mobile IP: Maintains secure sessions as users move across networks.

  • VoIP & Video: Secures real-time communication without tunneling overhead.

Why OSI Layer: Network Layer (Layer 3)?

  • IPsec operates directly on IP packets, securing all traffic regardless of application.

  • It works below the transport layer (TCP/UDP) and above the data link layer, making it part of the Network Layer.

  • This allows IPsec to secure all communication without modifying higher-layer protocols.

  • In this section, you are going to learn

  • Terminology

  • Version Info

IPsec Version

RFC

Year

Core Idea / Contribution

IPsec v1

RFC 18251829

1995

Initial IPsec architecture, ESP, AH, and key management (ISAKMP).

IPsec v2

RFC 24012412

1998

Revised architecture, ESP/AH updates, IKE v1 introduced.

IPsec v3

RFC 4301

2005

Updated architecture with better support for NAT, mobility, and modularity

ESP Enhancements

RFC 4303

2005

Defines the Encapsulating Security Payload (ESP) protocol.

AH Enhancements

RFC 4302

2005

Defines the Authentication Header (AH) protocol.

Algorithm Support

RFC 4305

2005

Cryptographic algorithm requirements for ESP and AH.

RFC 4835

2007

Updates algorithm requirements.

RFC 8221

2017

Further updates to cryptographic algorithm usage.

NAT Traversal

RFC 3947, 3948

2005

NAT traversal for IPsec using UDP encapsulation.

Mobility Support

RFC 4555

2006

IPsec support for Mobile IPv6.

Roadmap

RFC 6071

2011

Comprehensive roadmap of IPsec and IKE-related RFCs.

Setup

Setup

Transport Mode with AH

S.No

Protocol Packets

Description

Size(Bytes)

1

Transport Mode with AH

Adds authentication to the original IP packet without encryption

~4484 bytes

IP Header

Original IP header of the packet.

20 (IPv4) / 40 (IPv6)

Next Header

Identifies the type of the next payload (e.g., TCP, UDP).

1

Payload Length

Length of the AH header in 32-bit words, minus 2.

1

Reserved

Reserved for future use; must be zero.

2

Security Parameters Index (SPI)

Identifies the security association.

4

Sequence Number

Increments with each packet to prevent replay attacks.

4

Authentication Data

Integrity Check Value (ICV) for the entire packet.

12-32

Payload

Original transport layer data (e.g., TCP/UDP + application data).

Variable

Transport Mode with ESP

S.No

Protocol Packets

Description

Size(Bytes)

2

Transport Mode with ESP

Encrypts and optionally authenticates the transport layer payload.

~60100+ bytes

IP Header

Original IP header of the packet.

20 (IPv4) / 40 (IPv6)

ESP Header

Contains SPI and Sequence Number.

8

Payload Data

Encrypted transport layer data (e.g., TCP/UDP + application data).

Variable

Padding

Aligns payload to encryption block size.

0-255

Pad Length

Length of the padding.

1

Next Header

Identifies the type of data in the payload (e.g., TCP, UDP).

1

Authentication Data (optional)

Integrity Check Value (ICV) for the encrypted portion.

1232

S.no

Use Case

Description

1

Host-to-Host Security

Secures communication between two individual devices (e.g., server-to-server).

2

Intranet Protection

Encrypts internal traffic within a corporate LAN or WAN.

3

Data Confidentiality

Ensures privacy of IP packets without requiring a VPN tunnel.

4

Data Integrity

Verifies that data hasnt been altered during transit.

5

Authentication

Confirms the identity of endpoints using keys or certificates.

6

Secure VoIP and Video

Protects real-time communication from eavesdropping and tampering.

7

Mobile IP Security

Maintains secure sessions as users move across networks.

8

IoT Device Protection

Secures communication between IoT devices and control systems.

9

Lightweight Security

Provides encryption/authentication without full VPN overhead.

10

Application-Agnostic Security

Secures all IP traffic regardless of the application layer.

S.no

Feature

Description

1

Encryption

Encrypts individual IP packets to ensure confidentiality of data in transit.

2

Authentication

Verifies the identity of peers using pre-shared keys or digital certificates.

3

Integrity Checking

Ensures data hasnt been altered during transmission using hashing algorithms.

4

Transport Mode Support

Encrypts only the payload of IP packets, suitable for host-to-host security.

5

Key Exchange (IKE/IKEv2)

Negotiates security associations and keys between endpoints.

6

Security Associations (SAs)

Defines parameters for secure communication between hosts.

7

Protocol Support (ESP & AH)

Uses ESP for encryption and AH for integrity/authentication.

8

NAT Traversal

Enables IPsec to work through NAT devices using UDP encapsulation.

9

Replay Protection

Prevents attackers from reusing captured packets using sequence numbers.

10

Flexible Algorithm Support

Supports various encryption and hashing algorithms (e.g., AES, SHA-2).

Encryption - Testcases

#

Test Case

Description

Expected Result

1

IPsec Initialization

Enable IPsec on device

IPsec stack initializes successfully

2

SPI Generation

Generate Security Parameter Index

SPI is unique and valid

3

SA Creation

Create Security Association

SA is created with correct parameters

4

SA Deletion

Delete Security Association

SA is removed from system

5

SA Lifetime Expiry

Wait for SA to expire

SA is deleted automatically

6

Manual Keying

Configure manual keys

Keys are accepted and used

7

Replay Protection

Send duplicate packets

Packets are dropped

8

Encryption Algorithm Test

Use AES-CBC

Packet payload is encrypted

9

Authentication Algorithm Test

Use HMAC-SHA1

Packet integrity is verified

10

ESP Packet Format

Send ESP packet

Packet follows RFC format

11

AH Packet Format

Send AH packet

Packet follows RFC format

12

Inbound Packet Decryption

Receive encrypted packet

Payload is decrypted correctly

13

Outbound Packet Encryption

Send packet

Payload is encrypted before transmission

14

Integrity Check Failure

Modify packet

Packets are dropped due to hash mismatch

15

Encryption Key Rotation

Change encryption key

New key is used for encryption

16

Authentication Key Rotation

Change auth key

New key is used for integrity

17

SA Negotiation

Negotiate SA manually

SA parameters match

18

Traffic Selector Match

Send matching traffic

Traffic is encrypted

19

Traffic Selector Mismatch

Send non-matching traffic

Traffic is not encrypted

20

Packet Size Limit

Send large packet

Packet is fragmented or dropped

21

ICMP Packet Encryption

Send ICMP packet

Packet is encrypted

22

TCP Packet Encryption

Send TCP packet

Packet is encrypted

23

UDP Packet Encryption

Send UDP packet

Packet is encrypted

24

Multicast Packet Handling

Send multicast packet

Packet is dropped or bypassed

25

IPv6 Support

Send IPv6 packet

Packet is encrypted

26

IPv4 Support

Send IPv4 packet

Packet is encrypted

27

SA Database Inspection

List active SAs

Correct SAs are listed

28

Replay Window Size

Set replay window

Window size is respected

29

Packet Reordering

Send out-of-order packets

Packets are accepted if within window

30

SA Bundling

Use multiple SAs

Packets use correct SA

31

ESP Tunnel Mode

Send packet in tunnel mode

Outer IP header is added

32

ESP Transport Mode

Send packet in transport mode

Original IP header is retained

33

AH Tunnel Mode

Send packet in AH tunnel mode

Outer IP header is added

34

AH Transport Mode

Send packet in AH transport mode

Original IP header is retained

35

SA with No Encryption

Use null encryption

Payload is not encrypted

36

SA with No Authentication

Use null authentication

Payload is not authenticated

37

SA with Both Null

Use null encryption and auth

Payload is sent as-is

38

SA with AES-GCM

Use AES-GCM algorithm

Payload is encrypted and authenticated

39

SA with ChaCha20-Poly1305

Use ChaCha20-Poly1305

Payload is encrypted and authenticated

40

SA with DES

Use DES algorithm

Payload is encrypted (legacy)

41

SA with Triple DES

Use 3DES algorithm

Payload is encrypted

42

SA with SHA-256

Use SHA-256 for auth

Payload is authenticated

43

SA with SHA-512

Use SHA-512 for auth

Payload is authenticated

44

SA with MD5

Use MD5 for auth

Payload is authenticated (legacy)

45

SA with AES-CTR

Use AES-CTR

Payload is encrypted

46

SA with AES-CFB

Use AES-CFB

Payload is encrypted

47

SA with AES-OFB

Use AES-OFB

Payload is encrypted

48

SA with AES-XTS

Use AES-XTS

Payload is encrypted

49

SA with AES-KW

Use AES Key Wrap

Payload is encrypted

50

SA with Custom Algorithm

Use custom crypto module

Payload is encrypted using custom logic

Authentication - Testcases

#

Test Case

Description

Expected Result

1

Authentication Initialization

Enable IPsec authentication module

Module initializes successfully

2

SA with Authentication Only

Create SA with only authentication

Packets are authenticated, not encrypted

3

AH Header Validation

Inspect AH header

Header conforms to RFC

4

HMAC-SHA1 Test

Use HMAC-SHA1 for authentication

Packet integrity is verified

5

HMAC-SHA256 Test

Use HMAC-SHA256 for authentication

Packet integrity is verified

6

HMAC-SHA512 Test

Use HMAC-SHA512 for authentication

Packet integrity is verified

7

MD5 Authentication Test

Use MD5 for authentication

Packet integrity is verified (legacy)

8

Null Authentication

Use null authentication

No integrity check performed

9

Replay Protection Enabled

Enable replay protection

Duplicate packets are dropped

10

Replay Protection Disabled

Disable replay protection

Duplicate packets are accepted

11

Authentication Key Rotation

Change authentication key

New key is used for integrity

12

Authentication Failure

Send tampered packet

Packet is dropped due to hash mismatch

13

Authentication Success

Send valid packet

Packet is accepted

14

AH Transport Mode

Use AH in transport mode

Original IP header is retained

15

AH Tunnel Mode

Use AH in tunnel mode

Outer IP header is added

16

Authentication with IPv4

Send IPv4 packet

Packet is authenticated

17

Authentication with IPv6

Send IPv6 packet

Packet is authenticated

18

Authentication with TCP

Send TCP packet

Packet is authenticated

19

Authentication with UDP

Send UDP packet

Packet is authenticated

20

Authentication with ICMP

Send ICMP packet

Packet is authenticated

21

Authentication with Fragmented Packet

Send fragmented packet

Packet fragments are authenticated

22

Authentication with Large Packet

Send large packet

Packet is authenticated

23

Authentication with Multicast

Send multicast packet

Packet may be dropped or bypassed

24

Authentication with Broadcast

Send broadcast packet

Packet may be dropped or bypassed

25

Authentication with ESP

Use ESP with authentication

Packet is encrypted and authenticated

26

Authentication with AH

Use AH protocol

Packet is authenticated

27

Authentication with Manual SA

Configure SA manually

Authentication works as configured

28

Authentication with Dynamic SA

Use dynamic SA negotiation

Authentication works as negotiated

29

Authentication with Multiple SAs

Use multiple SAs

Correct SA used for authentication

30

Authentication with Traffic Selector

Match traffic selector

Packet is authenticated

31

Authentication with Mismatched Selector

Mismatch selector

Packet is not authenticated

32

Authentication with Expired SA

Use expired SA

Packet is dropped

33

Authentication with Invalid SPI

Use invalid SPI

Packet is dropped

34

Authentication with Valid SPI

Use valid SPI

Packet is authenticated

35

Authentication with Custom Algorithm

Use custom auth algorithm

Packet is authenticated

36

Authentication with AES-GCM

Use AES-GCM for combined auth/encryption

Packet is authenticated and encrypted

37

Authentication with ChaCha20-Poly1305

Use ChaCha20-Poly1305

Packet is authenticated and encrypted

38

Authentication with SHA-1

Use SHA-1

Packet is authenticated

39

Authentication with SHA-256

Use SHA-256

Packet is authenticated

40

Authentication with SHA-512

Use SHA-512

Packet is authenticated

41

Authentication with MD5

Use MD5

Packet is authenticated (legacy)

42

Authentication with AES-XCBC

Use AES-XCBC-MAC

Packet is authenticated

43

Authentication with AES-CMAC

Use AES-CMAC

Packet is authenticated

44

Authentication with AES-KMAC

Use AES-KMAC

Packet is authenticated

45

Authentication with Key Mismatch

Use incorrect key

Packet is dropped

46

Authentication with Key Match

Use correct key

Packet is authenticated

47

Authentication with SA Inspection

Inspect SA database

Authentication parameters are correct

48

Authentication with Logging

Enable logging

Logs show authentication events

49

Authentication with Monitoring

Monitor traffic

Authenticated packets are visible

50

Authentication with Wireshark

Capture packets

AH/ESP headers visible in capture

Integrity Checking - Testcases

#

Test Case

Description

Expected Result

1

Integrity Module Initialization

Enable integrity checking module

Module initializes successfully

2

HMAC-SHA1 Integrity Check

Use HMAC-SHA1 for integrity

Packet integrity is verified

3

HMAC-SHA256 Integrity Check

Use HMAC-SHA256 for integrity

Packet integrity is verified

4

HMAC-SHA512 Integrity Check

Use HMAC-SHA512 for integrity

Packet integrity is verified

5

MD5 Integrity Check

Use MD5 for integrity

Packet integrity is verified (legacy)

6

Null Integrity Check

Use null integrity algorithm

No integrity check performed

7

Integrity Check Failure

Modify packet content

Packet is dropped due to hash mismatch

8

Integrity Check Success

Send unmodified packet

Packet is accepted

9

Integrity with AH Protocol

Use AH for integrity

Packet is authenticated

10

Integrity with ESP Protocol

Use ESP with integrity enabled

Packet is authenticated

11

Integrity with AES-GCM

Use AES-GCM for combined encryption/integrity

Packet is encrypted and authenticated

12

Integrity with ChaCha20-Poly1305

Use ChaCha20-Poly1305

Packet is encrypted and authenticated

13

Integrity with AES-CMAC

Use AES-CMAC for integrity

Packet is authenticated

14

Integrity with AES-XCBC

Use AES-XCBC-MAC

Packet is authenticated

15

Integrity with AES-KMAC

Use AES-KMAC

Packet is authenticated

16

Integrity with SHA-1

Use SHA-1 algorithm

Packet is authenticated

17

Integrity with SHA-256

Use SHA-256 algorithm

Packet is authenticated

18

Integrity with SHA-512

Use SHA-512 algorithm

Packet is authenticated

19

Integrity with Manual SA

Configure SA manually

Integrity check works as configured

20

Integrity with Dynamic SA

Use dynamic SA negotiation

Integrity check works as negotiated

21

Integrity with IPv4

Send IPv4 packet

Packet is authenticated

22

Integrity with IPv6

Send IPv6 packet

Packet is authenticated

23

Integrity with TCP

Send TCP packet

Packet is authenticated

24

Integrity with UDP

Send UDP packet

Packet is authenticated

25

Integrity with ICMP

Send ICMP packet

Packet is authenticated

26

Integrity with Fragmented Packet

Send fragmented packet

Packet fragments are authenticated

27

Integrity with Large Packet

Send large packet

Packet is authenticated

28

Integrity with Multicast Packet

Send multicast packet

Packet may be dropped or bypassed

29

Integrity with Broadcast Packet

Send broadcast packet

Packet may be dropped or bypassed

30

Integrity with Expired SA

Use expired SA

Packet is dropped

31

Integrity with Invalid SPI

Use invalid SPI

Packet is dropped

32

Integrity with Valid SPI

Use valid SPI

Packet is authenticated

33

Integrity with Key Rotation

Change integrity key

New key is used for integrity

34

Integrity with Key Mismatch

Use incorrect key

Packet is dropped

35

Integrity with Key Match

Use correct key

Packet is authenticated

36

Integrity with Replay Protection

Enable replay protection

Duplicate packets are dropped

37

Integrity with Logging

Enable logging

Logs show integrity check results

38

Integrity with Monitoring

Monitor traffic

Authenticated packets are visible

39

Integrity with Wireshark

Capture packets

AH/ESP headers visible in capture

40

Integrity with Packet Reordering

Send out-of-order packets

Packets accepted if within replay window

41

Integrity with Packet Loss

Drop some packets

Remaining packets are authenticated

42

Integrity with VLANs

Send packets across VLANs

Integrity check works if routing is correct

43

Integrity with NAT

Send packets through NAT

Integrity check may fail if headers are changed

44

Integrity with Tunneling

Use tunnel mode

Outer headers are authenticated

45

Integrity with Transport Mode

Use transport mode

Original headers are authenticated

46

Integrity with Custom Algorithm

Use custom integrity algorithm

Packet is authenticated

47

Integrity with Multiple SAs

Use multiple SAs

Correct SA used for integrity

48

Integrity with Traffic Selector

Match traffic selector

Packet is authenticated

49

Integrity with Mismatched Selector

Mismatch traffic selector

Packet is not authenticated

50

Integrity with SA Inspection

Inspect SA database

Integrity parameters are correct

Transport Mode Support - Testcases

#

Test Case

Description

Expected Result

1

Transport Mode Initialization

Enable IPsec transport mode

Transport mode is activated

2

SA Creation in Transport Mode

Create SA for transport mode

SA is created with transport mode parameters

3

SA Deletion in Transport Mode

Delete transport mode SA

SA is removed successfully

4

Transport Mode with IPv4

Send IPv4 packet

Packet is processed in transport mode

5

Transport Mode with IPv6

Send IPv6 packet

Packet is processed in transport mode

6

ESP in Transport Mode

Use ESP protocol in transport mode

Payload is encrypted, IP header retained

7

AH in Transport Mode

Use AH protocol in transport mode

Payload is authenticated, IP header retained

8

Combined ESP+AH Transport Mode

Use both ESP and AH

Packet is encrypted and authenticated

9

Transport Mode with TCP

Send TCP packet

Packet is secured in transport mode

10

Transport Mode with UDP

Send UDP packet

Packet is secured in transport mode

11

Transport Mode with ICMP

Send ICMP packet

Packet is secured in transport mode

12

Transport Mode with Fragmented Packet

Send fragmented packet

Fragments are handled correctly

13

Transport Mode with Large Packet

Send large packet

Packet is processed or fragmented

14

Transport Mode with Replay Protection

Enable replay protection

Duplicate packets are dropped

15

Transport Mode with HMAC-SHA1

Use HMAC-SHA1 for integrity

Packet integrity is verified

16

Transport Mode with HMAC-SHA256

Use HMAC-SHA256 for integrity

Packet integrity is verified

17

Transport Mode with AES-CBC

Use AES-CBC for encryption

Packet payload is encrypted

18

Transport Mode with AES-GCM

Use AES-GCM for encryption and integrity

Packet is encrypted and authenticated

19

Transport Mode with ChaCha20-Poly1305

Use ChaCha20-Poly1305

Packet is encrypted and authenticated

20

Transport Mode with Manual SA

Configure SA manually

Transport mode works as configured

21

Transport Mode with Dynamic SA

Use dynamic SA negotiation

Transport mode works as negotiated

22

Transport Mode with Expired SA

Use expired SA

Packet is dropped

23

Transport Mode with Invalid SPI

Use invalid SPI

Packet is dropped

24

Transport Mode with Valid SPI

Use valid SPI

Packet is processed

25

Transport Mode with NAT

Send packet through NAT

Packet may be dropped or modified

26

Transport Mode with VLANs

Send packet across VLANs

Packet is processed if routing is correct

27

Transport Mode with Broadcast

Send broadcast packet

Packet may be dropped or bypassed

28

Transport Mode with Multicast

Send multicast packet

Packet may be dropped or bypassed

29

Transport Mode with Replay Window

Set replay window size

Replay protection respects window size

30

Transport Mode with Packet Reordering

Send out-of-order packets

Packets accepted if within replay window

31

Transport Mode with Packet Loss

Drop some packets

Remaining packets are processed

32

Transport Mode with Logging

Enable logging

Logs show transport mode activity

33

Transport Mode with Monitoring

Monitor traffic

Transport mode packets are visible

34

Transport Mode with Wireshark

Capture packets

ESP/AH headers visible, IP header retained

35

Transport Mode with Key Rotation

Change encryption/authentication keys

New keys are used

36

Transport Mode with Key Mismatch

Use incorrect key

Packet is dropped

37

Transport Mode with Key Match

Use correct key

Packet is processed

38

Transport Mode with Custom Algorithm

Use custom crypto module

Packet is processed using custom logic

39

Transport Mode with Traffic Selector

Match traffic selector

Packet is processed

40

Transport Mode with Mismatched Selector

Mismatch selector

Packet is not processed

41

Transport Mode with Multiple SAs

Use multiple SAs

Correct SA is selected

42

Transport Mode with SA Inspection

Inspect SA database

Transport mode parameters are correct

43

Transport Mode with TTL Handling

Send packet with low TTL

Packet may be dropped

44

Transport Mode with QoS Tags

Send packet with QoS tags

Tags are preserved

45

Transport Mode with DSCP Tags

Send packet with DSCP tags

Tags are preserved

46

Transport Mode with TOS Bits

Send packet with TOS bits

Bits are preserved

47

Transport Mode with IPv6 Extension Headers

Send IPv6 packet with extensions

Headers are preserved

48

Transport Mode with IPv4 Options

Send IPv4 packet with options

Options are preserved

49

Transport Mode with Tunnel Mode Comparison

Compare with tunnel mode

Transport mode retains original IP header

50

Transport Mode with Performance Test

Measure latency and throughput

Performance metrics are within expected range

Key Exchange (IKE/IKEv2) - Testcases

#

Test Case

Description

Expected Result

1

IKE Initialization

Start IKE daemon

IKE service starts successfully

2

IKEv2 Initialization

Start IKEv2 daemon

IKEv2 service starts successfully

3

SA Negotiation Start

Initiate SA negotiation

SA negotiation begins

4

SA Negotiation Success

Complete SA negotiation

SA is established

5

SA Negotiation Failure

Use mismatched parameters

SA negotiation fails

6

Authentication with Pre-Shared Key

Use PSK for authentication

SA is authenticated

7

Authentication with Certificates

Use X.509 certificates

SA is authenticated

8

Authentication with EAP

Use EAP method

SA is authenticated

9

Diffie-Hellman Group 2

Use DH Group 2

Key exchange completes

10

Diffie-Hellman Group 14

Use DH Group 14

Key exchange completes

11

Diffie-Hellman Group 19

Use DH Group 19 (Elliptic Curve)

Key exchange completes

12

Diffie-Hellman Group Mismatch

Use mismatched DH groups

Key exchange fails

13

IKEv2 Fragmentation

Enable fragmentation

Large messages are fragmented

14

IKEv2 Reauthentication

Trigger reauthentication

SA is reauthenticated

15

IKEv2 Rekeying

Trigger rekeying

New keys are generated

16

IKEv2 MOBIKE Support

Change IP address

SA remains valid

17

IKEv2 NAT Detection

Detect NAT presence

NAT detected

18

IKEv2 Keepalive

Enable keepalive

Connection remains active

19

IKEv2 Dead Peer Detection

Enable DPD

Dead peer is detected

20

IKEv2 Cookie Mechanism

Trigger cookie exchange

Cookie is exchanged

21

IKEv2 Vendor ID Exchange

Send vendor ID

Vendor ID is received

22

IKEv2 Notify Payload

Send notify payload

Notification is received

23

IKEv2 Configuration Payload

Send configuration payload

Configuration is applied

24

IKEv2 Traffic Selector Negotiation

Negotiate traffic selectors

Selectors are agreed upon

25

IKEv2 Child SA Creation

Create child SA

Child SA is created

26

IKEv2 Child SA Deletion

Delete child SA

Child SA is removed

27

IKEv2 Multiple Child SAs

Create multiple child SAs

All SAs are created successfully

28

IKEv2 SA Lifetime Expiry

Wait for SA to expire

SA is deleted automatically

29

IKEv2 SA Manual Deletion

Manually delete SA

SA is removed

30

IKEv2 SA Inspection

Inspect SA database

SA parameters are correct

31

IKEv2 with IPv4

Use IPv4 for negotiation

SA is established

32

IKEv2 with IPv6

Use IPv6 for negotiation

SA is established

33

IKEv2 with Fragmented Packets

Send fragmented packets

Packets are reassembled

34

IKEv2 with Large Payloads

Send large payloads

Payloads are handled correctly

35

IKEv2 with Invalid Payload

Send malformed payload

Negotiation fails

36

IKEv2 with Logging Enabled

Enable logging

Logs show negotiation steps

37

IKEv2 with Monitoring

Monitor negotiation

Exchange is visible

38

IKEv2 with Wireshark

Capture packets

IKEv2 packets are visible

39

IKEv2 with NAT-T

Enable NAT traversal

Negotiation succeeds through NAT

40

IKEv2 with Firewall

Block IKE ports

Negotiation fails

41

IKEv2 with Port Change

Change IKE port

Negotiation adapts or fails

42

IKEv2 with Certificate Revocation

Use revoked certificate

Authentication fails

43

IKEv2 with Certificate Expiry

Use expired certificate

Authentication fails

44

IKEv2 with Valid Certificate

Use valid certificate

Authentication succeeds

45

IKEv2 with CRL Check

Enable CRL check

Revoked certs are rejected

46

IKEv2 with OCSP Check

Enable OCSP validation

Cert status is verified

47

IKEv2 with PSK Mismatch

Use incorrect PSK

Authentication fails

48

IKEv2 with PSK Match

Use correct PSK

Authentication succeeds

49

IKEv2 with Custom DH Group

Use custom DH group

Key exchange completes

50

IKEv2 Performance Test

Measure negotiation time

Time is within expected range

Security Associations (SAs) - Testcases

#

Test Case

Description

Expected Result

1

SA Initialization

Initiate SA between two hosts

SA established successfully

2

SA Negotiation

Exchange IKE messages

SA parameters agreed

3

SA Rekeying

Trigger rekeying manually

New SA replaces old one

4

SA Lifetime Expiry

Wait for SA to expire

SA deleted after timeout

5

SA Deletion

Manually delete SA

SA removed from table

6

SA with ESP

Use Encapsulating Security Payload

Encrypted traffic flows

7

SA with AH

Use Authentication Header

Authenticated traffic flows

8

SA with Both ESP & AH

Use both ESP and AH

Encrypted and authenticated traffic

9

SA with Tunnel Mode

Use tunnel mode

IPsec tunnel established

10

SA with Transport Mode

Use transport mode

Payload protected, headers visible

11

SA with IPv4

Use IPv4 traffic

SA handles IPv4 correctly

12

SA with IPv6

Use IPv6 traffic

SA handles IPv6 correctly

13

SA with ICMP

Send ICMP packets

ICMP protected by SA

14

SA with TCP

Send TCP packets

TCP protected by SA

15

SA with UDP

Send UDP packets

UDP protected by SA

16

SA Replay Protection

Send replayed packets

Packets dropped

17

SA Integrity Check

Tamper with packet

Packet rejected

18

SA Encryption Check

Verify encryption algorithm

Data encrypted as expected

19

SA Authentication Check

Verify authentication algorithm

Data authenticated

20

SA with Manual Keying

Configure keys manually

SA established without IKE

21

SA with Dynamic Keying

Use IKE for key exchange

Keys negotiated dynamically

22

SA with PFS

Enable Perfect Forward Secrecy

Keys regenerated securely

23

SA with NAT Traversal

Enable NAT-T

SA works behind NAT

24

SA with Fragmentation

Send large packets

Packets fragmented and reassembled

25

SA with MTU Constraints

Test with small MTU

SA handles fragmentation

26

SA with Multiple Peers

Establish SA with multiple peers

All SAs established

27

SA with Policy Mismatch

Use mismatched policies

SA negotiation fails

28

SA with Invalid SPI

Use invalid SPI

Packet dropped

29

SA with Replay Window

Test replay window size

Replay protection works

30

SA with Traffic Selector

Use specific traffic selectors

Only selected traffic protected

31

SA with Dead Peer Detection

Enable DPD

Dead peer detected

32

SA with Keepalive

Enable keepalive

SA maintained

33

SA with Certificate Auth

Use certificates for auth

SA established with certs

34

SA with PSK Auth

Use pre-shared key

SA established with PSK

35

SA with CRL

Use certificate revocation list

Revoked cert rejected

36

SA with IKEv1

Use IKEv1

SA established via IKEv1

37

SA with IKEv2

Use IKEv2

SA established via IKEv2

38

SA with Aggressive Mode

Use aggressive mode

SA established quickly

39

SA with Main Mode

Use main mode

SA established securely

40

SA with DDoS Simulation

Simulate DDoS on IKE port

SA negotiation fails

41

SA with Packet Sniffing

Monitor traffic

Encrypted packets visible

42

SA with IPsec Bypass

Use bypass policy

Traffic not encrypted

43

SA with IPsec Drop

Use drop policy

Traffic blocked

44

SA with IPsec Accept

Use accept policy

Traffic allowed

45

SA with SA Status Query

Query SA status

SA details returned

46

SA with SA Statistics

Check SA stats

Packet counts visible

47

SA with SA Refresh

Refresh SA manually

SA updated

48

SA with SA Migration

Migrate SA to new peer

SA re-established

49

SA with SA Backup

Backup SA configuration

Backup created

50

SA with SA Restore

Restore SA from backup

SA restored successfully

Protocol Support (ESP & AH) - Testcases

#

Test Case

Description

Expected Result

1

ESP Basic Functionality

Send encrypted traffic using ESP

Traffic encrypted and received correctly

2

AH Basic Functionality

Send authenticated traffic using AH

Traffic authenticated and received correctly

3

ESP with Tunnel Mode

Use ESP in tunnel mode

Entire packet encrypted

4

ESP with Transport Mode

Use ESP in transport mode

Payload encrypted, headers visible

5

AH with Tunnel Mode

Use AH in tunnel mode

Entire packet authenticated

6

AH with Transport Mode

Use AH in transport mode

Headers and payload authenticated

7

ESP with AES Encryption

Use AES algorithm with ESP

Data encrypted with AES

8

ESP with 3DES Encryption

Use 3DES algorithm with ESP

Data encrypted with 3DES

9

AH with SHA-1

Use SHA-1 for AH authentication

Data authenticated with SHA-1

10

AH with SHA-256

Use SHA-256 for AH authentication

Data authenticated with SHA-256

11

ESP with Integrity Check

Enable integrity check in ESP

Packet integrity verified

12

ESP without Integrity Check

Disable integrity check

Packet encrypted but not authenticated

13

AH with Replay Protection

Enable anti-replay in AH

Replayed packets dropped

14

ESP with Replay Protection

Enable anti-replay in ESP

Replayed packets dropped

15

ESP with NAT Traversal

Use ESP behind NAT

Traffic passes using UDP encapsulation

16

AH with NAT

Use AH behind NAT

AH fails due to header modification

17

ESP with IPv4

Send ESP traffic over IPv4

Traffic encrypted and received

18

ESP with IPv6

Send ESP traffic over IPv6

Traffic encrypted and received

19

AH with IPv4

Send AH traffic over IPv4

Traffic authenticated and received

20

AH with IPv6

Send AH traffic over IPv6

Traffic authenticated and received

21

ESP with Fragmentation

Send large ESP packets

Packets fragmented and reassembled

22

AH with Fragmentation

Send large AH packets

Packets authenticated and reassembled

23

ESP with ICMP

Send ICMP traffic with ESP

Encrypted ICMP traffic received

24

AH with ICMP

Send ICMP traffic with AH

Authenticated ICMP traffic received

25

ESP with TCP

Send TCP traffic with ESP

Encrypted TCP traffic received

26

AH with TCP

Send TCP traffic with AH

Authenticated TCP traffic received

27

ESP with UDP

Send UDP traffic with ESP

Encrypted UDP traffic received

28

AH with UDP

Send UDP traffic with AH

Authenticated UDP traffic received

29

ESP with Invalid Key

Use incorrect encryption key

Decryption fails

30

AH with Invalid Key

Use incorrect authentication key

Authentication fails

31

ESP with Manual Keying

Configure ESP manually

Traffic encrypted manually

32

AH with Manual Keying

Configure AH manually

Traffic authenticated manually

33

ESP with Dynamic Keying

Use IKE to negotiate ESP keys

Keys exchanged and traffic encrypted

34

AH with Dynamic Keying

Use IKE to negotiate AH keys

Keys exchanged and traffic authenticated

35

ESP with PFS

Enable Perfect Forward Secrecy

Keys regenerated securely

36

AH with PFS

Enable PFS for AH

Keys regenerated securely

37

ESP with Packet Tampering

Modify encrypted packet

Packet rejected

38

AH with Packet Tampering

Modify authenticated packet

Packet rejected

39

ESP with SPI Validation

Use invalid SPI

Packet dropped

40

AH with SPI Validation

Use invalid SPI

Packet dropped

41

ESP with SA Expiry

Let ESP SA expire

Traffic stops

42

AH with SA Expiry

Let AH SA expire

Traffic stops

43

ESP with Rekeying

Rekey ESP SA

New keys used for encryption

44

AH with Rekeying

Rekey AH SA

New keys used for authentication

45

ESP with Multiple Peers

Use ESP with multiple endpoints

All peers communicate securely

46

AH with Multiple Peers

Use AH with multiple endpoints

All peers authenticate traffic

47

ESP with Policy Mismatch

Use mismatched ESP policies

SA negotiation fails

48

AH with Policy Mismatch

Use mismatched AH policies

SA negotiation fails

49

ESP with Logging Enabled

Enable logging for ESP

Logs show encrypted traffic

50

AH with Logging Enabled

Enable logging for AH

Logs show authenticated traffic

NAT Traversal - Testcases

#

Test Case

Description

Expected Result

1

NAT-T Basic Functionality

Enable NAT-T on both peers

IPsec tunnel established behind NAT

2

NAT-T with One Peer Behind NAT

Only one peer behind NAT

Tunnel established using UDP encapsulation

3

NAT-T with Both Peers Behind NAT

Both peers behind different NATs

Tunnel established using UDP encapsulation

4

NAT-T Detection

Detect NAT presence during IKE negotiation

NAT detected and NAT-T enabled

5

NAT-T with ESP in UDP

Use ESP encapsulated in UDP

ESP packets encapsulated in UDP

6

NAT-T with Port Mapping

NAT device remaps ports

Tunnel still established

7

NAT-T with IP Mapping

NAT device remaps IP addresses

Tunnel still established

8

NAT-T Keepalive

Send NAT keepalive packets

NAT binding maintained

9

NAT-T with IKEv1

Use IKEv1 with NAT-T

Tunnel established

10

NAT-T with IKEv2

Use IKEv2 with NAT-T

Tunnel established

11

NAT-T with UDP Port 4500

Use UDP port 4500 for NAT-T

Traffic flows through port 4500

12

NAT-T with UDP Port 500

Start negotiation on port 500

Switches to 4500 after NAT detection

13

NAT-T with Static NAT

Use static NAT configuration

Tunnel established

14

NAT-T with Dynamic NAT

Use dynamic NAT configuration

Tunnel established

15

NAT-T with PAT

Use Port Address Translation

Tunnel established

16

NAT-T with Symmetric NAT

Use symmetric NAT

Tunnel may fail or require workaround

17

NAT-T with Cone NAT

Use full cone NAT

Tunnel established

18

NAT-T with Restricted NAT

Use restricted cone NAT

Tunnel established

19

NAT-T with Multiple Clients

Multiple clients behind same NAT

Tunnels established with unique ports

20

NAT-T with IPsec Policy Match

Ensure policies match with NAT-T enabled

Tunnel established

21

NAT-T with Fragmented Packets

Send large packets requiring fragmentation

Packets reassembled successfully

22

NAT-T with MTU Constraints

Use small MTU

Tunnel handles fragmentation

23

NAT-T with DPD

Enable Dead Peer Detection

Peer failure detected behind NAT

24

NAT-T with Keepalive Timeout

Simulate NAT timeout

Tunnel re-established

25

NAT-T with Re-keying

Re-key SA while behind NAT

New keys negotiated

26

NAT-T with IPsec Transport Mode

Use transport mode with NAT-T

Tunnel established

27

NAT-T with IPsec Tunnel Mode

Use tunnel mode with NAT-T

Tunnel established

28

NAT-T with ESP Integrity Check

Verify ESP integrity behind NAT

Packets authenticated

29

NAT-T with AH

Use AH protocol behind NAT

Fails due to header modification

30

NAT-T with Firewall

NAT device with firewall enabled

Tunnel established if ports allowed

31

NAT-T with UDP Flooding

Simulate UDP flood on port 4500

Tunnel may drop or throttle

32

NAT-T with IPsec Logging

Enable logs during NAT-T

Logs show NAT detection and UDP encapsulation

33

NAT-T with IPsec Debugging

Enable debug mode

Detailed NAT-T negotiation visible

34

NAT-T with Packet Sniffing

Capture traffic with Wireshark

ESP-in-UDP packets visible

35

NAT-T with IPsec Bypass Policy

Use bypass policy

Traffic not encrypted

36

NAT-T with IPsec Drop Policy

Use drop policy

Traffic blocked

37

NAT-T with IPsec Accept Policy

Use accept policy

Traffic allowed

38

NAT-T with Mobile Clients

Use mobile device behind NAT

Tunnel established

39

NAT-T with Home Router

Use IPsec client behind home NAT router

Tunnel established

40

NAT-T with Enterprise Firewall

Use IPsec behind enterprise NAT/firewall

Tunnel established if ports open

41

NAT-T with IPsec Failover

Simulate failover of NAT device

Tunnel re-established

42

NAT-T with IPsec Migration

Change public IP of NAT device

Tunnel re-established

43

NAT-T with Multiple Interfaces

Use multiple interfaces on NAT device

Tunnel established on correct interface

44

NAT-T with IPsec SA Expiry

Let SA expire behind NAT

Tunnel re-established

45

NAT-T with IPsec SA Deletion

Delete SA manually

Tunnel torn down

46

NAT-T with IPsec SA Refresh

Refresh SA manually

Tunnel updated

47

NAT-T with IPsec Certificate Auth

Use certificates behind NAT

Tunnel established

48

NAT-T with IPsec PSK Auth

Use pre-shared key behind NAT

Tunnel established

49

NAT-T with IPsec Policy Mismatch

Use mismatched policies

Tunnel fails

50

NAT-T with IPsec Interop Test

Test NAT-T with different vendor devices

Tunnel established if standards followed

Replay Protection - Testcases

#

Test Case

Description

Expected Result

1

Replay Protection Enabled

Enable anti-replay on SA

Replayed packets are dropped

2

Replay Protection Disabled

Disable anti-replay on SA

Replayed packets are accepted

3

Replay Window Size Default

Use default replay window size

Packets within window accepted

4

Replay Window Size Custom

Set custom replay window size

Behavior matches configured window

5

Replay with Identical Packet

Resend same packet

Packet dropped

6

Replay with Modified Payload

Resend packet with changed payload

Packet dropped

7

Replay with Delayed Packet

Delay and resend packet

Packet dropped if outside window

8

Replay with Out-of-Order Packets

Send packets out of order

Accepted if within window

9

Replay with Duplicate Sequence

Send packet with duplicate sequence number

Packet dropped

10

Replay with Skipped Sequence

Skip sequence number and send

Packet accepted, gap noted

11

Replay with Wrapped Sequence

Force sequence number wraparound

New window starts, old packets dropped

12

Replay with ESP

Test replay protection with ESP

Replayed ESP packets dropped

13

Replay with AH

Test replay protection with AH

Replayed AH packets dropped

14

Replay with IKEv1

Use IKEv1 with replay protection

Replayed packets dropped

15

Replay with IKEv2

Use IKEv2 with replay protection

Replayed packets dropped

16

Replay with IPv4

Send IPv4 packets

Replay protection enforced

17

Replay with IPv6

Send IPv6 packets

Replay protection enforced

18

Replay with Fragmented Packets

Replay fragmented packets

Dropped if already received

19

Replay with NAT-T

Replay ESP-in-UDP packets

Dropped if replayed

20

Replay with Manual Keying

Use manual keys

Replay protection may not be enforced

21

Replay with Dynamic Keying

Use IKE-negotiated keys

Replay protection enforced

22

Replay with PFS

Enable Perfect Forward Secrecy

Replay protection still enforced

23

Replay with Re-keying

Replay old packet after rekey

Dropped due to SPI mismatch

24

Replay with Expired SA

Replay packet after SA expiry

Dropped due to invalid SA

25

Replay with Invalid SPI

Use invalid SPI in replayed packet

Packet dropped

26

Replay with Logging Enabled

Enable IPsec logs

Replay attempts logged

27

Replay with Debugging Enabled

Enable debug mode

Detailed logs of replay detection

28

Replay with Packet Sniffing

Capture and replay packet

Packet dropped

29

Replay with High Traffic

Replay during high traffic

Replay protection still enforced

30

Replay with Low Traffic

Replay during idle time

Packet dropped

31

Replay with Multiple Peers

Replay packet from one peer to another

Dropped due to SPI mismatch

32

Replay with Policy Mismatch

Use mismatched policies

SA not established

33

Replay with Firewall

Replay packet through firewall

Dropped if IPsec inspection enabled

34

Replay with NAT Device

Replay packet behind NAT

Dropped if replay protection enabled

35

Replay with Mobile Client

Replay packet from mobile device

Packet dropped

36

Replay with Static IP

Replay from static IP

Packet dropped

37

Replay with Dynamic IP

Replay from dynamic IP

Packet dropped

38

Replay with Time Sync Drift

Replay with clock drift

Packet dropped

39

Replay with Packet Injection

Inject replayed packet into stream

Packet dropped

40

Replay with DoS Simulation

Flood with replayed packets

Dropped and logged

41

Replay with Sequence Reset

Force sequence reset

Old packets dropped

42

Replay with Sequence Overflow

Force sequence number overflow

New window starts

43

Replay with SA Migration

Replay packet after SA migration

Dropped due to SPI mismatch

44

Replay with SA Backup/Restore

Replay packet after SA restore

Dropped if SPI or keys changed

45

Replay with IPsec Accept Policy

Use accept policy

Replay protection still enforced

46

Replay with IPsec Drop Policy

Use drop policy

All packets dropped

47

Replay with IPsec Bypass Policy

Use bypass policy

Replay protection not applied

48

Replay with Vendor Interop

Replay packet between different vendor devices

Packet dropped if compliant

49

Replay with Sequence Gap

Introduce large sequence gap

Older packets dropped

50

Replay with Sequence Replay Burst

Replay burst of old packets

All dropped

Flexible Algorithm Support - Testcases

#

Test Case

Description

Expected Result

1

AES-128 Encryption

Use AES-128 for ESP encryption

Traffic encrypted with AES-128

2

AES-256 Encryption

Use AES-256 for ESP encryption

Traffic encrypted with AES-256

3

3DES Encryption

Use 3DES for ESP encryption

Traffic encrypted with 3DES

4

ChaCha20 Encryption

Use ChaCha20 for ESP encryption

Traffic encrypted with ChaCha20

5

Blowfish Encryption

Use Blowfish for ESP encryption

Traffic encrypted with Blowfish

6

DES Encryption

Use DES for ESP encryption

Traffic encrypted with DES

7

SHA-1 Authentication

Use SHA-1 for AH authentication

Traffic authenticated with SHA-1

8

SHA-256 Authentication

Use SHA-256 for AH authentication

Traffic authenticated with SHA-256

9

SHA-384 Authentication

Use SHA-384 for AH authentication

Traffic authenticated with SHA-384

10

SHA-512 Authentication

Use SHA-512 for AH authentication

Traffic authenticated with SHA-512

11

MD5 Authentication

Use MD5 for AH authentication

Traffic authenticated with MD5

12

HMAC-SHA1

Use HMAC with SHA-1

HMAC applied correctly

13

HMAC-SHA256

Use HMAC with SHA-256

HMAC applied correctly

14

HMAC-SHA512

Use HMAC with SHA-512

HMAC applied correctly

15

RSA Key Exchange

Use RSA for IKE key exchange

Keys exchanged securely

16

DH Group 2

Use Diffie-Hellman Group 2

Keys exchanged securely

17

DH Group 5

Use Diffie-Hellman Group 5

Keys exchanged securely

18

DH Group 14

Use Diffie-Hellman Group 14

Keys exchanged securely

19

DH Group 19

Use Diffie-Hellman Group 19 (Elliptic Curve)

Keys exchanged securely

20

DH Group 20

Use Diffie-Hellman Group 20 (Elliptic Curve)

Keys exchanged securely

21

ECDSA Authentication

Use ECDSA for authentication

SA established with ECDSA

22

RSA Authentication

Use RSA certificates

SA established with RSA

23

PSK Authentication

Use pre-shared key

SA established with PSK

24

Certificate Revocation Check

Use CRL to validate certificates

Revoked cert rejected

25

Algorithm Negotiation

Negotiate algorithms during IKE

Compatible algorithms selected

26

Unsupported Algorithm

Use unsupported algorithm

SA negotiation fails

27

Algorithm Mismatch

Use mismatched algorithms

SA negotiation fails

28

Multiple Encryption Algorithms

Configure multiple encryption options

Best match selected

29

Multiple Authentication Algorithms

Configure multiple auth options

Best match selected

30

Algorithm Preference Order

Set preference order

Preferred algorithm selected

31

Algorithm Performance Test

Measure performance of each algorithm

Performance metrics recorded

32

Algorithm Interoperability

Test with different vendor implementations

SA established if compatible

33

Algorithm with IPv4

Use flexible algorithms over IPv4

Traffic protected

34

Algorithm with IPv6

Use flexible algorithms over IPv6

Traffic protected

35

Algorithm with ESP

Use flexible encryption algorithms with ESP

Traffic encrypted

36

Algorithm with AH

Use flexible auth algorithms with AH

Traffic authenticated

37

Algorithm with NAT-T

Use flexible algorithms behind NAT

Tunnel established

38

Algorithm with Fragmentation

Use flexible algorithms with large packets

Packets reassembled

39

Algorithm with Re-keying

Re-key SA with different algorithm

New algorithm applied

40

Algorithm with SA Expiry

Let SA expire and re-establish with new algorithm

New SA uses new algorithm

41

Algorithm with Manual Keying

Use manual keys with flexible algorithms

Traffic protected

42

Algorithm with Dynamic Keying

Use IKE with flexible algorithms

Traffic protected

43

Algorithm with Logging Enabled

Enable logs

Logs show algorithm used

44

Algorithm with Debugging Enabled

Enable debug mode

Detailed negotiation logs

45

Algorithm with Mobile Clients

Use flexible algorithms on mobile device

SA established

46

Algorithm with Embedded Devices

Use flexible algorithms on embedded systems

SA established

47

Algorithm with High Latency Network

Test algorithm performance under latency

SA maintained

48

Algorithm with Packet Loss

Test algorithm resilience to packet loss

SA maintained

49

Algorithm with DoS Simulation

Simulate DoS attack

SA protected

50

Algorithm with Vendor Compliance

Validate against vendor specs

Algorithms accepted

  • Reference links