Understanding 5G NAS Authentication and Security Procedures: Step-by-Step Message Flow Explained

Grasping 5G NAS Authentication and Security: A Breakdown of Message Flow

In 5G networks, it’s crucial to maintain a secure and authenticated connection between User Equipment (UE) and the 5G Core Network (5GC). This process, known as the NAS (Non-Access Stratum) Authentication and Security procedures, is all about protecting user data, confirming identities, and setting up encryption standards.

The diagram above showcases the message exchange among three essential components:

User Equipment (UE)

5G Node B (gNB)

5G Core (5GC) — which includes the Access and Mobility Management Function (AMF) and the User Plane Function (UPF)

This whole process guarantees that only verified devices can access the 5G network, and all communications are encrypted and safeguarded.

Overview of NAS Authentication and Security

The NAS (Non-Access Stratum) layer is responsible for control-plane signaling between the UE and the 5G Core (specifically the AMF). At this stage, authentication and security ensure:

The user’s identity gets verified (authentication).

The integrity and confidentiality of signaling messages are preserved (security).

Cryptographic keys are established for future data encryption.

In 5G, NAS authentication utilizes the 5G-AKA (Authentication and Key Agreement) protocol—drawing from 3GPP specifications. It’s designed to be compatible with LTE but comes with upgraded security.

Key Network Entities in the Procedure

Entity Description UE (User Equipment)The user's device (like a smartphone or IoT device) that starts the connection. gNB (Next Generation Node B)The 5G base station that manages radio communication and forwards NAS messages between the UE and AMF.AMF (Access and Mobility Management Function)The core entity overseeing registration, connection, authentication, and security management. UPF (User Plane Function)Handles data forwarding but isn’t directly involved in authentication.

Step-by-Step Message Flow

The diagram illustrates four main message exchanges that happen during the NAS Authentication and Security process:

Step 1: NAS Authentication Request (Message 16)

Direction: AMF → UE (via gNB)

After the UE starts a registration request, the AMF kicks off the authentication process. The AMF sends a NAS Authentication Request that includes:

ngKSI (NAS Key Set Identifier): This identifies the key set for NAS security.

RAND: A random challenge value.

AUTN: An authentication token generated by the network.

ABBA (Additional parameters): An anti-bidding down parameter that ensures integrity across various security levels.

👉 Purpose: This message challenges the UE to prove its identity through cryptographic operations.

Step 2: NAS Authentication Response (Message 17)

Direction: UE → AMF (via gNB)

Once the UE receives the challenge, it:

Uses its USIM (Universal Subscriber Identity Module) to compute the RES (Response) based on the received RAND and its stored authentication key (K).

Sends back an Authentication Response parameter to the AMF.

If this response matches the network’s expected value (XRES"), authentication is successful.

👉 Purpose: Confirms that the UE possesses valid credentials and is allowed access to the network.

Step 3: NAS Security Mode Command (Message 18)

Direction: AMF → UE (via gNB)

Following successful authentication, the AMF sends the NAS Security Mode Command, which contains:

Chosen NAS security algorithms (for encryption and integrity).

UE security capabilities (previously sent by the UE).

IMEISV request (optional, for device identification).

ngKSI (key set identifier).

Additional 5G security info.

This step makes sure that both UE and AMF agree on the encryption and integrity algorithms that will protect future NAS signaling.

👉 Purpose: Sets up the agreed security algorithms and activates the NAS security context.

Step 4: NAS Security Mode Complete (Message 19)

Direction: UE → AMF (via gNB)

The UE replies with NAS Security Mode Complete, which includes:

A NAS message container that confirms the activation of the security algorithms.

The IMEISV (International Mobile Equipment Identity - Software Version) if it was requested.

👉 Purpose: Confirms that the UE has implemented the agreed security settings and is ready for secure NAS signaling.

How Authentication Works: Cryptographic Flow

The key authentication mechanism employs the 5G-AKA or EAP-AKA’ protocol:

AMF requests authentication vectors from the AUSF (Authentication Server Function) and the UDM (Unified Data Management).

The AUSF verifies the UE's responses using keys derived from the subscriber’s permanent key.

Once verified, both UE and AMF derive shared session keys (K ext{}NASenc and K ext{}NASint) for encryption and integrity protection.

Key Hierarchy Overview:

K → Permanent key stored in the USIM

K ext{_}AUSF → Derived for AUSF validation

K ext{_}SEAF → Session key shared between UE and AMF

K ext{}NA Senc, K ext{}NA Sint → Derived for NAS encryption/integrity

This hierarchical structure keeps permanent keys from direct exposure, helping to reduce the risk of key compromise.

Significance of Each Message in Network Security

Message Function Security Role NAS Authentication Request Challenges UE Kicks off mutual authentication NAS Authentication Response Validates UE credentials Confirms UE authenticity NAS Security Mode Command Chooses algorithms Sets up encryption and integrity context NAS Security Mode Complete Activates protection Confirms secure NAS session setup

Every message here plays a role in ensuring mutual authentication, encryption setup, and secure signaling between the UE and the network.

Key Advantages of NAS Security in 5G

🔐 Mutual Authentication: Both the network and UE check each other’s identities.

🧠 Improved Key Management: 5G introduces K ext{}AMF and K ext{}gNB separation for control and user planes.

⚙️ Algorithm Flexibility: Supports a variety of ciphering and integrity algorithms.

🚀 Decreased Attack Surface: Prevents replay and man-in-the-middle attacks with unique session keys.

🌐 Backward Compatibility: 5G-AKA is interoperable with LTE’s authentication methods, easing transitions.

Common Problems and Troubleshooting Tips

Telecom pros frequently run into the following issues during NAS authentication:

Authentication Failure: Usually happens due to mismatched keys or out-of-sync sequence numbers (SQN).

Security Mode Reject: Often due to unsupported algorithms or integrity check failures.

UE Non-Response: Might indicate issues in the signaling path between gNB and AMF.

Best Practice: Always check UE capability parameters and make sure AMF-gNB security synchronization is good via the N2 interface.

Summary of Message Sequence

Here’s a simplified look at the NAS Authentication and Security exchange:

StepMessageDirectionPurpose16NAS Authentication Request AMF → UE Challenge UE17NAS Authentication Response UE → AMF Verify credentials18NAS Security Mode Command AMF → UE Establish security algorithms19NAS Security Mode Complete UE → AMF Confirm secure context

Conclusion

The NAS Authentication and Security procedure lays the groundwork for the 5G security architecture. It ensures that only authorized users and devices can access the network while keeping all signaling data encrypted and protected against tampering.

For telecom professionals and enthusiasts alike, getting a handle on these message exchanges—from Authentication Request to Security Mode Complete—is vital for troubleshooting, optimizing, and ensuring the integrity of end-to-end networks.

As 5G networks keep progressing, mastering these core processes will lead to better design, monitoring, and safeguarding of next-gen communication systems.