Network Reliability Engineers routinely face the challenge of extracting structured data from legacy CLI interfaces. While modern network operating systems support NETCONF, RESTCONF, and gRPC telemetry, legacy environments and troubleshooting workflows still rely heavily on traditional show commands.
Extracting structured JSON data from raw, unstructured CLI output for complex routing states is notoriously difficult. Relying on custom regex scripts leads to fragile automation that breaks during minor firmware upgrades. Solving this requires robust parsing engines like the Cisco Genie parser and Python TextFSM.
The Root Cause: Why Naive Regex Fails in IOS-XR Automation
Command-line interfaces are optimized for human readability, not machine consumption. When you execute a command like show bgp ipv4 unicast summary on an IOS-XR device, the router formats the output using arbitrary whitespace padding, pagination, and dynamic column widths.
Naive regex patterns fail because CLI structures are highly variable. A BGP peer with an exceptionally long IPv6 address or a 4-byte Autonomous System Number (ASN) will shift the surrounding columns, breaking strict positional regex matches. Furthermore, network state outputs often contain nested relationships, such as multiple Address Families (AFI/SAFI) operating under different Virtual Routing and Forwarding (VRF) instances.
To reliably handle BGP parsing in Python, automation pipelines require state machine-based parsers. These tools maintain contextual awareness as they read through the text line-by-line, accurately associating a specific BGP neighbor state with the correct parent VRF.
The Fix: Offline Parsing with Cisco Genie
The Cisco Genie parser (part of the pyATS framework) provides hundreds of pre-built, community-tested parsers for IOS-XR. While Genie is typically used with live SSH connections, testing automation logic safely requires parsing raw text offline.
The following Python solution demonstrates how to inject raw CLI text into a mocked pyATS device and extract a structured dictionary using Genie.
Prerequisites
Install the required pyATS and Genie libraries.
pip install pyats genie
Python Implementation
This script utilizes a Mock Device pattern. It allows you to leverage Genie's robust parsing logic on text files or strings gathered via asynchronous SSH libraries like Netmiko or Scrapli, without requiring a live pyATS connection object.
import json
from genie.libs.parser.iosxr.show_bgp import ShowBgpIpv4UnicastSummary
# 1. Define the raw, unstructured IOS-XR CLI output
raw_iosxr_output = """
BGP router identifier 10.0.0.1, local AS number 65000
BGP generic routing table state
BGP table state: Active
Table ID: 0xe0000000 RD version: 104
BGP main routing table version 104
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/Graceful restart is enabled
BGP is operating in STANDALONE mode.
Process RcvTblVer bRIB/RIB LabelVer ImportVer SendTblVer StandbyVer
Speaker 104 104 104 104 104 0
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
192.168.1.2 0 65001 145234 145678 104 0 0 10w2d 45
192.168.1.3 0 65002 0 0 0 0 0 00:00:00 Idle
"""
# 2. Create a Mock Device class to satisfy Genie's requirements
class MockDevice:
def __init__(self, output_string: str):
self.name = 'xr-router-01'
self.os = 'iosxr'
self.output_string = output_string
self.custom = {}
def execute(self, *args, **kwargs) -> str:
"""
Genie parsers call the execute() method on the device object.
We intercept this call and return our raw text string.
"""
return self.output_string
def parse_bgp_summary(raw_text: str) -> dict:
"""Parses raw IOS-XR BGP summary text into a structured Python dictionary."""
# Instantiate the Mock Device
device = MockDevice(output_string=raw_text)
# Initialize the specific Genie parser for IOS-XR BGP IPv4 Unicast Summary
parser = ShowBgpIpv4UnicastSummary(device=device)
# Execute the parse method
try:
parsed_data = parser.parse()
return parsed_data
except Exception as e:
print(f"Parsing failed: {e}")
return {}
# 3. Execute and output structured JSON
if __name__ == "__main__":
structured_data = parse_bgp_summary(raw_iosxr_output)
print(json.dumps(structured_data, indent=2))
Deep Dive: How the Parser Engine Works
When you invoke the parse() method on a Genie parser class, it does not execute a single monolithic regex. Instead, it relies heavily on Python TextFSM templates and structured regular expressions mapped to internal state machines.
Contextual State Transitions
TextFSM reads the raw_iosxr_output sequentially. When the engine encounters the line BGP router identifier 10.0.0.1, local AS number 65000, it captures the global router ID and ASN, transitioning the state machine to expect VRF or Address Family headers.
When it reaches the tabular neighbor data (Neighbor Spk AS MsgRcvd...), it transitions into a tabular parsing state. This ensures that the neighbor 192.168.1.2 is programmatically mapped to the default VRF and IPv4 Unicast address family within the resulting JSON hierarchy, rather than being parsed as an isolated string.
The Structured JSON Output
The returned dictionary strictly defines nested relationships. Automation scripts can now securely traverse the data structure using standard dictionary lookups:
{
"vrf": {
"default": {
"router_id": "10.0.0.1",
"local_as": 65000,
"address_family": {
"ipv4 unicast": {
"state": "Active",
"table_version": 104,
"nsr_state": "enabled",
"standby_mode": "STANDALONE",
"neighbor": {
"192.168.1.2": {
"spk": 0,
"as": 65001,
"msg_rcvd": 145234,
"msg_sent": 145678,
"tbl_ver": 104,
"in_q": 0,
"out_q": 0,
"up_down": "10w2d",
"state_pfxrcvd": "45"
},
"192.168.1.3": {
"spk": 0,
"as": 65002,
"msg_rcvd": 0,
"msg_sent": 0,
"tbl_ver": 0,
"in_q": 0,
"out_q": 0,
"up_down": "00:00:00",
"state_pfxrcvd": "Idle"
}
}
}
}
}
}
}
Common Pitfalls and Edge Cases
Firmware Version Drift
Cisco occasionally modifies CLI outputs in major IOS-XR releases (e.g., migrating from eXR 6.x to 7.x). If a parser fails, it will raise a SchemaEmptyParserError. Genie addresses this by supporting multiple regex variations per line within its source code. If you encounter a failure, verify you are running the latest pyATS release (pip install --upgrade pyats genie).
Full BGP Table Memory Exhaustion
Do not use CLI parsers for commands that generate millions of lines, such as show bgp ipv4 unicast. Loading a 900,000-route BGP table into memory as a raw string and processing it through a Python TextFSM engine will cause high CPU utilization and Out-Of-Memory (OOM) crashes. For full table dumps, rely on BMP (BGP Monitoring Protocol) or streaming telemetry. Reserve CLI parsing for summary, interface, and neighbor status commands.
Handling Unsupported Commands
If you encounter a proprietary command or an obscure IOS-XR feature lacking a native Genie parser, you can write a custom TextFSM template. Genie allows you to import external TextFSM files dynamically using the ntc-templates repository pattern, ensuring your IOS-XR automation remains unblocked while waiting for upstream library updates.
Conclusion
Transitioning from fragile regex scripts to state-aware parsing engines is mandatory for scalable network automation. By leveraging the Cisco Genie parser and Python TextFSM with a Mock Device pattern, Network Reliability Engineers can decouple their parsing logic from physical device connections, enabling rigorous CI/CD testing and reliable data extraction from complex IOS-XR environments.