In today's world, network devices have become the backbone of every organization. Ensuring their smooth functioning is crucial to maintain business operations. However, managing network configurations can be tedious, especially when dealing with multiple devices.
If you're like me, a bit older and wiser, you might remember tools like Rancid from Shrubbery Networks, inc., a tool which I was heavenly relying on in the early 2000s for backing up the configurations of network devices I was responsible for when working for an ISP. (the good old days, on-the-fly live config changes, no red tape, and 20 committees later to implement a simple config change :D )
Fast forward to 2023, I have created a Python script to automatically backup configurations of network devices by fetching configurations from devices using SSH, and then committing and pushing these configurations to a Git repository. This script interacts with Netbox, a popular open-source IP address management (IPAM) and data center infrastructure management (DCIM) tool, to retrieve the list of devices.
Before delving into the script details, checking off a few boxes from the list is essential.
Firstly, ensure that Netbox is installed and accessible via REST API (token required).
Secondly, ensure all network devices are configured in Netbox, with only active ones considered. Additionally, set the correct tag [iosxr, nxos, iosxe] under the device properties.
Thirdly, ensure device credentials are stored in environment variables (further explanation to follow).
Fourthly, ensure you have Python running (I am using Python version 3.10.8).
Lastly, a GitHub account is necessary for creating the repo and pushing configurations. ( I'm using SSH authentication, but HTTPS will work as well)
If you already have an existing key, you can copy the public key with pbcopy and add it easily to your GitHub profile.
My ssh key is known as id_rsa.pub, so in order for me to copy the key to my GitHub profile, I will use pbcopy.
After you successfully copy the public key to your GitHub profile, you can test it to see if you can authenticate to GitHub.
Now on to the good stuff. First, I am no programmer, so I build this script in parts. Initially, I looked at how can I get information from Netbox with Python, so I did a Google search and ended up looking into Pynetbox, which provided me with what I needed (with some trial and error). Secondly, I looked at how to connect to the different devices and fetch the config; in comes Scrapli, which provides drivers to connect to NX-OS, IOS-XR, IOS-XE, JUNOS, and EOS, which has all the functionalities that I needed. Lastly, and for me, the most challenging part was storing the configuration in a GitHub repository, which took a bit of work to figure out the right approach, and I ended up using GitPython.
In the next few paragraphs, I will briefly summarize the script of what each section is meant to do :D.
Importing Libraries: The script imports various Python libraries that offer specific functions
os: provides functions for interacting with the operating system.
json: used for parsing and manipulating JSON data.
requests: used for making HTTP requests.
pynetbox: a Python client for the NetBox IPAM/DCIM tool.
git: provides Git-related operations for Python applications.
pathlib: provides classes for handling filesystem paths.
datetime: provides classes for manipulating dates and times.
scrapli: an easy-to-use screen-scraping library for network devices.
subprocess: used for running new applications or programs through Python code by
creating new processes.
tempfile: generates temporary files and directories.
logging: provides a flexible framework for emitting log messages from Python programs.
Logging Configuration: The logging module in Python provides a way to configure the logging. Here, the script configures the basic settings for logging, where the level is set to INFO.
Environment Variables and Constants: The script defines several constants and environment variables. Constants like SUBNET, SSH_PORT, SOCKET_TIMEOUT, INIT_COMMIT_MESSAGE, and COMMIT_MESSAGE have default values but can be overridden by environment variables. It then sets variables for different components using environment variables, such as SSH_USER, SSH_PWD, NETBOX_URL, NETBOX_TOKEN, GIT_REPO_URL, and GIT_TOKEN.
Environment Variables Check: The script checks if all required environment variables are set and if any are missing, it logs an error message and stops execution.
Function Definitions: The script then defines several functions:
getDevices(): This function uses the Pynetbox library to connect to the NetBox server and fetch the list of devices. The devices are filtered based on the tags specified in the DEVICE_TAGS environment variable. It returns two lists: devices_netbox, which contains a dictionary for each device with details needed for SSH connection, and hostnames which is a mapping of IP addresses to hostnames.
getConfig(device): This function uses the Scrapli library to establish an SSH connection to the device and sends the command "show run" to fetch the device's current configuration. It returns the configuration as a string or None if there is an error.
saveConfig(): This function calls getDevices() to get the list of devices, then for each device, it calls getConfig() to fetch the device's configuration and saves the configuration to a text file in a directory named after the device.
initGit(): This function initializes a Git repository in the config_backup directory, creates README.md and .gitignore files, saves the initial configuration of devices by calling saveConfig(), stages all changes, makes an initial commit, and pushes the commit to the remote repository specified in the GIT_REPO_URL environment variable.
cloneGit(): This function clones the remote Git repository into a temporary directory, removes any existing files, changes the working directory to the config_backup directory, saves the new configuration of devices by calling saveConfig(), stages all changes, commits with a message, and forces push the commit to the remote repository.
create_github_repo(repo_name, token): This function makes a POST request to the GitHub API to create a new GitHub repository with the given name; it uses a GitHub token for authentication. If the repository is successfully created, it logs a success message. If the repository already exists, it logs a warning and if there's an error in creating the repository, it logs the error message.
main(): This is the main function that orchestrates the execution of the script. It first checks if the config_backup directory exists and if it's a Git repository. If it is, it calls cloneGit() to clone the repository, fetch the latest device configurations, commit, and push the changes. If it's not a Git repository, it calls initGit() to initialize a Git repository, fetch the initial device configurations, commit, and push them.
And last but not least, we are executing the script, which you can schedule to launch at regular intervals with a cron job.
Script Execution: The script starts executing from the main() function if it is run as a standalone script.
In a nutshell, this Python script is designed to automate the process of backing up network device configurations, storing them in a version-controlled manner using Git, and interacting with NetBox to retrieve device information.