The 5 Best Ways to Eliminate Boredom and Terror on Your Network: Part 2

This is the second of three installments on this topic. In our last post, we talked about the leading cause of network downtime: simple human error. These preventable errors result in nearly 80% of network outages, which cost organizations about $1 million and impact mission objectives. We also explored some compelling reasons why network configuration management is so difficult.

In this installment, we will talk about three specific best practices that will help you improve network uptime and operational success and save you time and money. In tomorrow’s post, we’ll talk about two more.

Ready, Fire, Aim!

Many of us work in a “ready, fire, aim” world. We are under tremendous pressure to get things done fast - so fast that we sometimes don’t feel we have time to properly aim before we fire!

What does this have to do with network configuration? This: Most technical problems can’t be solved by buying and using technology alone. Rather, resilient solutions take some thinking and a combination of technology and process.

This then brings us to the heart of our discussion, five best practices designed to improve network uptime. Today we will talk in depth about the first three.

Inventory and Profile Network Systems

Our objective in this first practice is to identify all devices under management. With hundreds of devices on your network, it’s important to know certain information about each device.

To make this easier, you need tools that perform an automatic discovery scan of your network and build a database of device profiles. Then, you will want to organize these devices (by vendor, location, or some other method) and collect and manage useful details about each device.

For example, what is the device’s serial number? Where is the device located? And who is the primary point of contact? Having a well-defined device profile can also help you improve control costs and compliance. Consider the following examples:

• Firmware upgrade. Be able to easily identify which devices need (and are compatable with) the upgrade and which devices were successfully upgraded. This is useful to meet service, security, and compliance objectives.
• Maintenance audit. Easily determine if there is agreement between devices installed and devices covered under the maintenance agreement.
• End of service. Easily identify which devices are no longer vendor supported and should be replaced. Has a budget been requested and approved? Have resources been scheduled to retire the device?

As mentioned earlier, it’s very helpful to have this information saved as part of the device profile. This makes it easier to maintain one authoritative source and share it with others throughout the organization. If this information is incomplete or managed externally to the device profile (perhaps using a spreadsheet), then extra work is required to keep it current and protect the integrity of the data (e.g., from getting out of sync due to multiple document versions).

By following this first practice, you will be able to holistically manage network configuration by creating a sound foundation that can help make informed decisions.

Develop and Deploy Standardized Device Configurations

Our objective with this second practice is to reduce errors by improving consistency and standardization across device configurations. To do this, you need to look at three specific recommendations:

1. Use standardized access properties (e.g., protocols, ports, IDs and passwords).
2. Use configuration templates to build standardized configuration tasks.
3. Use bulk deploy to push changes out to devices.

The first consideration looks at how you remotely access your devices and standardizes those methods, which include the device login information, communication protocol, and IP service ports. By standardizing how you remotely access devices you can:

• Ensure all devices are accessed using a secure communication protocol
• Ensure no devices are using vendor-supplied IDs and passwords and that all passwords are strong and conform to your security policy
• Ensure all account IDs and passwords are synchronized and updated easily and routinely

The other two considerations recommend using script-based templates to standardize and automate complex configuration changes. Using templates is an excellent way to reduce error because you can develop and test one template and then consistently apply changes accurately to a number of network devices.

You can use templates to perform routine tasks like changing VLAN memberships by port, configure device interfaces, and enable a variety services like IPSLA, NetFlow, and more. In addition to building a template, you can also schedule the execution to perform ongoing changes during off-peak times.

When you use templates, you can improve configuration management in the following ways:

• Reduce hundreds of command statements into a single script that can be tested and consistently applied error-free to as many devices as required
• Perform repetitive tasks with consistency

Protect Configurations from Changes

Our objective with this third practice is to protect stable configurations from both unauthorized changes and catastrophic device failure. Protecting your configurations makes sense for a number of reasons. Foremost, once you have invested considerable time and effort getting everything to run smoothly, you want to keep things that way. But there are other reasons as well, like being able to quickly reverse a mistake or rapidly standing up a device spare following a catastrophic failure. Therefore, when operationalizing this practice, consider these suggestions:

• Backup device configurations for fail-safe restores
• Monitor configurations in real-time for unintended changes, and respond immediately
• Use tools to identify specific changes to a configuration
• Maintain a change audit trail to always know who made changes

This first recommendation is important (and is a leading reason why people use a network configuration and change management solution) because when you need to reverse a mistake or quickly provision a spare, you need to have a backup copy of the most recent device configuration to restore. Doing backups of device configurations will help you automatically and remotely backup these configurations and restore them as the need arises.

The latter three suggestions speak to the need to monitor active configurations so you can detect changes and determine whether they are intended. If they are not intended, you then have the tools to reverse the change by restoring the most recent prior configuration. Furthermore, in addition to simply knowing that a change has occurred, you can isolate the change and determine whether it is merited.

When you actively protect your working device configurations from change, you can save yourself a great deal of time and effort. Whether an unintended change occurs or a device fails, you will always be able to tell what happened and quickly restore service.

Summary

These first three practices will help you identify all systems under management, standardize the way you manage these devices, and help you protect them. This is important as you devise a holistic approach to network configuration management and will result in reducing variability and improving reliability. In our final installment, we will review the remaining two best practices.