Change Control : How does it benefit the Security team and how do you go about enforcing it?

Most organizations should be aware of Change Control (hereafter referred to as ‘CC’), the process of only making system or network changes within a set of parameters. This should, of course, be addressed in your policy. CC has several direct benefits to the Security Team.

By requiring all critical machines to fall under CC, you don’t need a fancy finger-printer to tell you what you are running (and where). In one company where I worked, we had a policy which stated that we (the Security group) could scan *any* non-critical machine at *any* time. Period. We then gave a month for the business units to enroll their Critical machines. We ended up with around 4000 machines (3000 more than we thought we would get). We then had each business unit assign an asset value to the machine. At this point, we had a database of all critical machines as well as the exact asset value, OS, patch version, IP addresses, interfaces, hotfixes, running applications, system owner, hardware utilized, system location, etc. At this point, we hardly even needed our scanner any more. For example, if a flaw comes out in IIS 6.0, we know exactly which critical systems need to be addressed first (see my post on asset categorization and CVSS).

Having a Security Team member attend the weekly CC meetings allows you to have input into proposed changes. If a new business partner connection is coming up over the weekend or a new remote location is connecting into the Corporate network, you now have a SAY in how these changes come about (or even *if* the changes will come about). Too many people rely on their scanners and finger-printers to tell them how the network is laid out. IMO, you can get this by having a good CC process.

By participating in the CC process, you minimize your Risk of being blamed for network or system outages. Believe it or not, scanning critical systems can cause them to fail. The Security Group should only scan critical resources within an assigned CC window. This can stop a lot of finger-pointing.

I’m gonna go way OT for a second. Large organizations often have critical machines which are highly unstable. Not many pen-testers get access to multi-million dollar hardware or software. There are probably a thousand undocumented bugs in HP-UX, DG-UX, SCO, AS/400, AIX, Oracle, SAP, People Soft, etc. As a security team, you can easily fall into the rabbit-hole of communicating and QA’ing each of these flaws to the vendor. I know I’m on a slippery slope here, but you have to PRIORITIZE your time. I have a finite security team with a finite amount of time. I’m sorry that the vendors have written such sloppy code; however, my goal is to protect the companies resources while enabling the business. As I identify flaky applications on critical systems, I put them on a separate network behind a firewall. These devices should, typically, only be connected to by certain machines on certain ports. Let the firewall go to work for you. Minimize the Risk. Have the business unit assume the remaining Risk. Get on with life.

How do you go about enforcing CC? Of course, you are monitoring every critical machine, right? You have designed a network which is defensible, recoverable (that whole DR/BCP thing) and designed around critical resources, right? If that’s true, then you just have to monitor changes. I, personally, will know within 30 minutes if a new service pops up on a critical machine. I’ll know if the critical machine is initiating odd connections out of their allowed range. To enforce CC on your critical machines, you must simply have the ability to detect changes on these systems. And, you must have a compliance team that can address infractions with both the culprits as well as the higher-ranking brass. Believe me, after a few months of ENFORCING the CC, you’ll rarely find changes outside of a CC window. I know this from experience.

I’m gonna go OT one last time. With respect to ‘early adoption of new technologies or applications’, most organizations that I have worked for severely frown on it. Code or applications which have not been extensively tested are rarely allowed to pass through CC. Not being an early adopter can save your butt BIG TIME! Large corporations are like a big ship…i.e. they tend to move slowly and have a hard time quickly reversing themselves [1]. It may take months or years to fully roll out an Enterprise solution. If that solution turns out to be the *wrong* solution, it’ll take at least as long to undo your error.


[1] actually, the analogy doesn’t stop there…large ships must also be compartmentalized in order to ensure that a single hole doesn’t sink the entire ship…but, that’s a whole ‘nother post :)