.png)
As All-Hallows-Eve approaches and our thoughts swing on rusty hinges from cooler nights, hot mugs off cider, and bags of candy to ghouls, ghosts, and B-Movies that scare the bejesus out of us. It reminds us of the stories of monsters and villagers, the hunters and the hunted.
There is one particular technology of note. An IT boogey man. A story of the evil shadow system that IT must hunt down and destroy at all costs! But, as with all stories, there's more than one side to it.
Definition of a shadow system
Understanding the nature of this Shadow System monster involves first understanding what it is and its origins.
According to Wikipedia, a shadow system or shadow IT is a term used by IT for any application relied upon for business processes that is not under the jurisdiction of a centralized information systems department (IT). That is, the information systems department did not create it, was not aware of it, and does not support it.
So, it seems Wikipedia also identifies shadow systems as the monster and Information services (IT) as the hunter. A straightforward monster hunt then…right? Did you notice the plot twist?
Use of shadow systems for business processes
Business processes are used by the business and shadow systems are a business process. In fact, it is usually some business actor that created it. Regardless, IT may believe it has the mandate to control shadow systems and the mission to hunt them down; to destroy them. The plot thickens.
The monsters IT perceives aren't going to be seen by the business as particularly monstrous as the business created them and because they usually play an active role in the business. This makes them difficult to dispose of outright and maybe even a need to understand the impact of technology on workplace culture.
Perhaps the Wikipedia definition should be extended to explain; for every shadow system exists a business narrative that requires its existence. Going a bit further, shadow systems have their own set of rules, controls, and methods but they aren't known or managed by Information Services.
The shadow system hunt
Sometimes, hunting down shadow systems just means asking around but you must ask your questions in a meaningful way to discover their existence. Chances are the business won't even recognize the term 'shadow systems' particularly if you use the Wikipedia definition.
While the business is a good place to look for shadow systems, IT itself may also harbor them. In IT's case, if there are single use or specialty systems being employed to complete various tasks without the knowledge of the larger IT group, these are also shadow systems and can cause similar types of problems shadow systems can cause for the business.
Shadow systems most commonly involve data and are operated by one person from their workstation or laptop. A common theme is, something is done to a data set behind the scenes to transform or enhance it before it becomes part of another system or report. Shadow systems will often involve one or more spreadsheets.
How to find shadow systems
Here are a few ways to reveal the existence of shadow systems:
- Someone resigns or retires and suddenly, part of the enterprise stops working.
- New software is installed or an existing package is updated and a supposedly unrelated part of the enterprise fails.
- Systems that should have the same data don't. Data may be missing, deleted data may be present, or one or more specific records data is not the same across systems like when data is hand entered from system to another
- Reports with data from multiple sources which are not formally compiled (as in a data warehouse structure).
- Conflict sometimes called ‘data brawling’ occurs when formal system outputs are challenged by shadow system outputs
YOU MIGHT LIKE: Tackling Succession Planning Woes Head On
Mapped-out workflows may also aid in the search for shadow systems. Look at points where multiple processes merge or where a process has a person's name attached to it. Likewise, if a work or data flow's content is changed by a spreadsheet, program, or other piece of non-standard software, this can also be a clue that a shadow system is lurking nearby.
Examples of shadow systems in the utility industry:
- A project manager might use a shadow system to keep their own set of information on budgets or timelines
- Field crews might create their own work order system
- A utility employee creates their own system to compile and report water consumption data by customer type to submit to an agency on a monthly basis
Inhuming a shadow system
With the discovery of a shadow system, before burying, remember that shadow systems exist for a reason, and you might not have direct control over it anyway since it was created by the business likely with local technology such as Excel.
More critical, its genesis often arises from a genuine desire to improve work output, either to increase speed (e.g. not waiting around for long downloads) or a desire to have control over the data so mindful analysis can be conducted that formal IT systems may not currently support. Given this perspective, you are most likely going to have to negotiate a solution.
Shadow systems have been vilified in the business press for their risks. When found, their creators and the system are often criticized despite the good intentions or legitimate frustration by which they were created. This can be particularly painful to the business who may, in part, be reflecting their perception that their needs were not heard or met by IT.
Thus, this solution is ensuring the valuable part of the system is not lost or overlooked. Technology groups will have to learn to be ok with not controlling key business information.
If you are going to remove a shadow system, you will have to replace it with something that performs the same task or works around whatever issue was being addressed by the shadow system. You might want to find those individuals responsible for outcomes of the shadow systems (software advocates) to help you find new or replacement systems.
Don't panic if it doesn't appear there is an immediate solution that eliminates the shadow system. You will likely need some patience and time to negotiate a resolution with the business. Remember, your solution may be more complex than the shadow system it is replacing; adding more technical debt for IT to manage.
You don’t have to be afraid of the shadows
Shadow systems, while perhaps still monsters, are going to be a bit more complex than you might anticipate. While you should keep an eye out for them, don’t be surprised if one surfaces because of a system issue. Hunting them down and killing them directly is usually not an option. If a shadow system is part of an operational function, then remember one important rule, “Don’t break it”. Lastly, shadow systems usually require resolution with the business that is acceptable to both the business and IT.
Trinnex can help you transform hazy outlooks into clear solutions.
