We all have bad days, yes? Working in IT in any environment sometimes feels like the world set up this morning with a clear goal, to discredit what we are trying to achieve. All the platitudes of business transformation at the centre of what we do go out the window really quickly if run-first is missed. I love all aspects of my job, I always have, since those very early days in the National Programme for IT for the NHS. I remember the run up to a Christmas with Choose and Book in its most formative state and the SPINE being down and everyone running around trying to work out what to do, its times like those when a clear head and a view of how to make this ‘just’ work are so important.
But sometimes the IT part of these issues become a thankless task. If things don’t work, or are delayed the simple questions, “why?” gets asked a lot, and its really hard to say sometimes we just don’t know yet, we will and when we do we can fix it but until then we are in problem solving mode.
What an IT leader needs in those times of stress and conflict is some support, brain support to bounce ideas off, support of additional people can help and certainly encouragement always is welcome. An ask to report back more or allow more senior people to get involved is almost never welcome, and as leaders we need to learn to accept that. We have a team whose job it is to run-first and deliver, no matter how much we love the crisis or the superman moment we all need to ease off and allow those that know to simply do.
The role of a digital leader in a crisis is to protect the people that are the brains and brawn of the moment. The ask is to become that fire break for them to just get on with it. I guess sometimes we need to be the escalation conduit too. Whilst the CDIO of Leeds Hospitals I had a major outage of our entire EHR, a moment in time when Leeds really did realise that it was ‘fine’ to not invest appropriately in IT if it was prepared for running as a business continuity hospital more often, and that night the hospital definitely wasn’t. Twelve hours with no way to electronically admit a patient and therefore no way to move anyone ‘up the house’ led to gold command being set up, which was supportive in so many ways, but helpful, definitely not. We were probably an hour away from having to close one of the largest hospitals in the north when the heroic efforts of the traditional IT engineering roles got the disks whirring again and the system came back to life, slowly but surely we had normality and all the ‘interested’ parties could breath a sigh of relief. What happened next though; well firstly the IT team had to make sure it came up and stayed up, then it had to work out why it went down, then it had to make sure it didn’t happen again, including the writing and passing of the business case to make sure it was funded to not happen again, and that is a different story, see the blog entitled IT isn’t free for a bit of a restless nights sleep.
When something goes wrong its IT that is at fault, when something is successful our business can bask in the glory of its success. That is unfair I know and provocative on purpose but anyone with a few disasters under their belt by now knows, sadly this is kind of true.
It is important though that we know our place and can be happy in it, we are a foundation to success, technology provides the life blood to business, data, transactions, information, insight all flow because technology got it right and when it goes wrong its technology not digital that needs to step up and fix it. When we are asked to describe the value that IT delivers though it is often hard to gain acceptance for ‘stealing’ the business thunder of success that is facilitated by IT working so we hunt around (and mostly fail) to find other metrics we can use to show the value of what we do.
Sometimes, when IT is forced to ‘seize the story’ the world becomes different. Wannacry (sorry, that old chestnut) in Ireland is a good example and one where IT were the saviors, sadly a few years later they had to do it again in dire disturbing circumstances but before we dwell upon why it happened a second time lets give some air time to the incident itself and the IT reaction.
IT people are connected and mostly without too much personal competition, one of the key reasons Ireland faired so well during Wannacry was the network of people connectivity and the early warning the NHS was able to give the HSE in Ireland. The IT reaction then to pre-empt the disaster was something I remain proud of to this day; without question the people power of reaching out to a geographically dispersed cohort of IT professionals meant that within three hours of being warned of the issue the system was protected and could continue delivering care, almost unnoticeable to the clinical teams in every hospital across Ireland. The decision to then empower that team to explain to every person who worked in health and as the weekend went on to every possible patient as they came to enter the system meant that a single version of the truth for the fix, for the continuity of the solutions and to manage the slow and steady impact was possible. The system itself could be left to do its job, one of the most important jobs in any country, meanwhile IT could concentrate on the task in hand, maintaining the systems that were needed and as each little blip of pain occurred fixing it at speed.
The value of the IT by the Tuesday (4 days after the warning) was definitely felt; the minister for health coming to the IT help desk to meet the team and thank them for their hard work was so impactful, IT feeling loved by the wider ‘business’ was one shock quote I remember being told.
But sadly roll on a few years and the whole damn house of cards came tumbling down again, I wasn’t there this time but watching the media from afar it felt like the team had to do much of the same work, although this time in recovery rather than last hop protection. The media reaction and the limited system reaction I could pick up though still showed IT as the heroes of the hour, why, because they had been and remain brutally honest about the why and the what they needed to do and perhaps also less blunt but still making it clear that investment in IT is needed to stop cyber issues from being so impactful.
Incidents happen for so many reasons and sadly at many different points in time, resolution is hardest when we need to decipher where the issue is first and with whom. This is when the most heritage of IT skills comes to the fore, the ability to collaborate with many partners at once. My time at the National Institute of Health Research in the UK was where I honed so many CIO skills, it was after all my first role with the ‘badge’.
One of the biggest deliverables for me in that first CIO role was the design, procurement, build and go live of a new system that would manage the clinical trials portfolio of every clinical trial in the NHS. A huge system, complicated, desperately under invested in up to this point and the culmination of a modernisation programme that had delivered so much change over a three year period. But this time IT wasn’t the hero, the IT team involved were just not heard by the business leaders who knew what they wanted the system to do down to the last nut and bolt, a great place to be if you have plenty of time and money for the system to be built and delivered to such a detailed specification, but if you don’t then IT starts to fail the business really quickly. Before the system was even live we were in recovery, the partner at the time was stuck so heavily between what the business wanted the system to do and what the technology could actually give the business and every edge case that was required needed to be met. Ultimately the business requirements were met in a different way with a number of systems being created that offered the solution as a suite of systems but a really challenging period of time for everyone, but a period of time where people learnt so much. Incidents are caused by so many moments in time, in recent times when it is ‘grey IT’ that is the cause of the incident then this does become super complex.
Incidents always happen at the point in time when you don’t want them to and come in all shapes and sizes in reality. They always create a ‘bad day’ but as leaders we need to reflect on these moments in time when we come out of the other side. We need to reduce impact and frequency but we also need to be prepared for when not if. When issues do happen we need to be the best us we can be, even if there is lots of finger waving, sucking of teeth and demands on people that we wish we didn’t have to make. Take a moment next time the day is less good and reflect on how you felt when the incident was averted and the elation of giving the business you were the foundation of back their ability to do what they do best, serve customers, patients and colleagues. We are that fourth utility, and until we are not there we are just a foundation, when did you last thank the water company for keeping the pipe working that gives you the water you need in the middle of the night.
And, if the day really is ‘that’ bad cast a few words skywards to Saint Isidore of Seville, maybe he can help, the patron saint of technology… Its not that bad, we are all in it together, i promise.