Outages and any downtime in the regular operations of IT systems can significantly disrupt businesses around the globe. This is especially true in the realm of cloud computing. The recent CrowdStrike outage is a prime example of such disruptions, with the event costing US Fortune 500 companies over $5.4 billion. Their Microsoft Windows antivirus update inadvertently triggered a widespread system failure, underscoring the importance of robust disaster recovery strategies. For Absoft, it emphasised the vital role of expert IT support teams and presented an opportunity to demonstrate how our Basis team can effectively navigate these situations. We successfully mitigated the negative effects for several of our clients, showcasing the value of proactive, expert intervention during such crises.
Don’t hesitate to get in touch if you require any help or have a question.Â
Absoft’s Response: How Our Basis Team Stepped Up
On Friday, July 19th, many SAP clients unexpectedly found their systems out of service due to the CrowdStrike update. This update impacted SAP systems and crashed other non-SAP servers. Internal IT teams were required to work tirelessly to bring their servers back online. Amid this widespread disruption, Absoft’s Basis team was crucial in restoring and stabilising our clients’ SAP systems as per our ongoing support.
The unexpected reboot of SAP servers resulted in critical disruptions, taking both database and SAP implementations offline. Such an event had the potential to cause prolonged downtime, disabling many essential business operations. However, swift action from our Basis team kept downtime to a minimum and managed the crashes effectively and efficiently.
Proactive Measures and Expertise in Action
While the outage was unforeseen, our team was well-equipped to deal with such scenarios. Drawing on their expertise and experience, they took decisive action in several key areas:
- Rapid Identification and Isolation: The team immediately identified customers with affected systems upon detecting the issue. By isolating the affected machines, they prevented the problem from spreading further within the customers’ network, containing the bug swiftly.
- System Restarts and Health Checks: Our Basis team assisted clients in carefully restarting their SAP systems, ensuring the process was conducted smoothly to avoid further complications. Following the restart, the team performed thorough health checks to verify that batch SAP jobs and interfaces were functioning correctly. This step was crucial in restoring full functionality and ensuring business operations could resume.
- Client Communication and Support: Throughout the process, our team maintained clear and consistent communication with clients, keeping them informed of the situation and the steps being taken. This transparency was key to managing client expectations and ensuring confidence in the recovery process.
Best Practices and Lessons Learned
The CrowdStrike outage was a huge challenge for our team to address. As such it reinforced several critical best practices to take away:
- Importance of Localised Support: It was crucial to respond swiftly with a team that understood the intricacies of each client’s system. Our localised support allowed our team to address issues in real-time, drawing from experience working hands-on with the clients over time. This helped significantly accelerate the recovery process and reduce downtime.
- Thorough Pre-deployment Testing: This unforeseen incident stressed the importance of rigorous testing before deploying updates. Our Basis team comprehensively tests in offline environments to prevent similar issues from affecting key business systems.
- Regular Disaster Recovery Drills: Absoft ensures all clients have comprehensive disaster recovery plans, complete with procedures, checks, and regular audits. These plans are tested routinely to ensure that businesses are prepared for any unexpected events, promoting swift recovery and minimising the potential downtime of future such outages.
Conclusion
The CrowdStrike outage was a significant event that challenged the resilience of IT teams worldwide. For Absoft, it was an opportunity to demonstrate the expertise and dedication of our Basis team in navigating and resolving complex challenges. By responding swiftly and effectively, our team minimised the impacts on our clients, ensuring their operations continued with minimal disruption.
Ultimately, the event stresses the importance of having a trusted IT support partner like Absoft that provides immediate crisis management. A partner should also strive to improve IT infrastructure and processes for each client continuously. This ensures long-term stability and resilience to whatever may be coming around the corner.
To learn more about the Absoft team’s experience, tools, and capabilities, please reach out now!
