It was a tricky weekend for techies - not only did Microsoft's Azure Resource Manager fall over across the globe on January 21, 2024, local UK retailing giant Tesco also suffered an outage, cancelling multiple customer deliveries and orders due to "technical difficulties."

According to Microsoft, the Azure woes started at 0157 UTC and affected customers that use Azure Resource Manager (ARM) for resource management operations. "This impacted users of Azure CLI, Azure PowerShell, the Azure portal, as well as Azure services that depended upon ARM for their internal resource management operations."

Unsurprisingly, users were swift to take to social media to express their displeasure at the issues.

According to Microsoft: "A backend service made a configuration change that caused ARM web roles to crash."

The fix? Bypass the configuration change to allow things to return to normal. That legendary Microsoft quality control at work yet again.

Sadly, it took a little longer for all Azure services to recover, judging by user feedback. However, a seven-hour outage is not to be sneezed at, regardless. Microsoft promised a preliminary Post Incident Report would come on Wednesday.

Azure's wobble came as Tesco, a Microsoft customer, experienced a full-blown delivery catastrophe on Sunday morning. Tesco staff shared screenshots showing the company deciding that it had no option but to cancel customer orders due to "tech issues experienced this morning."

Tesco has yet to disclose the nature of the glitch, other than describing it as a "nationwide system issue." Some customers received notice that their orders were being canceled, and others did not. All who posted on social media regarding the short-notice cancellations seemed rather put out by the experience.

Words such as "carnage" and "shambles" were thrown around in relation to the issue by users claiming to be Tesco employees.

While there is no confirmation that Tesco's IT meltdown - one wag noted, "I think someone in IT is likely getting fired" - was directly related to Azure's wobble, the incidents are a reminder that backup plans are always a good idea, particularly where online systems are concerned.

And, in the case of Microsoft, a little more testing would be beneficial before backend services are allowed to wreak havoc with configuration changes. ®

https://www.theregister.com//2024/01/22/a_weekend_of_outages_tesco_azure/