The Azure Status page reports a huge Azure Outage. This outage is mainly because of the canonical Ubuntu 18.04 DNS errors. The ongoing Azure incident with VMs recently upgraded to system version 237-3ubuntu 10.54 is experiencing a DNS error.
The entire Azure is not down even though the Azure Status Page shows differently. We could access Azure Bastion and Azure VMs etc., without any issue. The nine (9) services that happen to use AKS / Ubuntu 18.04 internally might have reduced availability.
This reminds Microsoft outage caused by overloaded Azure DNS servers in April 2021 as per the comment from Victor Kupcis on the discussion thread. The outage was so widespread within Microsoft’s infrastructure that even their Azure status page, which is used to provide outage info, was inaccessible back in 2021.
There are many discussions on social media that the reduced availability of Azure can be caused by just one of the upgrade domains being down. However, Microsoft is still investigating the issue and has not published any root cause yet.
As per Microsoft – “Starting at approximately 06:00 UTC on 30 Aug 2022, several customers running Ubuntu 18.04 (bionic) VMs recently upgraded to systemd version 237-3ubuntu10.54 reported experiencing DNS errors when trying to access their resources. Reports of this issue are confined to this single Ubuntu version.”
Azure Status
You can check the latest status of Azure services from – The azure status – https://status.azure.com/en-us/status or https://azure.status.microsoft/en-us/status. You can refer to the following link to get the personalized status on Azure Outage scenarios.
Microsoft states that This bug and a potential fix have been highlighted on the Canonical / Ubuntu site, which we encourage impacted customers to read: Bug #1988119 “Update to systemd 237-3ubuntu10.54 broke DNS” : Bugs : systemd package : Ubuntu (launchpad.net).
Microsoft also identified that there are data access issues and missed or delayed alerts issues with the following services and more. This disruption has been caused by a Canonical Ubuntu update rolled out into one of our underlying services which Application Insights, Azure Monitor, and Sentinel rely upon.
- Application Insights
- Azure Monitor
- Sentinel.
Root Cause of the Azure Outage on August 2022
The root cause of the Azure Outage on August 2022 is unknown. However, Microsoft mentioned that they are aware of an ongoing incident with VMs that recently upgraded to system version 237-3ubuntu 10.54 experiencing DNS errors.
Microsoft provided the following workarounds as well. They also confirmed that this is because of a bug on Ubuntu. This bug and a potential fix have been highlighted on the Canonical / Ubuntu site.
- Another potential workaround customers can consider is rebooting impacted VM instances so that they receive a fresh DHCP lease and new DNS resolver(s).
- If you are running a VM with Ubuntu 18.04 image, and you are experiencing connectivity issues, we recommend you evaluate the above mitigation options.
- If you are not experiencing an impact on your Ubuntu 18.04 images but have unattended security updates enabled, we recommend you review this setting until the Ubuntu issue is mitigated.
Similar Issues – Email Alert Delay From Microsoft Admin Center Issue HTMD Blog (anoopcnair.com)
Author
HTMD Admin Account to provide news and latest updates on the known issue from Microsoft world. We cover Windows, Intune, Azure, AVD, and Windows 365 news.
Discussion