This blog post was published on 27th March 2023 and the information may be out of date.

On Thursday morning last week, a small subset of our users were told they couldn’t log in to Microsoft services even on their work computers. It turned out that two local Internet Service Providers (ISPs), Jurassic Fibre and Wessex Internet, had been interpreted by Microsoft as originating in Uzbekistan.

The security feature set in Microsoft Azure is complex and powerful. Blocking access by location is a useful and popular overarching rule that many organisations use to block sign-ins from countries they don’t expect their staff to be logging in from.

We saw on Reddit* that some other administrators had simply chosen to turn off country blocking, but that made me exceptionally nervous to even think about. There’s always a level of background noise in our sign-in logs from other countries, usually automated login attempts that are unavoidable. We chose to set up an exception for affected users instead, and only add those that we saw affected in the sign-in logs (and/or that had contacted our colleagues on the ICT Service Desk). That exception has since been removed, as Microsoft reported the issue resolved that evening.

So what exactly happened? Microsoft reported that the issue lay with the third party they use for their database that matches IP addresses to locations. Our initial assumption was that the two local ISPs had been growing fast (good for them!) and they had both recently acquired blocks of IP addresses that used to belong to ISPs in Uzbekistan. But then, the Reddit thread revealed that it had happened all over the world, so I guess it was just a bad update pushed to Microsoft.

It was definitely an interesting morning! We have a really hard-working, skilled and knowledgeable cyber security team. I don’t mention anyone by name in my public blogs, but I would never claim to detect or resolve anything on my own and I’m very grateful to work amongst supportive colleagues and friends. I enjoy professional and creative writing and I think it’s important to give the rest of the organisation an insight into what we do — not just pushing phishing training and making you sign up for MFA.

*I know, Reddit isn’t professional by a long shot, but the /r/sysadmin subreddit is a pretty great source of information, especially for discussions during the Windows patching cycle. On the subject of Reddit, here’s their very open, honest and detailed account of a security incident they had earlier this year.