Hi Guys,
During a cross tenant migration this week-end, we had a bug on Microsoft Office 365 backend.
The issue blocked the SMTP domain striping from one tenant, this issue blocked the migration because you could not bind a domain that is already linked to an Office 365 tenant into another tenant.
Little bit of context:
- both tenants are active tenants,
- both tenant have at least 14k Msol Users and UserMailboxes,
- both tenant where connected using AADConnect, but source tenant was disconnected from AADConnect,
- the migration needed to be done out of office hours and during a weekend,
- the source tenant was created in 2014 by Microsoft for a migration from Lotus Notes.
As a first step in the cross Office 365 tenant migration you need to strip the SMTP domains from every Office 365 Objects. To proceed with it you need to change the default domain to the tenant technical email address. You need to make the change on Azure AD as you cannot change default SMTP domain on Exchange Online side.
You can do this using this PowerShell Cmdlet:
[ps]Set-MsolDomain -Name "contoso.onmicrosoft.com" -IsDefault[/ps]
It should normaly change both default domains in Msol (Azure AD) and Exchange Online. For those of you who may not know, the informations stored in Azure AD are synced to a backend directory for each services (SharePoint Online, Exchange Online, Skype for Business and others).
The issue we faced was that the synchronization process for the source tenant for Domains objects was broken on Office 365 side, for at least a year. We found the issue when we start removing the old default domain of the customer.
What we where thinking to be the old default domain, kept coming back as an alternate email address on every Exchange Online Objects.
Using the PowerShell Cmdlets:
- For Msol (Azure AD):
[ps](Get-MsolDomain | Where-Object { $_.IsDefault -eq $true }).Name[/ps]
- For Exchange Online:
[ps](Get-AcceptedDomain | Where-Object { $_.Default -eq $true }).Name[/ps]
It displayed us two differents domains, and after further check we found out that domains added or removed for the last one year where not synced at all between Msol and Exchange Online.
You are now in a kind of deep shit. We spent 12h with Microsoft support, had 3 escalation engineer shift, get in touch with Azure AD Product Group and with the Exchange Online Product Group too, spent couple of hours with the Microsoft Technical Account Manager to unlock the situation and get the ticket to speed-up.