[ad_1]
The public folder (PF) migration endpoint in Exchange Online contains information needed to connect to the source on-premises public folders in order to migrate them to Office 365. PF migration endpoint can be based on either Outlook Anywhere (for Exchange 2010 public folders) or MRS (for Exchange 2013 and newer public folders).
In this blog post, we discuss PF migration endpoints for Exchange 2010 on-premises public folders. While Exchange 2010 is not supported, we have seen cases where customers still have these legacy Exchange servers and are working on their migrations.
To migrate Exchange 2010 PFs to Exchange Online you would typically follow our documented procedure. If you are reading this post, we assume that you are stuck at Step 5.4, specifically with the New-MigrationEndpoint -PublicFolder cmdlet. If this is the case, this article can help you troubleshoot and fix these issues. Some of this knowledge is getting difficult to find!
Exchange 2010 public folders are migrated to Exchange Online using Outlook Anywhere as a connection protocol. The migration endpoint creation will fail if there are issues with the way Outlook Anywhere is configured or if there are any issues with connecting to public folders using Outlook Anywhere (another sign of this is that Outlook clients cannot access on-premises public folders from an external network).
This means that from a functional perspective, you have an issue either with Outlook Anywhere or with the PF database (assuming that the steps you used to create the PF endpoints were correct).
Let’s look at the steps to troubleshoot PF migration endpoint creation.
Ensure Outlook Anywhere is configured correctly and is working fine
Outlook Anywhere uses RPC over HTTP. Before enabling Outlook Anywhere on the Exchange server, you need to have a Windows Component called RPC over HTTP (see reference here).
The next step is to check if Outlook Anywhere is enabled and configured correctly. You can run Get-OutlookAnywhere |FL in the Exchange Management Shell or use the Exchange Management Console to see if it is enabled (you would have a disable option if currently enabled):
This is a good and complete article on how to manage Outlook Anywhere.
If Outlook Anywhere is published on Exchange 2013 and servers, it must be enabled on each Exchange 2010 server hosting a public folder database.
Checking Outlook Anywhere configuration:
- External hostname must be set and reachable over the Internet or at least reachable by Exchange Online IP Addresses. Check your firewall rules and public DNS and verify that your users can connect to Exchange on-premises server using Outlook Anywhere (RPC/HTTP) from an external network. You can use Test-MigrationServerAvailability from Exchange Online PowerShell (as explained later in this article) to verify connectivity from EXO to on-premises but keep in mind that when you do this you will be testing only with the EXO outbound IP address used at that moment. This is not necessarily enough to ensure you are allowing the entire IP range used by Exchange Online. Another tool you can use for verifying public DNS and connection to on-premises RPCProxy is the Outlook Connectivity test on Microsoft Remote Connectivity Analyzer. Please note that the outbound IP addresses for this tool (mentioned in Remote Connectivity Analyzer Change List (microsoft.com) are different from the Exchange Online outbound IP addresses, so having a passed test result here does not mean your on-premises Exchange server is reachable by Exchange Online.
- Ensure you have a valid third-party Exchange Certificate for Outlook Anywhere.
- Check that authentication method (Basic / NTLM) is correct on Outlook Anywhere on the RPC virtual directory. Make sure you use the exact Authentication configured in Get-OutlookAnywhere when you build the New-MigrationEndpoint -PublicFolder cmdlet.
- Verify that your registry keys are correct:
The ValidPorts setting at HKLMSoftwareMicrosoftRPCRPCProxy should cover 6001-6004 range:
If you don’t have these settings for ValidPorts and ValidPorts_AutoConfig_Exchange, then you might want to reset the Outlook Anywhere virtual directory on-premises (by disabling and re-enabling Outlook Anywhere and restarting MSExchangeServiceHost). You should do this reset outside of working hours as Outlook Anywhere connectivity to the server will be affected.
As a last resort (if you still don’t see Valid Ports configured automatically), try to manually set them both to the following value: <ExchangeServerNetBIOS> :6001-6004; <ExchangeServerFQDN> :6001-6004; like in the image above. If the values are reverted automatically, then you need to troubleshoot the underlying Outlook Anywhere problem.
The PeriodicPollingMinutes key at HKEY_LOCAL_MACHINESystemCurrentControlSetServicesMSExchangeServiceHostRpcHttpConfigurator – by default the value is (15). It should not be set to 0.
The Rpc/HTTP port key for Store service is set to 6003 under HKEY_LOCAL_MACHINESYSTEMCurrentControlSetservicesMSExchangeISParametersSystem
Verify that your Exchange services are listening on ports 6001-6004. From a command prompt on the Exchange server, run these 2 commands:
netstat -anob > netstat.txt
notepad netstat.txt
In the netstat.txt file, search for ports :6001, :6002, :6003 and :6004 and make sure no services other than Exchange are listening on these ports. Example:
Note: We already assume that services like MSExchangeIS, MSExchangeRPC, MSExchangeAB, W3Svc, etc. are up and running on the Exchange server; you can use Test-ServiceHealth to double check. Also, in IIS manager verify that the Default Web Site and Default Application Pool are started. Finally, verify that your PF database is mounted.
Verify that you are able to resolve both the NetBIOS and FQDN names of the Exchange server(s) hosting the PF database. In my examples above, Ex2010 is the NetBIOS name and Ex2010.miry.lab is the FQDN of my Exchange server.
Verify that users can connect to public folders using Outlook
Let’s say you have you verified that Outlook Anywhere is configured and working fine but you are still unable to create a PF migration endpoint. Configure an Outlook profile for a mailbox in the source on-premises environment (preferably the account specified as source credential) from external network machine and verify that the account can retrieve public folders.
We will now get into the next level of troubleshooting section but most of the times, if you managed to cover the above section, you should be fine with creating the PF migration endpoint. If not, let’s dig in further:
Check Outlook Anywhere connectivity test in ExRCA
Use EXRCA and run the Outlook Anywhere test using an on-premises mailbox as SourceCredential.
The tool will identify Outlook Anywhere issues (RPC/HTTP protocol functionality and connectivity on ports 6001, 6002 and 6004, Exchange certificate validity, if your external hostname is a resolvable name in DNS, TLS versions compatible with Office 365, and network connectivity).
Save the output report as an HTML file and follow the suggestions given to address them.
If you are filtering the connection IPs, add the Remote Connectivity Analyzer IP addresses (you can find them in this page here) to your allow list and try the Outlook connectivity test again. Most importantly, ensure that you allow all Exchange Online IP addresses to connect to your on-premises servers.
You can use the Outlook Connectivity test with or without Autodiscover. Here is an example of how to populate the Remote Connectivity Analyzer fields for this test if you want to bypass Autodiscover:
Use the Test-MigrationServerAvailability to test both PF and Outlook Anywhere Connectivity
Test-MigrationServerAvailability command simulates Exchange Online servers connecting to your source server and indicates any issues found.
Connect-ExchangeOnline
Test-MigrationServerAvailability -PublicFolder -RPCProxyServer $Source_OutlookAnywhereExternalHostName -Credentials $Source_Credential -SourceMailboxLegacyDN $Source_RemoteMailboxLegacyDN -PublicFolderDatabaseServerLegacyDN $Source_RemotePublicFolderServerLegacyDN -Authentication $auth
For example, I ran this at 3:03 PM UTC+2 on May 1, 2021. The timestamp is very important to know as we will be checking these specific requests from Exchange Online to Exchange on-premises servers by looking at the IIS and optionally FREB logs, HTTPerr logs and eventually HTTPProxy logs (if the front end is an Exchange 2013 or Exchange 2016 Client Access server). Details on how to retrieve and analyze these logs are in the next section.
Gather Verbose Error from the New-MigrationEndpoint -PublicFolder cmdlet
This step is very important in order to narrow down the issue you are facing, as a detailed error message can tell us where to look further (for example you would troubleshoot an Access Denied error differently from Server Unavailable). You need to make sure you are constructing the command to create the PF migration endpoint correctly.
For this, we go first to the Exchange Management Shell on-premises and copy-paste the following values to Exchange Online PowerShell variables. Examples from my lab:
|
Exchange Online PowerShell Variable |
Exchange On-Premises Value |
|
$Source_RemoteMailboxLegacyDN |
Get-Mailbox <PF_Admin>).LegacyExchangeDN |
|
$Source_RemotePublicFolderServerLegacyDN |
(Get-ExchangeServer <PF server>).ExchangeLegacyDN |
|
$Source_OutlookAnywhereExternalHostName |
(Get-OutlookAnywhere).ExternalHostName |
|
$auth |
(Get-OutlookAnywhere).ClientAuthenticationMethod |
|
$Source_Credential |
Credentials of the on-premises PF Admin Account: User Logon Name (pre-Windows2000) in format DOMAINADMIN and password. Must be member of Organization Management group in Exchange on-premises. |
Then, connect to Exchange Online PowerShell and run this:
# command to create the PF Migration Endpoint
$PfEndpoint = New-MigrationEndpoint -PublicFolder -Name PublicFolderEndpoint -RPCProxyServer $Source_OutlookAnywhereExternalHostName -Credentials $Source_Credential -SourceMailboxLegacyDN $Source_RemoteMailboxLegacyDN -PublicFolderDatabaseServerLegacyDN $Source_RemotePublicFolderServerLegacyDN -Authentication $auth
Supposing you have an error at New-MigrationEndpoint, you would run the following commands to get the verbose error:
# command to get serialized exception for New-MigrationEndpoint error
start-transcript
$Error[0].Exception |fl -f
$Error[0].Exception.SerializedRemoteException |fl –f
stop-transcript
When you run the commands to start/ stop the transcript, you will get the path of the transcript file so that you can review it in a program like Notepad.
Cross-checking on-premises logs at the times you do these tests
IIS logs for Default Web Site (DWS): %SystemDrive%inetpublogsLogFilesW3SVC1 – UTC Timezone
If you don’t find the IIS logs in the default location, check this article to see the location of your IIS logging folder.
After you run Test-MigrationServerAvailability or New-MigrationEndpoint -PublicFolder in Exchange Online PowerShell, go to each CAS and see if you have any RPCProxy traffic in IIS logs at the timestamp corelated with Test-MigrationServerAvailability that would come from Exchange Online. Search for /rpc/rpcproxy.dll entries in Notepad++, for example.
For my Test-MigrationServerAvailability at 3:03 (UTC+2 time zone), I have 2 entries for RPC, port 6003 UTC time zone (13:03):
As you can see, only 401 entries are logged (indicating a successful test). This is because the 200 requests are ‘long-runners’ and are not usually logged in IIS. The 401 entries for port 6003 are a good indicator that these requests from Exchange Online reached IIS on your Exchange server.
If RPC traffic is not found in the IIS logs at the timestamp of the Test-MigrationServerAvailability (for example, MapiExceptionNetworkError: Unable to make connection to the server. (hr=0x80040115, ec=-2147221227), then you likely need to take a network trace on the CAS and if possible on your firewall / reverse proxy when you do test-MigrationServerAvailability.
Now is a good moment to consider the network devices you have in front of your CAS. Do you have load balancer or a CAS array; do you have CAS role installed on the PF server? Collect a network trace on the CAS.
Also check if you have any entries / errors for RPC in HTTPerr logs (if you don’t see it in the IIS logs):
HTTPerr logs: %SystemRoot%System32LogFilesHTTPERR – Server Timezone
Finally, check the Event Viewer. Filter the log for Errors and Warnings and look for events correlated with the timestamp of the failure or related to public folders databases and RPC over HTTP.
Enable failed request tracing (FREB)
Follow this article to enable failed request tracing. If required to write something in the status code, you can put for example a range of 200-599. Then, reproduce the issue and gather the logs from %systemdrive%inetpublogsFailedReqLogFilesW3SVC1
NOTE: Once you have reproduced the problem, revert the changes (uncheck Enable checkbox under Configure Failed Request Tracing). Not disabling this will cause performance problems!
Getting a MapiExceptionNoAccess error?
During a New-MigrationEndpoint or Test-MigrationServerAvailability test you might see a specific (common) error; we wanted to give you some tips on what to do about it.
Error text:
MapiExceptionNoAccess: Unable to make connection to the server. (hr=0x80070005, ec=-2147024891)
What it means:
This is an ‘Access Denied’ error for the PF admin and can happen when credentials or authentication is wrong.
Things to check:
- The correct authentication method is used (either Basic or NTLM).
- You are providing source credentials in domainusername format.
- The source credential provided is a member of Organization Management role group.
- As a troubleshooting best practice, it is recommended to create a new admin account (without copying the old account), make it a member of the Organization Management group, and try creating the PF migration endpoint using that account.
- Run the Outlook Connectivity test on Remote Connectivity Analyzer for domainadmin account and fix any reported errors.
- Check and fix any firewall/reverse proxy issues in the path before the Exchange server.
Thank you for taking the time to read this, and I hope you find it useful!
I would like to give special thanks to the people contributing to this blog post: Bhalchandra Atre, Brad Hughes, Trea Horton and Nino Bilic.
Source link
