Migrate user’s Outlook Data Files (PST) to Exchange Online

Hello!!! In this blog post I will explain what options are available for user PST’s migration to their Office 365 Mailbox. I will cover detailed understanding / information on Quest Migration Manager for PST’s (QMMPST), which was previously known as Quadrotech PST Flight Deck. I will focus on migrate PST to Exchange Online (EXO) – Office 365 using QMMPST tool and also brief other available options within a tools. QMMPST installation, configuration and recommendation on 2 or 3 PST migration scenarios will be cover in next blog post.

Background: Traditional way of email archiving is to create Outlook Data Files (.pst) and move email into it. As server storage was very costly as compared. Hence, organizations chose Outlook Data Files (.pst) as email archiving solution. With Outlook Data Files (.pst) there is always risk associated with data loss and not accessible from anywhere, as PST stored on local system or network storage. Now, Office365 or cloud is offering unlimited storage with lower-cost and access the archive data from everyplace with security and no/less data loss.

Now, organizations are choosing the Office365 online archive as an Email Archiving solution. Office365 Online Archive is budget-friendly and same time, secure and accessible from everyplace.

Broadly, there are 2 options for migrating PST from your local system and/or network storage, first option is Microsoft provided native way (Free), and second option is 3^rd party paid tools. In this blog post I will consider Quest Migration Manager for PST’s as 3^rd party tool. Following are key challenges with native PST migration option:

Discovery: This is one of the major challenge, discovery of PSTs and identity the ownership.
Administrator control: using native way, there are multiple touchpoints, where admin need to perform tasks manually.
PST ownership validate: This is limitation with native way, as there are no option to validate the ownership before migrating to user’s office 365 mailbox.
Control on PST modification: using native way, there are not option to prevent users to modify their PST.
Reporting: basis reporting with less details.

Please refer Overview of importing your organization’s PST files detail step for Microsoft provided native way of PST migration.

Quest Migration Manager for PSTs (Quadrotech PST Flight Deck):

Quest Migration Manager for PSTs is an enterprise grade tool for the migration of PST files into a desired target environment. It is a fast, and ZeroIMPACT solution to automate the identification, migration and elimination of PSTs. Quickly and reliably migrate data to targets such as Office 365, Exchange, Enterprise Vault and hybrid configurations.

Migration Manager for PSTs is designed to reduce the time and level of effort required to discover, centralize, process, ingest, and eliminate PST files. Migration Manager helps reduce security vulnerabilities, ensure compliance and enable robust, multi-platform email availability. Get advanced PST identification, comprehensive project management capabilities and world-class services and support to ensure success. Migration Manager for PSTs enables you to take control of PSTs once and for all.

Features:

Discovery: It will discover the PSTs and map the PSTs with individual user.
One console: Admin require to login into one console to perform all task, like validate discovery, enable for migration, and for reports.
PST ownership validate: Quest MMPST uses a very robust algorithm for deciding PST ownership. Using eight configurable parameters, it calculates an ownership probability that is used to assign ownership. In 99.9% of cases Quest haven’t seen migration of PST file to wrong user mailbox.
User control: QMM PST gives 2 options, enabling users to decide which PST required to migrate or which not? Other option, silent migration, where all discovered PST will be migrate.
Reporting: Audit-ready reporting, View management summaries, capacity planning reports and full chain of custody audits for all users and files. Live progress monitoring, Access comprehensive data around progress, files identified, files in queue and files migrated with the live dashboard and reporting engine. Quickly check the status, act if any issues occur and select the next batch of files or users for the migration queue directly from the dashboard.

Please refer Quest Site to get detailed features for the Quest Migration Manager for PSTs.

Supported source and target:

QMMPSTs gives multiple options for source of PST and target of PST migration. PST Source can be workstation, file server/ Network share, and Removable media, and the target will be Exchange online – Office 365 (primary or archive mailbox), On-premises Exchange (primary or archive mailbox), and Symantec Enterprise Vault (EV). Following diagram is self-explanatory about the supported the PST store location and that can be migrated to list of targets, with no/less impact to user(s).

Architecture:

Quest Migration Manager for PSTs is primary client server technology. Client used to discover PST on user local and/or mapped network shared and Server module used for ingest PST to user primary or archive mailbox. QMM PST tool use IIS to run modules/task and store all information in SQL.

With basic configuration, QMM PST requires at least one server, a SQL database, and Migration Agents deployed. The one server will have all required module i.e. Core, Modules (BITS for upload), processing (Extraction, Ingestion and post- ingestion), Console.

Core:

This is required component, which responsible for communication with the SQL, other Modules, AD, Office 365, Azure AD, and execution of scheduled/non-scheduled operations.

Background Intelligent Transfer Service (BITS – IIS component):

This is not a QMMPST component. This is the IIS component, which is part of pre-requisites of the QMM PST server. BITS is used for uploading the PST to upload folder/ centralized location. Background Intelligent Transfer Service (BITS) to download files from or upload files to HTTP web servers or SMB file servers. BITS continues to transfer files after an application exits as long as the user who initiated the transfer remains logged on and a network connection is maintained. BITS will not force a network connection. BITS resumes transfers after a network connection that had been lost is reestablished or after a user who had logged off logs back in.

Modules & Processing:

This is required component, which include Pre-ingestion, Content Scanning, ingestion, and post-ingestion.

Pre-ingestion: This component is responsible for validating the PST uploaded, taking backup, PST Extraction, and Repair & Deduplication (if required).

Content Scanning: This component is responsible for determining ownership of a PST file. It uses a very robust algorithm for deciding PST ownership. Using eight configurable parameters, it calculates an ownership probability that is used to assign ownership. It can provide more information to help determine ownership association based on the contents of the file. Files processed by the Content Scanner can have the most common sender and most common recipient recorded and taken into consideration when determining PST file ownership.

Ingestion: This component is responsible for an actions taken to ingest a PST file’s contents into the desired target.

Post-ingestion: This component is responsible for the tasks necessary to clean up upload directory and Source File after a file has been migrated.

Central Upload Agent (CUA): This is not a required component. The primary role of the CUA is to upload files stored on upload folder/centralized repositories more rapidly without having to use a workstation’s Migration Agent.

Microsoft SQL:

This is required component, and used for storing configuration details and the user information synced from AD, System, PST details fetched from QMM Agent.

Console or Interfaces:

This is required component, QMM PST offers 2 console Admin Console and Management Console.

Admin console provides the most access to configure a QMM PST environment. It is typically available on the Core server and is installed as an available feature by the Core installer. Access to the Admin Console is required during the initial install, setup, pilot, and ramp-up of your project, but is not typically required for the duration of your project. Alternatively, a project can be entirely managed from the Admin Console if desired.

Management Console is intended to provide operators a client to use on their local workstations. The Management Console provides the options necessary to perform the daily operations of a PST Flight Deck migration without requiring access to a PST Flight Deck server. The primary difference between the two consoles is the Management Console does not have the option for the “Settings” menu as the Admin Console does to perform configuration or setup related changes.

Agent:

This is a required component, which is installed on the user(s) system/workstation. This agent is critical for discovering PST files, providing meta-data to help determine PST file ownership, managing the user interaction with the migration process, and uploading PST files to a centralized location throughout a migration project. QMMPST Agent use BITS to upload the PST files to upload directory/ centralized location.

Network Connectivity diagram:

With basic configuration of QMM PST, it is required one server of QMMPST required components and one SQL. Following diagram is self-explanatory about the connectivity between client system to QMM PST server and from QMM PST server to Office 365.

Communication flow between QMMPST server and QMM Agent (Client System):

Following diagram is self-explanatory on communication flow between the QMM PST Server and QMM PST Agent. Once QMM PST Agent installed and configured on client system its will push the basis information like user and system information back to QMMPST server. Based on profile (will cover in detail during installation and configuration blog post) assign on user, configuration file push back on client system, which contains filters, folder/drive to be scan, excluded folder/file type, frequency of discovery, frequency of reported update on QMMPST server, upload folder/centralized repositories information, etc. This is cyclic process based on configuration.

Once User enabled for PST migration, QMM PST server pushes that information to QMM PST Agent. It will lock the PST modification action for the user as incremental PST sync is not supported. QMMPST Agent will create snapshots and using BITS start uploading the PST to upload folder/centralized repositories. QMM PST server will start next set of task and migrate the PST to users target mailbox and/or archive repositories.

PST Migration Workflow & Stages:

There are following stages:

Snapshot: Typically this stage executes on the client system and QMMPST Agent performs tasks. This will be excluded in case center upload agents upload PST files.

Upload: There are 2 options to upload PST files to upload folder/centralized repositories. 1^st is IIS BITS will start uploading the PST to upload folder/centralized repositories from the user system where QMMPST Agent is installed. 2^nd Center upload Agent this will use SMB protocol to copy the PST file to upload folder/centralized repositories from network shared (will cover in detail during installation and configuration blog post).

Backup: It executes on QMMPST server. It uses Windows internal tools to make a copy for uploaded PST files to different folders. This is an options stage.

Extraction: It executes on QMMPST server, which Filters and extracts PST files to disk to enable streamlined processing and delivery to the target. During the extraction it repair corrupted PST files.

Content Scanning: This stage will check all Items and contents of PSTs to identify the ownership of the PST.

Ingestion: This stage will ingest all items and contents of owned PST to into the desired target.

Clean-up: This stage will perform the clean-up of upload folder/centralized repositories and the PST file from original source location.

Following flow diagram is self-explanatory on migration stages.