Data migration has historically been a lengthy and risky process. Specialists would have to manually identify and correct issues in the source data—like data loss, errors, and mis-mapping—on top of formatting templates and implementing selection criteria. As a result, the entire data migration process, especially in the crucial early stages, would be incredibly time-consuming. DATAMA—Absoft’s cutting-edge data migration tool—aims to negate these issues.Â
DATAMA is a flexible and scalable ETL tool that accommodates projects and migrations of many timeframes for companies of various sizes. By merging Python with SQL, DATAMA allows more flexibility when migrating data from an existing system to a new one – SAP to SAP or otherwise. Using SQL, Absoft’s data migration specialists can reuse existing reports for data migration. This allows DATAMA to automate many parts of the process, de-risking instances of human error and saving time. Equipped with DATAMA, Absoft’s team has completed full data migrations in as little as six weeks.Â
In this blog post, Absoft will walk you through the data migration process step-by-step and highlight where and how DATAMA simplifies it.Â
Don’t hesitate to get in touch if you require any help or have a question.Â
Table of Contents
Extraction
In the first step of the data migration process, a user will use DATAMA to determine how best to extract data from the source system. They will ascertain the correct permissions and access for that source system. These permissions are received directly from SAP for an SAP-to-SAP migration.
A clear and comprehensive understanding of the differences between the source and target systems is indispensable. Case in point, ECC and S/4HANA set up ‘Business Partners’ very differently. Identifying differences such as these early can save time in the future.
The first part of the extraction is to figure out what data the source system holds and extract it. There are a handful of ways a user can use DATAMA to automate the extraction. Perhaps the most effective way is to apply a rule set they wish to utilise beforehand to reduce the overall amount of data they need to extract. Depending on the exact criteria, this may only extract data from the past two years rather than the system’s entire lifetime. The user should create a copy of all the available data if possible. This will act as a backup for later in the process to refer to and in case they need to add extra tables, for example.
When workshopping to configure the new system early in the explore phase, Absoft’s data migration specialists will extract all the source system’s data on the same day and freeze it from that point in time. All the data used for the rest of the process will always work from that day. They do this to ensure consistency within the data and perform it early in the process.
Field Mapping and Selection Criteria
After the extraction, the user will train DATAMA on how to map the data to the target system. No two systems categorise data in the same way—for instance, plant maintenance in SAP and Maximo work very differently. Therefore, the user must ensure accurate data transfers to the correct locations between the two systems. Where a field does not exist, the user will capture that data from the source system and create a new field in the target one.
Next, the user enters their selection criteria into DATAMA to refine what data will be transferred. When undertaking these migrations, avoiding selecting data that is no longer relevant or useful to the organisation is important. An old enough system and a company with many clients are bound to have outdated and irrelevant data. Excluding this data from the migration process will keep the new system clean. The organisation and the partner will jointly decide which data types and timeframes to select for the migration. Typically, the partner advises only migrating the most pertinent information and not moving historical data. For example, the user will select open invoices from the past year, but will not select completed invoices from several years ago.
The quality of data maintenance in the previous system largely determines this. To circumvent any issues in accidentally duplicated information, DATAMA has a de-duplication function. This allows it to recognise the same data presented in different formats and synthesise them.
At this stage, the user will produce a cleansing report to identify potential issues that might arise later in the process, such as disparities in data input and presentation. For instance, the old system may have a standard short description length of 60 characters vs 40 in the new one. DATAMA can assist in highlighting these differences. Producing this report early aims to identify and resolve potential problems swiftly.
Transformation
Once the functional team and the client review the newly configured system, they will make necessary changes to align with their business needs. At this point, the user will know how the target system works and can produce an initial transformation as a test fit. From here, the user will implement any relevant modifications. This can entail adding custom company codes, object types, and other classifications from the previous system into the new one.
It also entails translating the data into a form that suits how the new system thinks and operates. For instance, the user may need to merge data organised into 10 categories into three in the new system (or vice versa). DATAMA can synthesise the data into those three categories, and the user can choose what transformations the tool should apply. Once the user has written these transformations, DATAMA carries out this process automatically. This mitigates the risk of manual errors from inputting them incorrectly or forgetting to enter and delete values. The only manual parts of this process are when the user reviews the logic and the mappings.Â
The user will give the client a sheet containing all the transformations to the client for review and approval. This outline of the original fields helps the client understand the new data. In cases where the customer is unfamiliar with SAP, Absoft’s migration specialists assist them in adjusting to the new system.Â
Load
Users can download templates from the SAP Data Migration Cockpit directly from the Fiori app. DATAMA will populate these templates with the data they have extracted from the source system.
The next step in the Migration Cockpit is the preload validation. The user executes this step to identify any issues with the newly transformed data set, such as duplicate fields, incorrect dates, and character counts. This is necessary to guarantee that the data meets the system’s requirements so the user doesn’t need to correct these errors later.Â
The SAP Data Migration Cockpit requires mapping from the template into the new system. Absoft has set up DATAMA to apply all the necessary transformations, so within the cockpit, the mappings will be one-to-one (dates to dates, price to price, hours to hours, etc). As the cockpit’s mapping function has limited scope, the user will ensure the Load Template matches the loaded data in the target SAP system and is validated. This guarantees that no audits require additional mapping files.
Simulation and Final Checks
Finally, the user will run a migration simulation in the SAP Data Migration Cockpit to identify any issues before the load can occur. Â
If the simulation passes, the load can then proceed as normal. If it fails, the system will generate error codes to help identify the issues in the dataset. The user extracts and applies changes to the data and repeats the process using a new template or a delta file.
Once the data has been loaded into the source system, the user extracts and loads it into DATAMA. Here, it compares the data side-by-side to determine if that data has been loaded correctly.
In non-public Cloud environments, SAP LSMWs (SAP Legacy System Migration Workbench) can assist in data transfer. The user fully builds this before providing it to DATAMA, allowing all the steps to occur in one transaction.
Learn more about how the Absoft team completed the process of asset acquisition, with seamless data migration, in less than seven weeks for Petrogas
Conclusion
As is evident, the overall migration process remains largely unchanged and immediately familiar to any specialist. The key difference is that DATAMA automates templates’ populating, formatting, and mapping—the SAP Data Migration Cockpit will take care of the rest. By automating these processes, Absoft’s team has avoided many common challenges during data migrations. This includes incorrectly inputting values, mis-mapping, and data loss, while simultaneously saving time.Â
To learn more about the Absoft team’s experience, tools, and capabilities, please reach out now!
