Disclaimer: The articles in this blog post are those that I found interesting and relevant to the topic of ERP and technology in general. I have no commercial association with any of the entities mentioned in this article. I may be following a few of these entities on LinkedIn and even some of these entities may be on my LinkedIn or Social Media network. These articles are selected purely based on their relevance to the objective of this blog, to promote ERP. Finally, the summary is mine. While I stay close to the points in the articles, I also elaborate a few of them based on my understanding.
The Short URL for this post is http://bit.ly/ERP_Data_Migration
The steps in ERP Implementation are:
5. ERP Implementation: ERP Project Management
6. ERP Implementation: Requirements Gathering
7. ERP Implementation: Solution Design
8. ERP Implementation: Solution Testing
9. ERP Implementation: End user training
10. ERP Implementation: RICE
11. ERP Implementation: Integration
12. ERP Implementation: Data Migration / Conversion
6. ERP Implementation: Requirements Gathering
7. ERP Implementation: Solution Design
8. ERP Implementation: Solution Testing
9. ERP Implementation: End user training
10. ERP Implementation: RICE
11. ERP Implementation: Integration
12. ERP Implementation: Data Migration / Conversion
13. Post Implementation Support and Stabilization
This week's topic is Data Migration. I believe that Data Migration and Configuration are the two most important activities in ERP Implementation. The first article compares two different approaches to migration, migrating transaction data or tables. The articles explains situations where each approach can be used and the advantages and disadvantages of each method. Second article by Scott A Holter discusses five keys to successful data migration. These include Multiple Iterations, loading only open transactions etc. The third article from ERCERP blog discusses 8 steps to a successful data conversion.
The fourth article gives 14 best practices to be used in data migration. This article explains a 'Tailored' Microsoft Sure step method towards data migration. The fifth article that I have considered is a White paper from Oracle. This excellent document provides detailed information on data migration. I suggest that every ERP consultant downloads and reads this document before planning their data migration.
Data migration is my favorite topic. You can be sure that I reviewed these articles with a hawk's eye before finding them worthy of being in this blog post.
Do read on... Tell me what you think.
Top 5 Articles
Top 5 Articles
1. ERP Data migration: Migration of transactions or tables? Drs A Biewenga and R Akca for Compact.nl blog
https://www.compact.nl/articles/erp-data-migration/
Data migration in ERP can be done in two ways. Transaction Driven Approach and Table Driven Approaches.
Transaction driven is the common approach, in this open data is extracted from legacy systems and uses the ERP systems processes to load the data in ERP. The advantages are one, less migration data volume and two, the organization can use the new features of ERP from day one. Other advantages are that it gives organization an opportunity to review the legacy data and identify any data mismatches as well as uses the integrity checks embedded in the ERP application as data moves into it.
The disadvantage is that historical data has to be maintained separately based on compliance requirements.
Since this approach calls for data validation, it is important to start the data identification process early. You have to identify open transaction data, open masters data and open submasters data. All of these flow from the first one. Open transaction data is the set of transactions that are in process on the date of implementation. It is important to decide on the definition of open transaction data at the onset of the implementation and progressively identify the same as project progresses.
In one of my implementation, for creditors we decided that we will take only open invoices. Which meant that payments be expedited for all the pending receiving transactions. As we neared the implementation, payment to vendors was stopped. Vendors were informed about 20 days in prior to go live that their payments will be released only after go live.
In Table Driven approach, entire legacy table is moved to the target system. This approach is very rare. The advantage is that historical data is available in the ERP application from day one. The disadvantages are one, higher data volume leading to higher migration time, process improvements start only after data migration activity is completed and the in-built validations of the target system are not used and all validations have to be built in manually. This approach also increases the migration costs. Given all these disadvantages, this approach is only done in two cases, one, source and target systems have similar data models and two, if the company requires the entire historical data to be moved to the new ERP system.
Data migration can cost between 15-25% of project budget. Other cost drivers include Setup logic, Conversion Logic, Reconciliation scripts and Testing costs.
Nice article. I had not heard of Table Migration approach. Well, one lives and learns...
2. Five keys to a successful ERP Data Migration: Scott A Holter for Meadenmoore.com blog
https://blog.meadenmoore.com/blog/erp/is-data-conversion-and-migration-an-erp-implementation-projects-black-hole
The five keys to successful data migration are:
1. Multiple iterations: The advantages are you become more familiar with your legacy data as well as the new features of the ERP System. Often you see the design and approach change after each iteration.
2. Do not bring historical data into the new ERP system: House the legacy data in a separate database with a 'need to access' basis. This reduces the data volume that is coming in to the ERP system.
3. Scrub your data: Review and remove redundant data. As the previous article says, only bring in items, customers and suppliers that are associated with the open transaction data.
4. Import vs. manual entry: Always opt for manual data entry where the data volume is low. This will familiarize the users with the application that they are going to use on a daily basis. (This is an
innovative idea)
5. Test thoroughly: Ensure to do both unit and system testing. Ensure to test validations and exceptions. Deliberately create errors to see how system handles them. Document the test results thoroughly.
Good article.....
3. Converting data from legacy system to ERP: ERCERP Blog
https://ercerp.wordpress.com/converting-data-from-a-legacy-system-to-erp/
The steps in data conversion are:
1. Types of data to be converted:
- Master data like customer master, vendor master etc,
- Transaction data like open supplier invoices, open customer invoices etc
- Sub-master data like Unit of Measure, Payment Terms etc.
- Historical data like sales analysis etc
2. Identify the data to be converted: You could decide to load only the open data or the complete data including closed, historical data. Both these approaches have their own challenges. It is recommended not to load historical data.
3. Data preparation: Process owners should be the owners of data as well
4. Conversion timing and sequencing: Normally Sub-masters data is loaded first followed by master data and finally the transaction data. While the first one can be loaded any time, master data is loaded about 15 days prior to Go Live and Transaction data is migrated just before go live.
5. Data cleansing: This is important because some of the data may:
- Not be relevant
- Be meaningless
- Have errors
- Be 'Fudged'
- Be in a different format than is expected in ERP. Mostly numbers are entered in 'Text' format in many legacy systems for example
6. Testing: For data volume, data accuracy and matching control totals. Testing helps verify that data load is complete and correct
7. Change management: This is an important part of data migration. If the customer has confidence in the data in the new ERP system, the ERP buy-in will significantly improve.
8. Training: Do not underestimate training. There is nothing like 'sufficient' training. Not sure what is the relevance of training to data migration though.
Nice article.
4. 14 data migration best practices for successful ERP projects: to-increase.com blog
https://www.to-increase.com/business-integration/blog/14-data-migration-best-practices-for-successful-erp-projects
This blog from the consulting firm To Increase explains how they have tailored the Microsoft 'Sure Step' methodology to ensure successful data migrations. The fourteen best practices are divided into four stages:
- Analyze
- Design
- Deploy
- Operation
- Define the scope of the project and identify the applications that you want to migrate
- Determine the scope of the data.
- Prepare test documentation
- Identify the migration team. The company has created a proprietory 'Team Step' methodology to assign the team
- Define data mapping
- Keep the data model simple. This makes it easier to trace errors and quickly update data.
- Ensure to store the data files in a single folder. Use naming conventions to specify file names for each migration
- Create migration templates for each company that you migrate to.
- Have a standalone migration application. This company has an application called 'To Increase Migration Studio' to design and develop migrations.
- Use one project definition with one set of files for each data migration project.
- Use a stable set of data to migrate
- Ensure to establish data dependency so that you have the data available before they are used in downstream migrations. For example, ensure that vendor is set up before using the same as a preferred supplier in Item Master
- Speed up the process by running multiple migrations in parallel
- During migration validate data and apply all business rules.
5. Successful Data Migration (downloadable pdf): Oracle.com whitepaper
https://www.oracle.com/technetwork/middleware/oedq/successful-data-migration-wp-1555708.pdf
In writing this blog series, I am always looking for white papers and best practices document, that adds value to me as well as to the readers of this blog. This white paper from Oracle is one such excellent document that gives a great perspective to data migration.
The white paper is divided into four Parts, Importance of Data Quality, Formulating a strategy, Discovering your data and Essential steps to success. It covers the entire gamut from identifying the data to be migrated to post migration regular maintenance.
Accurate data is the raw material that maximizes the value of enterprise applications. A data that is sufficient for the legacy system may be inaccurate for the new system. Transferring data into a more sophisticated application will amplify the negative impact of any incorrect or irrelevant data, perpetuate any hidden legacy problems, and increase exposure to risk. More often than not, the difficulty of migration is underestimated.
The key drivers of data complexity could be Corporate growth, Compliance, Data volume, Data diversity (data in new formats like SMS, RFID etc), Data decay etc.
There are two migration strategies, big bang and trickle migration. Big bang migrations involve completing the entire migration in a small, defined processing window. Trickle migrations take an incremental approach to migrating data. Rather than aim to complete the whole event in a short time window, a trickle migration involves running the old and new systems in parallel and migrating the data in phases. In this approach, it must be possible to track which data has been migrated. In addition, both legacy and new system will run in parallel adding complexity and risk.
Some of the pitfalls in normal data migration are validating with minimal amount of data, focusing more on metadata and less on content leading to erroneous data being migrated, lack of information about the source system, lack of knowledge about target system features, inadequate testing etc.
A successful migration starts with a clear understanding of the source data. The data has to be validated based on factors including relevance, accuracy, consistency, integrity and completeness. The six phases are planning, understanding of data, designing (segment the data migration into small increments with continuous data validation), executing, testing and verifying and follow up data maintenance.
Nice article. Do read on for more on data migration
I learned a lot from reading this document. I loved its structured approach to data migration.
Additional Reading
1. Five steps towards successful ERP Data Migration: Processproerp.com blog
https://www.processproerp.com/blog/take-these-5-steps-toward-successful-erp-data-migration
No comments:
Post a Comment