Code Conversion Excelerator Archive - Bitwise https://www.bitwiseglobal.com/en-us/blog/tag/code-conversion-excelerator/ Technology Consulting and Data Management Services Mon, 24 Apr 2023 05:15:06 +0000 en-US hourly 1 https://cdn2.bitwiseglobal.com/bwglobalprod-cdn/2022/12/cropped-cropped-bitwise-favicon-32x32.png Code Conversion Excelerator Archive - Bitwise https://www.bitwiseglobal.com/en-us/blog/tag/code-conversion-excelerator/ 32 32 3 Essentials For Migrating Your Data Integration (ETL) Platform https://www.bitwiseglobal.com/en-us/blog/3-essentials-for-migrating-your-data-integration-etl-platform/ https://www.bitwiseglobal.com/en-us/blog/3-essentials-for-migrating-your-data-integration-etl-platform/#respond Mon, 24 Aug 2015 14:05:00 +0000 https://www.bitwiseglobal.com/en-us/3-essentials-for-migrating-your-data-integration-etl-platform/ 1) Does Migrating the Data Integration or ETL Platform to Something Different Add Value? When looking at migrating your Data Integration or ETL platform to another, you need to really look at the WHY behind it. It’s not just the platform that adds or takes away value, it’s the environment it operates in that influences ... Read more

The post 3 Essentials For Migrating Your Data Integration (ETL) Platform appeared first on Bitwise.

]]>

1) Does Migrating the Data Integration or ETL Platform to Something Different Add Value?

When looking at migrating your Data Integration or ETL platform to another, you need to really look at the WHY behind it.

It’s not just the platform that adds or takes away value, it’s the environment it operates in that influences the platform largely.

It could be that the custom legacy code to update four different databases is insufficient to handle your increased data warehouse (DW) & business intelligence (BI) needs. Or you might be missing new platform feature sets like Data Profiling, Data Quality, Advanced Scheduling Features, etc. For example, the value for one Fortune 500 client was derived from being able to Eliminate out the business rules from the operational batch which were tightly coupled and thereby less adaptable to changing the business strategy.

The vendor licensing strategy for the platform and associated features it provides may be less than adequate over the course of time which forces you to reevaluate the impact on your departmental budget and productivity.

In any case, migrating your ETL system to a different platform is like running a marathon. In the short term, there is a distinct possibility that the migration itself costs more in terms of time and effort than the license agreement with the existing vendor for another year but over the long term would be more than worth cost borne.

The platform you pick needs careful consideration of not only the licensing costs but the Developer learning curve and other feature considerations like Data Profiling, Data Lineage, Data Quality, Scheduling, Source Control, connectors to Big Data platforms, etc. From an administration standpoint, vendor support, Ease of Maintainability and Ease of Administration Eliminate can the ability to scale up in terms of supporting the increasing demands in quantum of processing without any performance degrades. And having a good debate with your architect on open source and proprietary would come in handy.

2) How to derive the most value?

etl-help-2

There was a classic comment on the Windows XP migration blog, “If you are thinking about moving your enterprise off Windows XP NOW- you are already too late.” This was April 2013 when there was a year still left for MS to support XP.

Very often, licensing agreements run till the end of the calendar year which coincides with the reporting and real-time information needs of the business. If the operations or technology people cannot guarantee the up-time of the warehouse and the relevance of its data, the business folks cannot depend on its predictive analysis and risk assessment of marketing or sales strategies.

Timing your migration with a view and in cooperation with the business team is critical to ensuring the success of your migration.

Moving from one ETL platform to another is a time-consuming process that needs thorough planning and execution. At Bitwise, some of the critical migrations we have been involved with have ranged from 18 man-months to 240 man-months. What can help reduce the effort is building platform reassessment into the life cycles of the applications using the tools and planning moving across vendors based on these reassessments.

Adding value by cutting costs (and not corners) is the way forward for today’s lean enterprises. This is challenging the set paradigms for design and architecture to offer solutions that don’t necessarily reinvent the wheel – but do away with it altogether.

It is forcing stakeholders like analysts, modelers, and design and development teams to leave the silos they work in and come up with an integrated approach. Like the separation of critical business SLAs on proprietary products and leaving non-critical functions on open source. The integration of the two then takes on its own criticality and needs to be treated with the same respect the SLA-linked business functions get.

3) How Do I Curb My Costs?

etl-help-3

Before jumping on any new platform it is essential to run a scaled-down version of your enterprise application in a Proof of Concept.

Furthermore, it would be a good idea to break down the application into segments that need to be migrated vs. elements that can easily or seamlessly be ported over to the new platform. Scripts that are used by the Enterprise-authorized batch scheduler and XML Metadata that can be parsed to generate Source to Target mappings are a few of the examples of elements that are easily ported. However, custom scripts that are specific to administration or managing the execution of the product in your environment do not port easily and are better redone from scratch.

Metadata management and Data Lineage tracking is a key elements of any enterprise Data Integration/ETL solution and the most appropriate platform for your enterprise should be including this as a focused offering. Besides the help it provides the Analysts, having effective data lineage in place reduces the time required for setting up test data for the migration team.

As is with every critical system, budget for extensive testing and data validation while also allowing parallel execution of the old and the new systems with the same input data sets for Production validation.

To efficiently get this done, it may be a good idea to seek out tools/services that can automate such migration and associated testing from source to The target platform. Having a data validation suite in place that can be used by both the migration team as well as the end business users helps reduce turnaround time for the UAT.This would save the efforts of manual conversion and bring in efficiencies both from a cost and Time-to-market perspectives. Bitwise’s ETL Converter is a good candidate for such an automated migration service offering which can serve as your starting point.

And, with a view of assessing the platform you are migrating to in a couple of years down the line, ensure the thought process behind a particular course of action is as well documented as the action itself.

To Summarize

Migrating your Data Integration/ETL platform to another can always add value to your enterprise if assessed, planned, and executed.

A successful data migration project needs the team armed with a methodology, a proven migration automation tool/service with migration test automation. Connect with us if you want ballpark estimates to get you started on your migration project.

The post 3 Essentials For Migrating Your Data Integration (ETL) Platform appeared first on Bitwise.

]]>
https://www.bitwiseglobal.com/en-us/blog/3-essentials-for-migrating-your-data-integration-etl-platform/feed/ 0
Modernization – The New Mantra For Legacy Migration https://www.bitwiseglobal.com/en-us/blog/modernization-the-new-mantra-for-legacy-migration/ https://www.bitwiseglobal.com/en-us/blog/modernization-the-new-mantra-for-legacy-migration/#respond Mon, 24 Aug 2015 13:51:00 +0000 https://www.bitwiseglobal.com/en-us/modernization-the-new-mantra-for-legacy-migration/ Legacy languages are hard to support The legacy languages and development tools needed to support the legacy system are increasingly difficult or expensive to obtain. This is a very common occurrence with 4GLs popular in the 1970s. People are scarce People that know the legacy languages are becoming difficult to find and retain. Younger staff ... Read more

The post Modernization – The New Mantra For Legacy Migration appeared first on Bitwise.

]]>

Legacy languages are hard to support

The legacy languages and development tools needed to support the legacy system are increasingly difficult or expensive to obtain. This is a very common occurrence with 4GLs popular in the 1970s.

People are scarce

People that know the legacy languages are becoming difficult to find and retain. Younger staff are reluctant to learn “legacy” languages because it does not appear to advance their long-term career.

The underlying platform is hard to support

Many legacy systems run on legacy hardware systems. Such hardware systems are becoming more expensive to maintain, and personnel that know these systems are also more difficult to find.

Legacy software does not integrate well with other IT systems

The architecture of legacy languages often does not lend itself to building bridges to other IT systems that have grown up around it.

Challenges

Migrating large software applications needs automation. The need to migrate mainframe software is often motivated by a mixture of motives including:

  • The high cost of operating the mainframe
  • The shrinking pool of IT personnel that understand the mainframe system languages and structure
  • Need for faster application evolution in response to changing requirements by use of more modern software engineering methods
  • The need to integrate the application and its data more effectively with the balance of the organization

Yet most of the legacy software performs an extremely valuable function that must not be disturbed by a migration. Hand-migrating large applications by re-coding from scratch usually causes huge management overheads when the migration team attempts to re-validate the requirements with management. Hand-migrating by translating line for line introduces errors by damaging implicit business rules, and suffers from scope creep. In addition, there is a continual war between the application maintenance staff, which must keep the application running during the migration process, and the migration staff, which does not want to see any change during the lengthy migration process. As a consequence, many manual migrations fail or run significantly over time and cost budgets. Offshore hand-migration suffers the same problems as well as communication and coordination problems.

Strategies/Planning

Many organizations face the problem of having to modernize their existing software systems by migrating to more capable systems. Modernizing legacy software systems is a complex engineering problem that includes most aspects of traditional software development with more constraints. A successful migration effort requires both a sound development plan and a sound migration plan.

blog-strategies

A development plan addresses the selection of the appropriate software processes, methods, tools, and hardware and software platforms. A migration plan created in conjunction with the development plan is necessary to help ensure that the operational transition to the new system goes smoothly.

A good migration plan should weigh programmatic and technical drivers for system development against customer priorities. Because of this, the plan may impact system development and certainly should impact system deployment. Iteration among the key stakeholders is necessary for an effective migration effort. Like development, migration planning involves tradeoffs among cost, schedule, risk, and resources.

Legacy system modernization is often a large, multiyear project. Because these legacy systems are often critical in the operations of most enterprises, deploying the modernized system all at once introduces an unacceptable level of operational risk. As a result, legacy systems are typically modernized incrementally. Initially, the system consists completely of legacy code. As each increment is completed, the percentage of legacy code decreases. Eventually, the system is completely modernized. A migration strategy must ensure that the system remains fully functional during the modernization effort.

Making of software modernization decisions is a process within some organizational context. “Real-world” Decision making in business organizations often has to be made based on “bounded rationality”. Besides that there exists multiple (and possibly conflicting) decision criteria, the certainty, completeness, and availability of useful information (as a basis for the decision) is often limited.

blog-strategies2

Conclusion

Legacy systems are still alive because of their distinct characteristics and good pedigree. In the last 40 years, we have learned that it is neither practical nor affordable to migrate $5 trillion worth of legacy code into other technologies which were short-lived. However, it is possible to either eliminate or integrate legacy systems by following an effective migration strategies and appropriate migration tools.

The post Modernization – The New Mantra For Legacy Migration appeared first on Bitwise.

]]>
https://www.bitwiseglobal.com/en-us/blog/modernization-the-new-mantra-for-legacy-migration/feed/ 0