FabCon Europe in Stockholm this September was a resounding success, bringing together Microsoft Fabric enthusiasts and partners from across the continent. The event showcased the latest advancements in the platform and provided valuable insights for organizations looking to leverage Fabric for their data initiatives.

In this blog post, we’ll recap some of the key Microsoft Fabric announcements from FabCon Europe and highlight Bitwise’s Microsoft Fabric Readiness Program, designed to help organizations accelerate their adoption of this powerful data analytics platform.

3 Key Microsoft Fabric Announcements

1. Enhancements for Fabric Data Factory Pipelines

One of the most exciting announcements at FabCon Europe was the introduction of new enhancements for Fabric Data Factory pipelines. These enhancements aim to streamline data integration and transformation processes, making it easier for organizations to build and manage complex data pipelines. Some of the key features include:

• Improved performance: Enhanced pipeline execution speed and resource utilization.
• Enhanced monitoring and diagnostics: Improved tools for troubleshooting and optimizing pipeline performance.
• Expanded data source support: Support for additional data sources and formats.

2. Private Preview of Microsoft Fabric Capacity Calculator

Microsoft also announced the private preview of the Microsoft Fabric SKU Calculator that helps organizations estimate the required compute and storage resources for their Fabric workloads, ensuring they have the right capacity to meet their needs.

The calculator is designed to simplify the process of provisioning Fabric resources and optimize costs. With a straightforward interface, the calculator provides outputs on recommended Fabric SKUs for the infrastructure based on inputs.

3. DP-700 Exam for Data Engineering Using Fabric

For individuals looking to validate their skills in data engineering using Microsoft Fabric, Microsoft introduced the DP-700 exam. This beta exam, available from late October 2024, covers a wide range of topics related to data integration, transformation, warehousing, governance, and Fabric administration.

By passing the DP-700 exam, individuals can demonstrate their expertise in implementing data engineering solutions using Microsoft Fabric and open up new career opportunities, especially as data engineering skills become essential in the era of AI.

Microsoft Fabric Readiness Program from Bitwise

Bitwise is committed to helping organizations maximize the value of Microsoft Fabric. Our Microsoft Fabric Readiness Program offers a comprehensive suite of services designed to accelerate your adoption of the platform. These services include:

• Assessment and planning: We conduct a thorough assessment of your current data landscape and develop a tailored roadmap for Fabric adoption
• Migration and modernization: We assist with migrating your existing data workloads to Fabric and modernizing your data architecture.
• Training and enablement: We provide training and support to help your team become proficient in using Fabric.
• Ongoing support: We offer ongoing support to ensure the success of your Fabric implementation.

By partnering with Bitwise, you can benefit from our deep expertise in Fabric and our proven track record of delivering successful implementations. Discover how Microsoft Fabric can revolutionize your organization’s data management and analysis capabilities.

See You at FabCon in 2025

FabCon Europe was a valuable event for anyone interested in Microsoft Fabric. The key announcements highlighted the platform’s continued evolution and its ability to meet the growing demands of modern data management.

As a Microsoft Solutions Partner working closely with the Fabric product team to strengthen our capability to migrate data, ETL and BI workloads to Fabric, Bitwise is excited to be a key part of our customer’s transformation to unlock the full potential of Fabric.

Stay tuned to get more updates from Microsoft Fabric Community Conference and we look forward to seeing you in Las Vegas or Vienna in 2025!

The post 3 Key Microsoft Fabric Announcements from FabCon Europe   appeared first on Bitwise.

Challenges with the Legacy ETL Platforms

How did we get here? Over time, organizations have a build-up of legacy platforms, often through mergers and acquisitions, or bringing in different tools for specific use cases. Integration of other transformation assets, like SQL packages, adds to the mix.

Why is it a problem? With multiple tools come multiple licenses and exploding costs. The combination of legacy platforms and SQL packages results in complexity and presents challenges for governance and lineage. Also, legacy tools were not built for big data or cloud, which are now part of the ecosystem.

According to Dzone, The traditional ETL process is slow, time-consuming, and resource-intensive.

As traditional, on-premise enterprise data warehouses are moving to cloud data lake environments, organizations see a need for modern ETL platforms like Azure Data Factory and AWS Glue to meet their requirements.

Even if companies do not have multiple ETL tools, there is often a need to upgrade their old platform in order to take advantage of newer capabilities like self-service BI and real-time analytics, which can require migration effort. Businesses must evaluate their ETL procedures and consider switching tools as technology continues to advance. So, we will discuss the necessity of migrating from one ETL to another in 2023.

1. Enhanced features and functions:

The new “tool’s” enhanced capabilities and features are one of the primary reasons to switch from one ETL tool to another. Newer ETL tools with advanced features like cloud integration, real-time data processing, and AI/ML-powered data transformation have emerged as a result of technological advancements. Businesses may benefit from increased speed, accuracy, and efficiency as a result of these features.

2. Cost reduction:

Businesses may be able to save money by switching to a new ETL tool. Support and maintenance costs for older ETL tools may be significant, which can add up over time. Additionally, more recent ETL tools frequently provide more adaptable pricing options, such as subscription-based or pay-per-use pricing, which may prove to be more cost-effective in the long run.

3. Scalability:

Data processing requirements rise with business expansion. Businesses can better manage their expanding data requirements by migrating to a new ETL tool with scalability. Modern ETL services are intended to deal with large volumes of information and can scale processing resources up or down as needed.

4. Compliance and security enhancements:

In this digital age, data security and privacy are of the utmost importance. It’s possible that older ETL tools lack the necessary security features to safeguard sensitive data. Businesses can ensure that their data complies with GDPR, CCPA, and HIPAA regulations by migrating to a new cloud-native ETL tool.

5. Increased productivity:

With drag-and-drop interfaces and pre-built connectors, newer ETL tools are designed to be easier to use. Improved productivity and streamlined data integration procedures are two potential outcomes of this. Furthermore, newer ETL tools frequently offer better automation capabilities, decreasing the requirement for manual “intervention” and opening up assets for different undertakings.

The post 5 reasons to migrate one ETL to another ETL in 2025 appeared first on Bitwise.

1) Does Migrating the Data Integration or ETL Platform to Something Different Add Value?

When looking at migrating your Data Integration or ETL platform to another, you need to really look at the WHY behind it.

It’s not just the platform that adds or takes away value, it’s the environment it operates in that influences the platform largely.

It could be that the custom legacy code to update four different databases is insufficient to handle your increased data warehouse (DW) & business intelligence (BI) needs. Or you might be missing new platform feature sets like Data Profiling, Data Quality, Advanced Scheduling Features, etc. For example, the value for one Fortune 500 client was derived from being able to Eliminate out the business rules from the operational batch which were tightly coupled and thereby less adaptable to changing the business strategy.

The vendor licensing strategy for the platform and associated features it provides may be less than adequate over the course of time which forces you to reevaluate the impact on your departmental budget and productivity.

In any case, migrating your ETL system to a different platform is like running a marathon. In the short term, there is a distinct possibility that the migration itself costs more in terms of time and effort than the license agreement with the existing vendor for another year but over the long term would be more than worth cost borne.

The platform you pick needs careful consideration of not only the licensing costs but the Developer learning curve and other feature considerations like Data Profiling, Data Lineage, Data Quality, Scheduling, Source Control, connectors to Big Data platforms, etc. From an administration standpoint, vendor support, Ease of Maintainability and Ease of Administration Eliminate can the ability to scale up in terms of supporting the increasing demands in quantum of processing without any performance degrades. And having a good debate with your architect on open source and proprietary would come in handy.

2) How to derive the most value?

There was a classic comment on the Windows XP migration blog, “If you are thinking about moving your enterprise off Windows XP NOW- you are already too late.” This was April 2013 when there was a year still left for MS to support XP.

Very often, licensing agreements run till the end of the calendar year which coincides with the reporting and real-time information needs of the business. If the operations or technology people cannot guarantee the up-time of the warehouse and the relevance of its data, the business folks cannot depend on its predictive analysis and risk assessment of marketing or sales strategies.

Timing your migration with a view and in cooperation with the business team is critical to ensuring the success of your migration.

Moving from one ETL platform to another is a time-consuming process that needs thorough planning and execution. At Bitwise, some of the critical migrations we have been involved with have ranged from 18 man-months to 240 man-months. What can help reduce the effort is building platform reassessment into the life cycles of the applications using the tools and planning moving across vendors based on these reassessments.

Adding value by cutting costs (and not corners) is the way forward for today’s lean enterprises. This is challenging the set paradigms for design and architecture to offer solutions that don’t necessarily reinvent the wheel – but do away with it altogether.

It is forcing stakeholders like analysts, modelers, and design and development teams to leave the silos they work in and come up with an integrated approach. Like the separation of critical business SLAs on proprietary products and leaving non-critical functions on open source. The integration of the two then takes on its own criticality and needs to be treated with the same respect the SLA-linked business functions get.

3) How Do I Curb My Costs?

Before jumping on any new platform it is essential to run a scaled-down version of your enterprise application in a Proof of Concept.

Furthermore, it would be a good idea to break down the application into segments that need to be migrated vs. elements that can easily or seamlessly be ported over to the new platform. Scripts that are used by the Enterprise-authorized batch scheduler and XML Metadata that can be parsed to generate Source to Target mappings are a few of the examples of elements that are easily ported. However, custom scripts that are specific to administration or managing the execution of the product in your environment do not port easily and are better redone from scratch.

Metadata management and Data Lineage tracking is a key elements of any enterprise Data Integration/ETL solution and the most appropriate platform for your enterprise should be including this as a focused offering. Besides the help it provides the Analysts, having effective data lineage in place reduces the time required for setting up test data for the migration team.

As is with every critical system, budget for extensive testing and data validation while also allowing parallel execution of the old and the new systems with the same input data sets for Production validation.

To efficiently get this done, it may be a good idea to seek out tools/services that can automate such migration and associated testing from source to The target platform. Having a data validation suite in place that can be used by both the migration team as well as the end business users helps reduce turnaround time for the UAT.This would save the efforts of manual conversion and bring in efficiencies both from a cost and Time-to-market perspectives. Bitwise’s ETL Converter is a good candidate for such an automated migration service offering which can serve as your starting point.

And, with a view of assessing the platform you are migrating to in a couple of years down the line, ensure the thought process behind a particular course of action is as well documented as the action itself.

To Summarize

Migrating your Data Integration/ETL platform to another can always add value to your enterprise if assessed, planned, and executed.

A successful data migration project needs the team armed with a methodology, a proven migration automation tool/service with migration test automation. Connect with us if you want ballpark estimates to get you started on your migration project.

The post 3 Essentials For Migrating Your Data Integration (ETL) Platform appeared first on Bitwise.