Hemant Belwalkar, Author at Bitwise Technology Consulting and Data Management Services Wed, 31 Jul 2024 09:07:39 +0000 en-US hourly 1 https://cdn2.bitwiseglobal.com/bwglobalprod-cdn/2022/12/cropped-cropped-bitwise-favicon-32x32.png Hemant Belwalkar, Author at Bitwise 32 32 5 Use Cases for Driving ROI with Machine Learning https://www.bitwiseglobal.com/en-us/blog/5-use-cases-for-driving-roi-with-machine-learning/ https://www.bitwiseglobal.com/en-us/blog/5-use-cases-for-driving-roi-with-machine-learning/#respond Thu, 18 Jul 2024 10:28:57 +0000 https://www.bitwiseglobal.com/en-us/?p=48515 Machine Learning Use Case Development The race is on for companies of all sizes and industries to deliver impactful artificial intelligence and machine learning applications. Fueled by the hype and fear of being left behind, many business leaders are throwing money and resources at implementing AI/ML use cases without having a clear understanding of a ... Read more

The post 5 Use Cases for Driving ROI with Machine Learning appeared first on Bitwise.

]]>
Machine Learning Use Case Development

The race is on for companies of all sizes and industries to deliver impactful artificial intelligence and machine learning applications. Fueled by the hype and fear of being left behind, many business leaders are throwing money and resources at implementing AI/ML use cases without having a clear understanding of a specific business problem to solve or business outcome to achieve. Having successfully implemented machine learning solutions at a wide range of customers, Bitwise has seen first-hand the importance of identifying the right use case and building a strategy around identified success metrics to achieve a positive ROI with AI/ML.

Let’s look at five use cases for driving ROI with machine learning, including success metrics, to help identify the right use case for you to start with AI/ML in your organization.

Identifying the Right Use Case for Machine Learning

Finding the right use case to get started sounds easy enough but can often be a challenge, especially with all the noise around the transformational nature of AI/ML technology. To cut through the noise, Bitwise recommends to start small and identify a pain area or gap in your business processes. Even a simple use case can achieve a meaningful ROI, which will provide a strong foundation to build your internal competency and stakeholder buy-in to develop more use cases.

Some of the early use cases like recommendation engines and reducing customer attrition, for example solving merchant attrition using machine learning in the financial industry, have been widely adopted and continue to deliver results. The below use cases provide other examples for implementing machine learning with associated success metrics to help you get the ball rolling.

5 Machine Learning ROI Use Cases

1. Billing Optimization

Complex billing processes can be difficult to manage and provide a poor customer experience. Optimizing billing with machine learning can streamline the process and improve efficiencies and accuracy.

  • Cost Savings: 15% reduction in operational costs. This metric reflects the reduction in overall operational costs achieved through streamlined billing processes, directly enhancing profitability.
  • Error Rate: Decrease from 2.1% to 0.5%. This measures the reduction in billing discrepancies, ensuring accuracy and enhancing customer trust.
  • Cycle Time: Reduced from 30 days to 18 days. This tracks the time required to complete the billing process from initiation to closure, facilitating quicker revenue recognition and improved cash flow.

2. User Support Chatbot

User support is a critical business process that typically requires heavy human interaction. Using machine learning to optimize the support process can help support staff be more efficient and provide a better customer experience.

  • Resolution Rate: 85% of queries resolved without human intervention. This metric indicates the proportion of queries resolved by the chatbot augmenting human interaction, showcasing the efficiency and reliability of automated systems.
  • Customer Satisfaction Score (CSAT): Increased up to 90%. This score assesses customer approval of interactions, reflecting commitment to providing seamless support.
  • Average Handling Time: Reduced to 3 minutes per interaction. This measures the time it takes to handle inquiries, demonstrating the speed and efficiency of augmented customer support.
  • Service Efficiency: Optimized support workforce for critical customer support, reducing support cost by 40%.

3. Predicting Wait Times

Industries like Hospitality and Heathcare are often plagued by wait times for their customers. Using machine learning to predict wait times for patrons and patients can provide a competitive edge for organizations where customer service is critical.

  • Accuracy of Predictions: up to 95% accuracy level. This metric evaluates the precision of predicted wait times against actual experiences to ensure accurate expectations.
  • Customer Satisfaction: Improved by up to 20%. This tracks improvements in customer or patient satisfaction, which correlate with accurate wait time predictions and efficient service delivery.
  • Service Efficiency: 30% increase in patient throughput. This assesses how effectively wait time predictions contribute to optimizing workforce allocation and enhancing service speed.

For example, Bitwise helped a national restaurant chain with predicting order wait time using machine learning to provide an improved experience for their online and dine-in customers by showing how long they will have to wait for their order to be served with 87% accuracy.

4. Pricing Optimization

In today’s omnichannel digital landscape, customers have a variety of options to purchase goods and services. Optimizing pricing with machine learning can provide a competitive advantage while maximizing margins.

  • Revenue Growth: 10% increase in quarterly revenue. This metric monitors the increase in revenue attributable to strategic pricing decisions, underlining the effectiveness of pricing models.
  • Price Elasticity: Sales volume changes by 5% for every 1% price adjustment. This measures how sensitive customer purchase volumes are to changes in pricing, helping to tailor pricing strategies based on consumer behavior.

5. Competitor Analysis

Companies have a trove of data that can lead to valuable insights when asking the right questions. Analyzing the competition using machine learning can help optimize marketing and customer acquisition efforts.

  • Customer Acquisition Cost (CAC): Reduced by 12%. This measures the efficiency of the customer acquisition initiatives, especially those driven by insights from competitor analysis, optimizing marketing spend and improving return on investment.

Identify Your AI/ML Use Case to Drive ROI

The examples of billing optimization, user support chatbot, predicting wait times, pricing optimization, and competitor analysis showcase a variety of machine learning use cases but by no means cover the full gambit of what’s possible with AI/ML.

Looking internally at business processes and pain areas can be a great place to start identifying your AI/ML use case with a focus on driving ROI. For example, we’ve found that for some of our customers, errors with data pipelines and data inconsistencies caused problems impacting their critical applications and resulted in cost increases and revenue losses. By implementing proactive monitoring of data platform using machine learning, potential problems can be prevented and avoid costly delays in time-to-production.

Ready to take the next step? Let’s talk about your AI/ML objectives and explore how Bitwise can help you identify the optimal use case and success metrics to put you on the path to achieving ROI and driving adoption in the organization.

The post 5 Use Cases for Driving ROI with Machine Learning appeared first on Bitwise.

]]>
https://www.bitwiseglobal.com/en-us/blog/5-use-cases-for-driving-roi-with-machine-learning/feed/ 0
How to use AI to modernize your PL/SQL code in Synapse or Snowflake https://www.bitwiseglobal.com/en-us/blog/how-to-use-ai-to-modernize-your-pl-sql-code-in-synapse-or-snowflake/ https://www.bitwiseglobal.com/en-us/blog/how-to-use-ai-to-modernize-your-pl-sql-code-in-synapse-or-snowflake/#respond Mon, 01 Jul 2024 06:44:53 +0000 https://www.bitwiseglobal.com/en-us/?p=48471 PL/SQL versus Synapse and Snowflake PL/SQL is a procedural language designed to be embedded in SQL statements. It is a powerful language that can be used to perform a wide range of tasks, including data manipulation, error handling, and complex logic. However, PL/SQL can also be difficult to maintain and update, especially for large and ... Read more

The post How to use AI to modernize your PL/SQL code in Synapse or Snowflake appeared first on Bitwise.

]]>

PL/SQL versus Synapse and Snowflake

PL/SQL is a procedural language designed to be embedded in SQL statements. It is a powerful language that can be used to perform a wide range of tasks, including data manipulation, error handling, and complex logic. However, PL/SQL can also be difficult to maintain and update, especially for large and complex codebases.

Synapse and Snowflake are popular cloud-based data warehouses that offer a variety of features and benefits, including scalability, performance and cost-effectiveness. They also provide SQL-like languages that are more modern and easier for building artificial intelligence and machine learning applications.

Challenges of migrating PL/SQL to cloud

There are a number of options for converting PL/SQL code to cloud-native systems. Tools like SnowConvert from Snowflake and AWS Schema Conversion tool can apply for certain scenarios and there are manual conversion and other third-party tool options.

Even with these tools, migrating PL/SQL code to Synapse or Snowflake can be a challenging and time-consuming process. Challenges include:

  • Understanding the legacy code – PL/SQL code can be complex and difficult to understand, especially for code that was written many years ago.
  • Reproducing the functionality – The goal of the migration is to reproduce the same functionality as the legacy code in the new environment. This can be difficult to do, especially if the code is not well-documented.
  • Testing the migrated code – Once the code has been migrated, it needs to be thoroughly tested to ensure that it is working correctly. This can be a time-consuming and error-prone process.

Using AI to overcome challenges and accelerate data modernization

When harnessed properly, artificial intelligence (AI) can help overcome a lot of the complexity that causes challenges when migrating to the cloud. Key areas where you can use AI to modernize your PL/SQL code in Synapse or Snowflake include:

  • Analyze the legacy code – AI can help identify patterns, dependencies, other important information that can be used to make the code easier to understand and accelerate migration.
  • Generate new code – using AI to replicate the functionality of the legacy PL/SQL code can save a significant amount of time and effort when converting to Synapse or Snowflake.
  • Test the migrated code – testing the migrated code and identifying any errors or defects is a critically important and difficult step in the modernization process, which can be assisted with AI to ensure that the migrated code is working correctly.

Generative AI approach to PL/SQL code conversion

Generative AI opens new doors for confronting the issues of tedious code conversion and optimization to accelerate your data modernization journey. With our advanced knowledge of PL/SQL code and deep experience modernizing data in Synapse and Snowflake, Bitwise has created powerful modules for transforming and validating code, including:

  • Code Converter – provides effortless conversion that automates code migration and modernization utilizing Gen AI. Its batch processing feature allows increased efficiency for automated PL/SQL conversion.
  • Code Optimizer – assesses code with Gen AI and suggests optimization designed to your specific goals. Code optimizer reduces space complexity, time and improves error handling assuring fine-tuned performance.
  • Code Documenter – automates the commenting and documentation process allowing a clear, comprehensible code base and can delve into variables, functions and defined objects. This not only enhances code readability but also adding to long-term maintainability.
  • Migrated Code Validation Utility – provides support for heterogeneous data sources and file types using a unique approach of in-memory comparison without moving the data across data stores. The utility generates comprehensive comparison and summary reports with synopsis of mismatched data and overall comparison stats to pinpoint any potential errors.

Conclusion

AI-powered PL/SQL code conversion to Synapse and Snowflake can be a challenging task, but it is a necessary step for enterprises that are looking to modernize their data and move to the cloud. AI can be used to overcome the challenges of migration and accelerate data modernization initiatives.

While using AI can be a game-changer for modernizing your PL/SQL code in Synapse or Snowflake, building the right AI competencies and using optimal prompt engineering with Generative AI comes with its own set of challenges. Our team has gone through extensive trial and error to perfect the steps needed to effectively harness AI to successfully convert PL/SQL code. Explore our Data Migration and Modernization solutions to see how we accelerate PL/SQL code conversion.

The post How to use AI to modernize your PL/SQL code in Synapse or Snowflake appeared first on Bitwise.

]]>
https://www.bitwiseglobal.com/en-us/blog/how-to-use-ai-to-modernize-your-pl-sql-code-in-synapse-or-snowflake/feed/ 0
Unlock the Best Value Out of Your Big Data Hadoop https://www.bitwiseglobal.com/en-us/blog/unlock-the-best-value-out-of-your-big-data-hadoop/ https://www.bitwiseglobal.com/en-us/blog/unlock-the-best-value-out-of-your-big-data-hadoop/#respond Sat, 21 May 2016 09:09:00 +0000 https://www.bitwiseglobal.com/en-us/unlock-the-best-value-out-of-your-big-data-hadoop/ Planning A Hadoop administration team’s responsibilities starts when a company kick-starts with the Hadoop POC. An experienced team like Bitwise comes up with a roadmap right at the beginning to help scale from POC to production with minimal wastage of initial investment and effective guidance on investment decisions, be it in-house infrastructure, POC or to ... Read more

The post Unlock the Best Value Out of Your Big Data Hadoop appeared first on Bitwise.

]]>

Planning

A Hadoop administration team’s responsibilities starts when a company kick-starts with the Hadoop POC. An experienced team like Bitwise comes up with a roadmap right at the beginning to help scale from POC to production with minimal wastage of initial investment and effective guidance on investment decisions, be it in-house infrastructure, POC or to choose PAAS options.

For any organization, understanding of the estimated investments is mandatory in the initial phases.
Capacity planning/estimation is the next step after successful completion of the POC. Choosing the right combination of storage and computing hardware, interconnected network, operating system, storage configuration/disk performance, network setup etc. play an important role on the overall cluster performance. Similarly, special considerations are required for the master and slave node hardware configuration. The right balance of needs vs. greed can be achieved only after years of implementation experience.

Deployment

Once you have the hardware defined and in place, the next stage is the planning and deployment of the Hadoop cluster. This involves configuring the OS with the recommended configuration changes to suite the Hadoop stack, configuration of SSH and Disk, choosing and installing a Hadoop distribution (Cloudera, Hortonworks, MapR or Apache Hadoop) as per the requirements, meeting the configuration requirements for Hadoop daemons for optimized performance. All of these setups vary based on the size of your cluster, so it’s imperative that you configure and deploy after covering all the aspects and pre-requisites.

Another important aspect is designing of cluster from development perspective, various environments (Dev, QA, Prod etc.) and usage perspective, i.e. access security and data security.

Managing a Hadoop Cluster

After implementation of the Hadoop cluster, the Hadoop admin team needs to maintain the health and availability of the cluster round the clock. Some of the common tasks include management of the name node, data nodes, HDFS and Mapreduce jobs which forms the core of the Hadoop eco system. Impact to any of the components can negatively affect the cluster performance. For e.g. unavailability of a data node, say due to a network issue, will cause the HDFS to replicate the under-replicated blocks which will bring a lot of overhead and cause the cluster to slow down or even make it inaccessible in case of multiple data node disconnections.

Name node is another important component in a Hadoop cluster and acts as a single point of failure. Consequently, it is important that a backup of fsimage and editlogs are taken periodically using the secondary name node so as to recover from a name node failure. The other administrative tasks include:

  • Managing HDFS quota at application or user level
  • Configuring scheduler (FIFO, Fair or Capacity) and resource allocation to different services like YARN, HIVE, HBASE, HDFS etc.
  • Upgrading and applying patches
  • Configuring logging for effective debugging in case of failures or performance issues
  • Commissioning and decommissioning nodes
  • User management

Hardening your Hadoop Cluster

Productionization of a Hadoop cluster mandates implementation of hardening measures. Hardening of Hadoop typically covers:

  1. Configuring Security: This is one of the most crucial and required configuration to make your cluster enterprise ready and can be classified at user and data level.
    1. User Level: User security addresses the authentication (who am I) and authorization (what can I do) part of the security implementation along with configuring access control over resource. Kerberos takes care of the authentication protocol between the client/server applications and is majorly used to sync with LDAP for better management. Different distribution recommends different authorization mechanism. For e.g. Cloudera has good integration with Sentry that provides a fine grained row level security to Hive and Impala. Further integration with HDFS ACL’s percolates the same access to other services like Pig, HBASE, etc.
    2. Data Level: Data Security, HDFS transparent encryption provides another level of security for data at rest. This is one of the mandatory requirements for some of the organizations to be complied with different government and financial regulatory bodies. Having transparent encryption built into HDFS makes it easier for organizations to comply with these regulations.
  1. High Availability: Name node as mentioned earlier is a single point of failure and unavailability of the same results in making the whole cluster unavailable, which is not a recommended approach for a production cluster. Name node HA helps to mitigate this risk by having a standby node which automatically takes over from the primary name node in cases of failure.
  1. Name Node Scaling: This is mostly applicable in case of a large cluster. As name node stores data in memory with large volume of files, name node memory can become a bottleneck. HDFS federation helps in resolving the issues by facilitating multiple name nodes with each name node managing a part of the HDFS namespace.

Monitoring

Proactive monitoring is essential to maintain the health and availability of the cluster. General monitoring tasks includes monitoring cluster nodes and networks for CPU, memory, network bottlenecks, and more. The Hadoop administrator should be competent to track the health of the system, monitor workloads and work with the development team to implement new functionality. Failure to do so can have severe impact on the health of the system, quality of data and ultimately will affect the business user’s ease of access and decision making capability.

Performance Optimization and Tuning

Performance tuning and identifying bottlenecks is one of the most vital tasks for a Hadoop Administrator. Considering the distributed nature of the system and a manifold of configuration files and parameters, it may take hours to days to identify and resolve a bottleneck, if not get started in the right direction. Often it is found that the root cause is at a different end of the system rather that what is pointed out by the application. This can be counterbalanced with the help of an expert who can assist with a detailed understanding of the Hadoop ecosystem along with the application. Moreover, an optimized resource (CPU, Memory) is essential for an effective utilization of the cluster and aids in the distribution between different Hadoop components like HDFS, YARN, and HBASE etc. To overcome such challenges, it’s important to have the statistics in place in the form of benchmarks, tuning of the configuration parameters for best performance, strategies and tools in place for rapid resolutions.

This blog is part of the Hadoop administration blog series and aims to provide a high level overview of Hadoop administration, associated roles, responsibilities and challenges a Hadoop admin faces. In the future editions, we will dwell further into the above mentioned points, various aspects of Hadoop Infrastructure Management responsibilities and further understand how each phase plays an important role in administering an enterprise Hadoop cluster. For more on how we can help, visit.

 

The post Unlock the Best Value Out of Your Big Data Hadoop appeared first on Bitwise.

]]>
https://www.bitwiseglobal.com/en-us/blog/unlock-the-best-value-out-of-your-big-data-hadoop/feed/ 0
Crossing Over Big Data’s Trough of Disillusionment https://www.bitwiseglobal.com/en-us/blog/crossing-over-big-datas-trough-of-disillusionment/ https://www.bitwiseglobal.com/en-us/blog/crossing-over-big-datas-trough-of-disillusionment/#respond Mon, 24 Aug 2015 15:33:00 +0000 https://www.bitwiseglobal.com/en-us/crossing-over-big-datas-trough-of-disillusionment/ Defining this Trough of Disillusionment Enterprises are feeling the pressure that they should be doing “something” with Big Data. There are a few organizations that have figured it out and are creating breakthrough insights. However, there’s a much larger set that has maybe reached the stage of installing say 10 Hadoop nodes and are wondering ... Read more

The post Crossing Over Big Data’s Trough of Disillusionment appeared first on Bitwise.

]]>

Defining this Trough of Disillusionment

Enterprises are feeling the pressure that they should be doing “something” with Big Data. There are a few organizations that have figured it out and are creating breakthrough insights. However, there’s a much larger set that has maybe reached the stage of installing say 10 Hadoop nodes and are wondering “now what?”

Per Gartner, this is the phase where excitement over the latest technology gives way to confusion or ambiguity – referred to as the “Trough of Disillusionment.”

Data Democracy – The Foundation for Big Data

Use cases involving Analytics or Data Mining with an integrated Social Media component are being thrown at enterprise executives. These use cases appear “cool” and compelling upfront but upon thorough analysis reveal that they are missing some necessary considerations such as Data/Info Security, Privacy regulations, Data Lineage from an implementation perspective, and in addition fail to build a compelling ROI case.

One needs to realize that for any “cool” use case to generate eventual ROI, it is very important to focus on Big Data Integration (i.e. Access, Preparation, and Availability of the data – see firms must not overlook importance of big data integration). Doing so essentially will empower the enterprises to implement ANY use case that makes the most sense to their particular business.

“Data Democracy” should be the focus. This focus also helps address the technology challenge of handling ever-growing enterprise data efficiently and leverage the scalable and cost-effective nature of these technologies – and an instant ROI!

Concept to Realization – Real Issues

Once this is understood, the next step is to figure out a way to introduce the use of these new technologies to achieve the above goals and doing so in the least disruptive and most cost-effective way. In fact, enterprises are looking at ETL as a standard use case for Big Data technologies like Hadoop. Using Hadoop as a Data Integration or ETL platform requires developing Data Integration applications using programming languages such as Map Reduce. This presents a new challenge in combining of Java skillsets with the expertise of ETL design and implementation. Most ETL designers do not have Java skills as they are used to working in a tool environment and most Java developers do not have experience in handling large volumes of data resulting in massive overheads of training, maintaining and “firefighting“ coding issues. This can cause massive delays and soak up valuable resources for organizations to solve half the problem.

Moreover, while making the investments in the form of hardware and skillsets like Map Reduce, when the underlying technology platforms inevitably would advance, development teams would be forced to rewrite the application to leverage these advancements.

Concept to Realization – a Possibility?

Yes it is. One of the key criteria for any data integration development environment on Hadoop is code abstraction to allow users to specify the data integration logic as a series of transformations chained together in a directed acyclic graph that models how users think about data movement making it significantly simpler to comprehend and change than a series of Map Reduce scripts.

Another important feature to look out for is technology insulation – provisions in the design to change the run-time environments such as Hadoop with any future technologies prevalent at that time.

Conclusion

The “3 V’s” in Big Data implementations are well defined – Volume, Variety, and Velocity – and relatively quantifiable. We should begin to define a 4th ‘V’, for “Value.” The fourth is equally important, or more important in some cases, but less tangible and less quantifiable.

Having said that, jumping off a diving board into a pool of Big Data doesn’t have to be a lonely job. The recommended approach would be to seek help from Big Data experts like Bitwise to assess whether you really need Big Data. If yes, what business areas will you target for the first use case, which DI platform will you use? And lastly, how will you calculate the ROI of the Big Data initiative?

The post Crossing Over Big Data’s Trough of Disillusionment appeared first on Bitwise.

]]>
https://www.bitwiseglobal.com/en-us/blog/crossing-over-big-datas-trough-of-disillusionment/feed/ 0