DevOps Testing Services

There are subtle differences in Agile and DevOps Testing – the most prominent differences being continuous testing and engaging test automation at each phase.

Even though both Agile and DevOps advocate a culture, testing in DevOps is more fast-paced than in an Agile setting and relies on automating every possible task from testing to deployment.

Recommendations for successful implementation of DevOps testing:

• Adherence to continuous testing.
• Automating feasible tasks.
• Layered tests (Smoke test, subset of regression tests, full-blown regression tests) to ensure faster feedback at each checkpoint.
• Intelligent regression testing.
• Ability to set up testing environments.
• Execution of automated tests (cross-platform, cross-browser, and cross-device).
• Scaled up performance/load testing.
• Creation of a wholly evolved test automation harness.

Your organization will benefit from Agile testing implementation by enabling close collaboration with stakeholders and developers, thus aiding communication within the team.

Furthermore, you will see a reduction in time-to-live with parallel execution, achieve optimum test coverage with test automation and be able to identify and execute tests for a particular build, yielding quicker feedback.

Continuous Testing with Speed and Quality

Whether starting the process of implementing Agile testing or advancing your capability to a DevOps paradigm, continuous testing is essential to delivering better quality and quicker time-to-market of your applications in today’s competitive environment
Keeping ahead of changing industry trends, Bitwise is well versed in providing test automation tools, technology, and capabilities to enable a smooth transition to Agile and DevOps maturity and accelerate the deployment of highly complex enterprise applications.

The Science of Story Points

Why use story points? They are not even real, right? Let’s see about that.

The traditional approach of estimation has been to break down the work and then assign the amount of time an “ideal” developer would take to accomplish it. These ideal estimates (in days or hours) are based on some assumptions about the rate at which software can be produced by this fictitious developer. The assumed productivity rate is based on various factors such as skills, experience, coding style, etc.

The story point approach makes it simple and data driven. These are hypothetical numbers assigned to tasks based on the amount of work expected to complete the task and the complexity involved. How could an imaginary story point be translated into the real effort spent by the team? Here’s how:

To begin with, you need to have a system in place to measure the actual hours spent by developers on tasks labelled as one story point. It’s fine if it varies by person and by team.

The central limit theorem The central limit theorem in probability states that the arithmetic mean of a sufficiently large number of iterates of independent random variables (hours consumed per story point in this case) will be distributed like a bell curve. This means if you plot the hours taken to build a story of size 1 story point against the number of stories on a chart, the graph will look somewhat like the diagram here. You clearly see a bell shaped pattern with a mean value “x”. Now similarly, plot the hours taken to build a story of size 2 story points, its mean will be 2x. This “x” is your conversion factor for translating story points into person hours.

What if you don’t have the historical data or a system to gather the data? Start with assigning an estimate of 1 story point to a familiar task that usually takes 8 hours.

The Science of Relative Sizing

Agile estimation techniques recommend sizing up tasks relative to each other. The estimator compares the task or story being estimated with one or more other tasks. If the task involves twice the work or is twice as complex, it is estimated twice as large.
So what are the benefits in doing that?

When you estimate tasks relative to each other, your estimation process is not only more accurate but significantly faster. There is an evidence that humans are better at estimating relative size than the absolute size (Lederer and Prasad 1998; Vicinanza et al. 1991).

Looking at a map, it’s hard to guess that the distance from New York to Chicago is 800 miles and New York to Boston is 200 miles. But it’s not so hard to tell that the distance from New York to Chicago is four times as much as the distance from New York to Boston.

The Coastline Paradox

How much effort should a project of size x take to estimate accurately? Well, the lesser the better. Estimates by definition are supposed to be just usable approximations. Estimation effort beyond a point yields very little value to the project.

The Coastline Paradox It may seem counter-intuitive but the more you break down a task to a micro level, the more are the chances that your estimate is going to be wrong.

A similar challenge is experienced by surveyors trying to measure the coastline of a country. It can produce varying results depending upon the length of the scale chosen for the job.

With a smaller the scale, and you can measure every little crevice along every creek on the shore. Take a bigger scale and it smooths out all details. This paradox of a constant finite area having varying length of perimeter is called coastline paradox.

In short, whether you are estimating efforts or measuring a coastline, choose the level of detail you want to consider wisely. Living with a measure of ambiguity at times will not only save your efforts but will also increase effectiveness of your efforts. Sound counter-intuitive? That’s why it’s called a paradox.

•  
• Misconception About Agility

Agile BI is much more than a project management technique. Creating a truly agile BI organization requires the involvement of leaders, developers, testers, architects, business analysts, stakeholders, business partners, as well as project managers. Agile BI requires substantial change equally for all members of delivery team, both procedural and cultural. This will ensure that the team is galvanized around a shared set of principles and techniques.Agile is not a magic wand that when waved over the BI team, will result in greater team capacity. Compartmentalizing responsibility for agile BI to the technical team is a sure-fire way to experience poor results.Scrum is perhaps the most well-known and easily understood of the agile methods. Many incorrectly perceive it as the very definition of agility. Many teams continue with Big Design Up Front (BDUF) instead of Just Enough Design Up Front (JEDUF) and testing as manual and final phase activity. They fail to deliver value every few weeks or seek feedback from business customers. This mistake is so common it has become known as “scrummerfall.”

• Creating a Comprehensive Design Up-Front

You do not need to create a comprehensive, detailed model up front. You only need a high-level vision at the beginning of the project and the details can be identified on a just-in-time (JIT) basis via model storming. There are several reasons for this.First, like it or not, the requirements are going to change throughout the project.Second, by waiting to analyze the details JIT, you have much more domain knowledge than if you had done so at the beginning of a project.Each sprint should be treated as a thin slice across the DW/BI landscape that delivers value to the customer. For agile DW/BI, the working software that adds value is a combination of queryable database schemas, ETL processes and BI reports/dashboards. The minimum set of valuable working software that can be delivered per iteration is a star schema, the ETL processes that populates it and a BI tool or application configured to access it.

• Wish-Based Planning

Agile BI is not a technique for getting more done faster with fewer people. Rather, it is a technique for maximizing value (by focusing on the most important things first) within the team’s limited capacity.Many leaders wish their team’s capacity were greater in order to meet all the demands of the business. These leaders have difficulty accepting these limitations, so they pressure the team to overcommit. Agile expert, Jim Highsmith, refers to this tendency as “wish-based planning.” Agile BI emphasizes value-based prioritization and adaptive scoping. This enables a team to build as much as possible within capacity limits. Even when they are unable to deliver 100 percent of the scope, they will have delivered the maximum value before time and budget runs out.

• Limiting End Users Collaboration

“Customer collaboration over contract negotiation” is one of the core values expressed in the Agile Manifesto and one of the first sacrificed by many agile BI teams.True customer or end user collaboration is an essential differentiator between projects that delight business users and those that receive lackluster response. When end users are engaged early in the project and their time is treasured, they become enthused by having their voices heard on a regular basis. They become an extension of the BI team, helping to shape the evolving product. They develop a sense of ownership in the outcome and become project champions with other members of the user community. Proper customer collaboration is difficult, but well worth the effort.

• Focusing on the Wrong Metrics

Velocity is a valuable tool self-managing team to assess its own performance and plan within its capacity. Velocity is a measure of how many story points an agile BI team delivers and users accept in each sprint.Unfortunately, management often misuses velocity as a measure of team productivity. Worse yet, some managers even attempt to allocate velocity to individuals to track each team member’s performance. When managers focus on velocity as a performance metric, agile BI teams respond by reducing testing, cutting design corners, and opting for speed over quality, all of which have undesirable consequences in the name of increased velocity.Agile BI emphasizes the importance of measuring the outputs of self-organizing teams rather than the ways those outputs are produced. Tools that monitor value delivered, such as burn-up charts, are more effective than performance measures such as velocity.

Focus should be on metrics that validate quality, such as test coverage and technical debt analysis, as these are more meaningful than task-completion metrics and work breakdown monitoring.