Agile Education Case Study

Improving Forecasting Accuracy with Scrum at Scale

This case study examines the journey of a large, non-profit health insurance provider as they implemented and scaled Scrum practices to enhance their digital product portfolio. The organization faced challenges with Product Owner cycles, release planning, and forecast accuracy, leading to missed goals and low morale. Through strategic implementation of an Executive Metascrum, they improved forecasting accuracy and customer satisfaction, achieving significant organizational improvements over a six-month period.


Trainer Name: Giovanna Berry
Industry: Nonprofit, Healthcare, Insurance, and Emergency Services
Organization Size: Large
Topic: Executive MetaScrum (EMS), Product and Release Feedback, Product Ownership, Release Planning
Date: 2024

Case Study   

Summary: Improving Forecasting Accuracy with Scrum at Scale

This case study focuses on a large non-profit health insurance provider based in the Midwest. The organization was on a journey to scale their Scrum practices and improve their digital product portfolio, valued at $20 million. With 100 team members spread across six Scrum Teams and four Scrum of Scrums, they had experienced organic growth and success until encountering several significant challenges. That’s when Giovanna and her team were called in. Right away, it was clear to Giovanna that the organization’s primary challenges centered around Product Ownership. Here’s what they saw:

  • Lack of a Product Owner Cycle: The organization struggled with an underdeveloped Product Owner (PO) cycle, which impeded their ability to efficiently manage and prioritize product backlogs.
  • Executive Metascrum Issues: The absence of a functional Executive Metascrum (EMS) limited visibility into roadmaps and release plans, which were often non-existent, outdated, or decentralized.
  • Inaccurate Delivery Forecasts: Delivery forecasts were largely speculative and unreliable due to inadequate release planning.
  • Inability to Forecast Team Capacity Needs: The organization faced challenges in balancing team sizes and adjusting support when demand fluctuated. This led to missed goals and delayed releases.
  • Low Team Morale and Unhappy Customers: The delays and mismanagement led to decreased team morale and dissatisfaction among customers who experienced delays in promised functionalities.

The Solution and Results

Giovanna and her team took a strategic and incremental approach to address these issues. They decided to implement an EMS reference model, focusing on a single Scrum of Scrums with seasoned POs who had sound practices in place. They would track progress for 6 months (12 Sprints). In addition to the experimental group, they also tracked the control group’s progress throughout. Here’s what the implementation looked like: 

  • 1st Month: The team started with monthly EMS events to test the process and address initial challenges.
  • 2nd Month: In month two, they added an ask for a transparent release plan with clear dependencies identified. They stored those in a centralized location to serve as a reference for those not involved in the experiment.
  • 3rd Month: After the first few months, shifted to bi-weekly EMS events to enhance planning and coordination. They also began reporting on the accuracy levels of scheduled forecasts. Of note, the experimental group’s forecast accuracy started at 65-70%. In contrast, the control group was at a 30-60% range.
  • 6th Month: By month six, the experimental group had achieved 80-85% forecasting accuracy. The control group’s accuracy did not fluctuate meaningfully over the experiement.

Key Learnings and Themes

  • Customer Satisfaction: There was a notable 10% increase in customer satisfaction scores in the final two months of the experiment compared to the previous year.
  • Leveraging Change Agents: Involving internal change agents in the process significantly increased buy-in and effectiveness compared to using external consultants alone.
  • Increased Team Autonomy: Allowing teams more autonomy within the Executive Metascrum framework improved participation and engagement.
  • Self-Organizing Teams: Teams that were given the freedom to self-organize were more engaged and successful.

Giovanna leaves us with wise words from Steve Jobs that guided her team during this transition: “It doesn’t make sense to hire smart people and tell them what to do; we hire smart people so that they can tell us what to do.” She shared that while she and her team knew that they were Scrum subject matter experts and knowledgable in the framework and change management, they understood that they couldn’t discount the insights and capabilities of those already doing the work day-to-day. Giving them clear, outlined objectives, parameters, and guidelines offered them the structure to encourage autonomy and to allow the teams themselves to define those specifics and nuances to make the process work for them.

Next Steps

Building on the success of the initial experiment, the organization plans to expand the reference model across additional teams and Scrum of Scrums. They aim to use established metrics to monitor ongoing performance and foster a culture of continuous improvement.

About Giovanna Berry

Registered Scrum Trainer and Scrum@Scale Trainer Giovanna Berry is a PMP, CSP-SM, and MBA. She is an energetic, seasoned project and program management professional and leader. For over a decade, Giovanna has led teams focused on custom software development, application development, developing creative designs, implementations, and migrations. Giovanna has always had a passion for training, coaching, and mentoring others.

More Scrum@Scale Case Studies

Transforming Logistics: From Silos to Speed

Transforming Logistics: From Silos to Speed

Unbox Robotics Private Limited, a company specializing in order sorting and product consolidation for logistics, e-commerce, and warehouse companies, undertook an Agile transformation to improve efficiency and market responsiveness. When Pasha and his team began their work within the organization, teams were not coordinated which was impacting speed to delivery, overall processes were inefficient due to heavy bureaucracy, and the organization was up against a key trade show deadline. With this in mind, the team was given six months to reach their goals and a limited budget to make it happen. Read on to hear how Scrum@Scale is transforming logistics, helping Unbox Robotics increase speed to market and achieve 5x revenue in 12 months.
Achieve 5-10x Faster Market Delivery with Scrum at Scale

Achieve 5-10x Faster Market Delivery with Scrum at Scale

One of the Netherlands' largest DIY retail franchises had a siloed management structure and projects that took 3-12 months or even longer to get to market. They had a waterfall delivery style, with the entirety of the delivery at the end and no early value delivery for customers. Registered Scrum@Scale Trainer Serge Beaumont's approach started with the creation of cross-functional, autonomous teams, each solely responsible for a different part of the process. This allowed for work to proceed very quickly with each team simultaneously able to complete their part of a project without first waiting on another team. The results were immediate and impressive. The team achieved 5-10x faster market delivery to customers, there was now alignment between what was expected and what was delivered, delivery was made in smaller chunks allowing for early value delivery to customers, and e-Commerce sales soared.
Creating Customer-Centric Social Media Teams Using Agile

Creating Customer-Centric Social Media Teams Using Agile

In this case study, Christoph Dibbern tackles a myriad of challenges that were plaguing a social media-based organization – lack of prioritization, no organizational refactoring, and a lack of shared values. These challenges created an unproductive and unhealthy environment, where mistrust and misalignment were common and innovation was stifled. Utilizing Scrum@Scale, the organization was able to align around a customer-centric focus, work as a cohesive unit, and deliver high quality results to customers faster.