Ryan McCorvie: Inside the Real Work of Statistical Consulting
Organizations often reach a point where their reports stop lining up with how they experience their own operations. People notice odd patterns or inconsistent figures. Meetings fill with questions that never quite get answered. By that time, the complications have usually been developing for a long period of time.
“Early decisions about what to measure and how to record it set the stage for everything that comes later,” says Ryan McCorvie, mathematician and owner of Martingale. “When those decisions are rushed, the problems grow quietly until someone is forced to confront them.”
McCorvie has described this challenge in direct terms, noting that “most data problems begin before the first number is collected, because people decide what to track without deciding what they want to understand.”
His point reflects what consultants see regularly. Many organizations inherit measurement choices that never matched their goals. Once those choices become part of the workflow, the team treats the data as fixed even though the foundation beneath it is unstable.
Survey work provides a clear example of how things begin to drift. A set of questions might appear straightforward, yet respondents interpret them in ways the team never anticipated. Even well-meaning edits can introduce new sources of confusion. Once the responses show up in a spreadsheet, the organization treats them as definitive even though the underlying structure cannot support clear meaning. People try to correct the issue by gathering more responses and end up reinforcing the problem.
Experimental work runs into its own set of difficulties. If test groups differ in subtle ways or if timing shifts from one run to another, results can vary for reasons no one notices. Teams often assume the model is at fault. In reality, the design of the study never created conditions that allow for firm conclusions. The numbers might look precise, but the foundation beneath them is not.
Mismatched definitions create a different kind of trouble. Two departments can use the same term while meaning completely different things. Reports circulate through leadership with measurements that do not align. Decisions are made on top of conflicting information. By the time a consultant enters the picture, the confusion has shaped months of internal planning. The consultant begins by tracing how each definition was formed and why the organization now finds itself stuck.
“These issues do not appear all at once. Small choices accumulate until the structure of the data can no longer support the level of certainty the organization expects,” says McCorvie. “The consultant’s first job is to unwind those choices and rebuild a foundation that reflects what the team wants to understand.”
What Statistical Consulting Looks Like Day to Day
A consultant’s work begins with conversation rather than computation. Teams describe their goals, frustrations, and constraints. These early discussions reveal gaps that are not obvious from the dataset alone. The consultant listens for inconsistencies between what the organization wants to know and what the data can actually provide. This phase is slow by design because it shapes every decision that follows. Many organizations underestimate how much uncertainty exists inside their own systems. A global survey reported through Harvard Business Review found that 62 percent of employees do not trust their data, which reflects how common it is for teams to work with information they do not fully understand, as shown in this analysis.
After gathering context, the consultant evaluates how the data was collected and how it has been handled. This includes identifying gaps, unusual patterns, or points where prior analysis used assumptions that do not hold. Sometimes the information is suitable for the intended question. Other times, it needs restructuring before it can support any meaningful interpretation. The consultant explains these concerns plainly so the team understands why certain steps must be taken.
“Communication takes up more time than most people expect,” says McCorvie. “Consultants spend many hours translating technical ideas into language that helps people make decisions.” This includes explaining what the results mean, what they cannot support, and how confident the team should be in any interpretation. When people understand the conditions behind the findings, they are less likely to overstate or misinterpret the outcome.
Another part of the job involves documenting choices. Every method relies on assumptions. Rather than burying them in a technical appendix, consultants state them clearly so the team sees how each conclusion was reached. This helps future work remain consistent and reduces confusion when staff members or priorities shift.
Although statistical modeling appears central to the profession, it represents only a portion of the job. Much of the real value comes from diagnosing the structure of the problem, guiding teams toward realistic expectations, and ensuring that the analysis supports the decisions at hand. The daily work is more conversational and investigative than most people expect.
The Turning Point: When Better Statistical Design Changes the Outcome
Organizations usually sense a shift once the design of a study aligns more closely with the real question. They begin to understand why earlier efforts produced contradictory results. A clearer structure allows the analysis to reflect stable relationships rather than random variation. As the internal picture becomes more coherent, teams recognize how stronger design reshapes the decisions they feel prepared to make. This is consistent with findings from McKinsey, which reported that organizations using effective people-analytics are 4.8 times more likely to make better decisions, as detailed in their 2023 organizational analysis.
In many cases, the consultant’s work reshapes how the organization defines each variable. When measurements line up with the intent of the study, results become easier to interpret. Teams stop arguing over what the data means and start discussing what they should do next. The reduction in confusion alone changes how quickly the organization can move.
This improved design often exposes relationships that were previously hidden. When the steps of the study match the structure of the question, patterns begin to make sense. The team can then evaluate whether those patterns point toward operational adjustments, further investigation, or a confirmation of the current approach. The clarity brings a steadier rhythm to decision-making because the organization no longer feels lost in conflicting signals.
“Time savings become another noticeable benefit,” says McCorvie. “Before the redesign, teams may have spent weeks revisiting the same issues. Afterward, they reach conclusions with fewer meetings and fewer revisions.” Reports become more direct. People spend less energy defending interpretations and more time planning future actions.
Once a stronger framework is in place, organizations often rethink their long-term measurement plans. They can track progress with greater confidence, refine their internal metrics, and design future studies that build on what they have learned. The turning point is not just in the numbers but in how the organization understands its own work.
Techniques That Matter (And the Ones People Overuse)
Consultants rely on a range of statistical tools, yet they choose methods based on what the question requires rather than what sounds advanced. A straightforward regression or a clear comparison between groups can provide more insight than a complex model. Simpler methods often reveal more because they impose fewer assumptions and are easier for teams to understand.
“Many organizations ask for intricate techniques when they are not necessary,” says McCorvie. “This usually happens when the team hopes that complexity will resolve deeper design issues.” In practice, the opposite occurs. A complicated model can amplify weaknesses in the data. When this happens, consultants show why a more modest technique is not a step backward but a more direct path to an accurate interpretation.
Some methods are repeated out of habit. Teams may reach for the same test every time because it worked in earlier projects. When the structure of the problem changes, that method may no longer fit. Consultants help teams identify these patterns and introduce alternatives that match the new context.
The suitability of any technique depends on the data. Certain methods rely on assumptions about how the information behaves. If those assumptions are not met, the results can be misleading. Consultants check these details carefully. When they find mismatches, they suggest methods that respect the actual properties of the data.
Technique selection becomes meaningful only when paired with strong design. A well-chosen method cannot compensate for unclear goals or inconsistent data collection. Consultants help organizations match the question, the method, and the data so the analysis supports decisions rather than complicating them.
Ryan McCorvie: Where Statistical Consulting Makes the Biggest Difference
Statistical consulting strengthens decision-making in areas where information must withstand scrutiny. Operations that depend on accurate measurements benefit from careful evaluation of how those measurements are taken. When an organization relies on numbers to assess performance or risk, consultants help ensure that those numbers reflect real conditions. This need is clear in findings from BARC, which reported that 58 percent of companies base at least half of their regular business decisions on instinct rather than data, as shown in their global survey on data-driven decision-making.
Departments that manage complex processes often seek outside support. In these settings, small inconsistencies in measurement can accumulate quickly. Consultants work with teams to refine definitions and set up procedures that reduce variation introduced by the process itself. This improves the reliability of internal reporting and helps teams respond more effectively to changes.
“Roles tied to customer behavior or product usage face a different challenge,” says McCorvie. “Patterns fluctuate for many reasons, and it can be hard to separate meaningful changes from temporary shifts.” Consultants help identify which patterns deserve attention and which can be set aside. This prevents teams from chasing signals that do not point toward long-term adjustments.
Forecasting is another area where consulting adds structure. Internal projections often vary widely, especially when different groups use different assumptions. Consultants bring clarity by defining consistent rules for how forecasts are built. They also explain what range of uncertainty should be expected. Leaders can then plan future steps with a more stable understanding of likely outcomes.
The value of statistical consulting increases when organizations base decisions on information that must remain trustworthy. When the analysis is structured, repeatable, and connected to the real question, decision makers operate with greater confidence.
The Mistakes Clients Repeat (Even Smart Ones)
A frequent error involves treating reports as complete descriptions of reality without examining the assumptions behind them. Every dataset carries decisions about definitions and handling of missing information. When teams overlook those decisions, they often overstate what the results can support. Consultants remind organizations that interpretation depends on understanding how the numbers were produced.
“Another recurring issue is mistaking patterns for explanations,” says McCorvie. “Teams sometimes see two variables move together and assume one causes the other.” Without evaluating the design of the study, this assumption can lead to actions that do not address the underlying problem. Consultants help separate descriptive relationships from conclusions that require stronger evidence.
Some organizations gather large amounts of data before deciding what they want to learn. When the goals are unclear, the dataset cannot support the level of analysis leadership expects. Consultants then have to determine whether the information can be salvaged or whether new collection is needed. This conversation reveals gaps that went unnoticed during the initial planning.
Technology introduces a different kind of pitfall. Teams sometimes rely heavily on dashboards or automated systems, assuming these tools provide complete answers. Automation can be useful but cannot replace careful design. Consultants explain where these tools help and where they fall short. This prevents teams from leaning on reports that create a false sense of certainty.
These mistakes persist because organizations face pressure to make decisions quickly. Consultants help slow the process just long enough to understand what the data actually supports. This saves time in the long run because fewer decisions need to be revisited.
When Statistical Consulting Is Not the Right Answer
Consulting does not fit every situation. Some teams approach a consultant before gathering any meaningful data. In these cases, the consultant cannot analyze something that does not yet exist. Instead, the focus shifts to planning. The organization learns what information needs to be collected so the later analysis will be useful.
Occasionally, the issue is operational rather than analytical. If a process is inconsistent or poorly documented, the data will reflect those inconsistencies. The consultant might recommend improving the process first. Once the workflow becomes more stable, the analysis becomes far more reliable.
Certain decisions rely on stable inputs and straightforward logic. A simple rule may provide a clearer answer than a formal model. Consultants help teams recognize when a basic approach is sufficient. This prevents organizations from assuming that more complicated methods automatically produce better results.
Some projects lack agreement on the central question. When leaders cannot define what they want to understand, the consultant cannot build a coherent analysis. The team must settle on a goal before moving forward. Without that clarity, any statistical work risks becoming disconnected from the decisions that matter.
Recognizing these limits helps organizations avoid wasting time. A good consultant identifies when the conditions are not suitable for meaningful analysis and guides the team toward steps that prepare the groundwork.
Why Judgment Will Matter More Than Software
Software continues to improve, yet it cannot replace the judgment required to understand the structure of a problem. A model can run rapidly, but it cannot determine whether the question is framed properly. Consultants evaluate how people interpret results, how departments coordinate decisions, and how assumptions shape the final conclusions.
“Judgment plays a central role when dealing with uncertainty,” says McCorvie. “Software can estimate uncertainty, but only people can determine whether the level of risk is appropriate for the decision at hand.” Consultants help teams understand which outcomes are plausible and which interpretations stretch the limits of the data.
Ethical considerations require human oversight as well. Decisions about what to measure, how to interpret the results, and how information will be used involve more than technical skill. Consultants guide these discussions by explaining how different design choices affect the interpretation and the potential impact on the organization.
Automation introduces speed and consistency, but it also increases the need for careful design. Tools follow instructions without questioning their relevance. Consultants help ensure that those instructions reflect the real problem rather than an assumption carried over from an earlier project.
Human judgment connects the analysis to the decisions that follow. Software handles computation, but people determine what the results mean in context. That combination will continue to define the value of statistical consulting.
Clear Thinking Is the Product
Statistical consulting gives organizations a structured way to interpret information that may otherwise feel confusing or contradictory. The process begins with understanding the question and determining whether the available data can support it. When teams see how each step influences the final outcome, they make decisions with more clarity.
“This clarity changes how organizations operate,” says McCorvie. “Meetings shift from debates over conflicting reports to conversations about what actions make sense. Leaders gain a more stable view of their options. Teams become better at identifying which questions can be answered and which require more information.”
Clear thinking becomes the practical outcome of the consulting process. When the structure is sound, people can focus on choices rather than uncertainty. The organization develops a more confident approach to interpreting information, and that confidence supports more consistent and thoughtful decisions.
