Hyperion Global Partners CEO, Eyal Iffergan, in one of his recent blog posts, Legal Metrics: Defining Success Beyond Spend Management, highlights a critical issue that we see corporate law departments — and the law firms and legal service providers who support them — struggle with every day: meaningful measurement. As the article notes, “successful outcomes determine the law department’s value to the enterprise,” and in order to achieve successful outcomes, it’s about having not just the right processes and tools in place, but also the right metrics and key performance indicators (KPIs).
The fact that most Global 1000 legal departments have implemented electronic billing and legal spend automation systems is a start, but there are a number of other KPIs related to eDiscovery that could — and should — be implemented to provide in-house counsel the insights needed “to better understand case status and support decision-making” that ensure successful outcomes.
The EDRM metrics model provides a framework for understanding how information, tasks and outcomes — specifically custodians, systems, media, status, formats, QA, and activities — can be measured in terms of volume, cost, and time. This framework is useful at a high level “to facilitate dialogue about the elements of a project or engagement, and how these will be performed and completed, and at what cost.” This is a positive step, but it is only likely to help organizations find incremental efficiencies. In order to have a deeper, more transformative effect, the industry needs to define KPIs based not just on the process itself, but on specific areas that are either common choke points or deadline-/resource-busters.
Pre-review data reduction
Review is by far the costliest portion of the eDiscovery process, consuming 73 cents of every eDiscovery dollar spent1. We’ve written about the need to reduce the volume of data before it enters the review process, and the need to leverage technologies that combine increased visibility and automation earlier in the process. If the goal is to increase the rate of data reduction as early in the process as possible, it’s critical to track all the potential reduction points and measure how much data is defensibly reduced at each stage.
Another opportunity to significantly reduce the cost of eDiscovery — especially for organizations that routinely deal with litigation or investigations that involve the same sets of data — is to eliminate duplicate processing, which is when the same documents are collected, processed, and classified over and over again across multiple matters. As a result, processing costs are higher and timeframes are longer than they need to be. By tracking how much duplicate data is collected, processed, classified, and reviewed across cases, you can configure technology and processes to eliminate duplication.
Volume of junk data processed
Not all documents contained in enterprise systems are business records; there’s a lot of data that is irrelevant to the business — especially in email — that can get swept up as part of eDiscovery. This “junk” data provides no business value and will never be responsive to any matter — and it adds to the time and cost of the discovery process. In addition to tracking the volume of responsive vs. nonresponsive data that is collected and processed, it’s important to measure how much of the non-responsive data is actually junk and should be completely eliminated from all discovery — and even overall organizational information governance — processes and repositories. This will allow you to reduce the volume of junk data over time to increase the efficiency of the discovery process.
There are a number of key decision points across the eDiscovery process — such as whether to settle or who should be on the core custodian list, to name just a few — that help shape the direction of the case. At each decision point, you have to balance the need for complete information with the need for timely action. Rushing to a conclusion without all the key facts could result in a decision that has negative consequences, but waiting too long can compress timeframes and strain resources. You need to know where the key strategic decision points are throughout the process and what information is needed. Then you can measure how quickly decisions can be effectively made, with the goal to reduce the time needed to reach an informed decision.
Late-stage discovery of key information
It’s the scenario everyone dreads: You think you are in the home stretch, then you discover a key custodian you missed — and now have to scramble to collect, process, and make sense of a whole new set of documents. Uncovering data, custodians or systems late in the process is disruptive, puts you at risk for sanctions, and can even have a negative effect on outcomes. Measuring how often key data, custodians, systems, and other information surface late in the process — and digging into why they weren’t uncovered sooner — can help eliminate, or at least reduce, these surprises.
Accurate scoping of an eDiscovery project is important to ensure that adequate resources are available, meet deadlines, and keep budgets in check. Under-scoping a project can have a negative effect on quality and execution, while over-scoping can tie up resources unnecessarily. It’s important to know how good your organization is at forecasting so that you can improve accuracy. By tracking forecasted vs. actual budgets, resources, etc., you can identify where you need improvements to prepare appropriately for each and every matter.
Measure what matters
It’s easy to fall into the trap of building KPIs based on the data you have. The problem is that sometimes the data you have doesn’t reflect what’s really important to the organization. Instead, it’s important to start from what matters. Understand what metrics are meaningful to the business. In the case of eDiscovery, these will be related to cost, risk, and outcomes. Then can you build the KPIs to track those metrics. Only once you understand what you need to measure can you determine how to measure it and what specific goals you’re looking to meet.
1Rand: Pace, Nicholas; Zakaras, Laura, Where the money goes: Understanding litigant expenditures for producing electronic discovery, (April 2012)