Bloomberg Law
Nov. 13, 2020, 3:42 PM

ANALYSIS: Lifecycle Metrics — The How & The Why

Jacquelyn Palmer
Jacquelyn Palmer
Research and Data Analyst

As use of legal technologies has increased over time, so too have resulting efficiencies in lawyer workflows. But few organizations have metrics in place to help identify the barriers that impede the use of available tech in those workflows. Integrating metrics into organizations’ strategy that capture the life cycle of the technologies in use will highlight pain points in the process and allow management to address any issues. Organizations can take simple steps to help identify proper metrics and make sure they have the right tools in place to suit lawyers’ needs.

Metrics Are Rarely Used to Evaluate Tech

Lawyers are increasing their use of technologies over time. However, Bloomberg Law’s Legal Technology 2020 Survey reveals that only 7% of respondents have formal legal tech metrics in place in their organization. And 21% report that they have no informal methods of measuring the value of their legal tech either.

So what do the 79% of respondents who have informal measurements in place say they consider when evaluating their legal tech? Top factors include the amount of time lawyers spend on routine tasks and matters of increased sophistication.

And so far, according to results from the survey, tech is allowing lawyers to focus more time on higher-level tasks and less time on lower-level tasks, but only somewhat.

Our survey also shows that lawyers are encountering barriers to using their technologies, which include lack of familiarity, lack of training, and poor integration. These barriers can lead to low use and reduce the positive impact that tech can have on lawyers’ workflows and well-being.

Phases of Lifecycle Metrics

There are many reasons why tech may not be living up to its full potential within an organization, and until organizations develop relevant lifecycle metrics with which they can evaluate their tech, they will never know where the issues lie.

What are lifecycle metrics? A collection of hard and soft measurements that span the life cycle of a tool — from the initial need that the tool is aimed at solving to its impact on work product and efficiency. Hard measurements include standard performance-based evaluations, but adding softer employee-based metrics can help provide much needed information about phases of the life cycle that are frequently ignored, such as effectiveness of training, internal communication about the tool, and ease of troubleshooting.

By following a process of assessment across certain areas, management can identify issues to optimize the utilization of technology within their organization.

What’s the Problem?

Needs change over time. To ensure the suitability of their tech solutions, organizations must periodically reassess their needs and the tools they have in place. For example, management may find that lawyers’ needs have shifted and that the tools in place are no longer working, or that other solutions may be more appropriate.

What’s Your Strategy?

By having a well-defined strategy in place for tech, management can make sure that they are addressing areas with the highest unmet needs, and that they do not have tools in place that serve overlapping functions. Tech strategies should not only consider what tools are needed now, but also how lawyers’ workflows are evolving and what tools they may need in the future.

An understanding of the overall strategies at play — in the organization, department, project, and elsewehere — is core to needs identification and tech alignment. Strategic adoption and implementation will ensure that lawyers are working on tasks using similar tools, which will facilitate collaboration and allow lawyers to more easily rely on each other when they have questions or need backup.

Do People Know It Exists?

If utilization is low, one of the easiest first steps to determining the cause is to simply ask, “Are lawyers aware this tool exists in the organization?” Is the answer “No” or “Maybe”?

It is important that managers have an internal communication strategy that provides lawyers with information including a list of available tech, an explanation of how that tech is useful in workflows, and guidance related to utilizing the tech. Management should periodically reassess their communication strategy to ensure that they are adequately expressing their support of workflow tools.

Is It Integrated?

Poor integration, yet another frequently identified barrier among lawyers, can make tools difficult to access and a headache to use. The more seamlessly tech is integrated into existing systems and workflows, the more likely lawyers will turn to the tool when it is appropriate to do so. Organizations should ensure that their tools are easy to access and play nice with other existing applications.

How’s Your Training?

Ineffective training and lack of time to familiarize oneself with tech tools are common barriers reported by lawyers. Evaluating the types of training in place and surveying the effectiveness of different types of training would provide guidance to those implementing tech strategies and help ensure that lawyers are using tools to their utmost advantage. However, management must also support lawyers’ attempts to fit tech skills development into their busy schedules. Reducing obstacles, adding a tech competence component to performance evaluations, and/or incentivizing training by allowing lawyers to receive billable-hour credit for the hours they spend in tech training are all ways to help ensure that lawyers dedicate the time needed to familiarize themselves with tools.

What’s Your Culture Like?

Leading by example is crucial. No amount of training can replace a supportive organizational culture. Management should be utilizing tech in their workflows. This will encourage lawyers to stop relying on archaic work processes like hand-drafting, referencing hard copy books and manuals, or requesting printed versions of electronic documents. As executive support for advancing workflows becomes more pronounced, the organization’s culture may shift toward being more open to new ways of working.

Is There Support?

There are bound to be issues with tech, no matter how world-class the training and how seamless the integration. While issues are (hopefully) rare and able to be resolved quickly, adequate support can help ease the burden when troubles do arise. Technical support can come in many forms, and management should monitor how often technical support is needed, as well as the effectiveness, efficiency, and type of support provided. Doing so can provide crucial insight into the success of the tool and the needs of the users. It could also help avoid users becoming disenchanted with the tech as a result of receiving little to no constructive assistance or suffering long delays when trying to resolve issues.

How’s It Performing?

Performance-based metrics are commonly used to evaluate technologies across all industries, and having core metrics in place — those used as a way to benchmark progress over time — are always a good idea. Core performance-based metrics may focus on time spent on higher- and lower-level tasks, average time to complete tasks, volume of work completed, and quality of work performed.

But organizations should also have metrics tailored to their specific tools. For example, drafting tools should be evaluated on metrics such as the accuracy of the language, whether the tech includes natural language processing, and how much of the drafting process the tool is able to automate.

So … What’s the Problem Again?

Organizations should periodically reassess their technical needs to see if those needs still exist.

And they should reassess their technical tools to determine if those tools are still meeting their needs.

So back to the beginning they go.

Bloomberg Law subscribers can find related content on our In Focus: Legal Technology resource.

If you’re reading this on the Bloomberg Terminal, please run BLAW OUT <GO> in order to access the hyperlinked content.