Knowledge management shouldn’t start with a demo

The success of knowledge management systems is wider than just turning on a system (or demo). It’s about the resource you have and how knowledge management affects your business for the better, says Jack Shepherd, legal practice lead at iManage.

Lots of conversations around a new knowledge management system start with the question, “can we get a demo?”. I get it. The problem is that knowledge management isn’t an area driven purely by tech – it’s also driven by the knowledge you have. The tech is a vehicle to exposing that knowledge. It’s the same with document automation and contract lifecycle management – you can buy all the tools you want, but unless they are exposing well-organised, clean content, you won’t get much success.

I often use the “garden centre” analogy to explain this. Last weekend, I went to the garden centre to buy an amazing ornamental pineapple plant, only to stick it on the windowsill at home and watch it die in a matter of days. It looked great in the garden centre, but at home I simply didn’t have the right soil, environment or willpower to sustain it.

The success of knowledge management systems is wider than just turning on a system (or demo). It’s about the resource you have, how better knowledge management can improve your business, and what people’s cultural attitudes to it are. As far as knowledge management systems go, you will only get much value out of them if you are prepared to invest the time in building a base of high-quality and well-labelled knowledge assets.

Don’t get me wrong – it is vital for you to see how a knowledge management system works in action. But you must do so as part of a wider discussion around what knowledge you actually have and how it is shared and stored.

See no tech, hear no tech

In a recent study released by Thompson Reuters, more than three-quarters of respondents who were clients of law firms said they “were not aware of any firm with whom they work actually adding value by using technology.”

The State of the UK Legal Market Report then goes on to say that while many law firms have invested in technology, partners aren’t often aware of what’s available to them or how to pass the benefit on to clients.

That’s not hugely surprising when you think of the implications of time-saving tech on an industry run by billable hours. It’s important for law firms to distinguish between client-facing tech, i.e. things clients access themselves, and under-the-hood tech, where clients experience the impact but not the tech itself. Sometimes the former can lead to innovation theatre rather than value, and the latter requires difficult discussions around business models and ROI.

The success of AI is in the information we choose

During the recent CLOC Global Institute conference, I had the opportunity to witness Mary O’Carroll deliver a compelling presentation on the transformative power of generative AI within the corporate legal sector.

While I agree that technology possesses the potential to propel us beyond the arduous limitations of our existing tech landscape, I’d stop short of being convinced that document management and knowledge management systems will be a casualty of the AI revolution.

Why? Large language models produce impressive output, but they don’t do it in a human-like way. These models form sentences based on “most likely next word” and do so in a random way. Humans, on the other hand, reason using learned principles, experiences and tacit knowledge that cannot be reduced to text.

My hypothesis is that large language models have their limits because they are not reasoning like a human would. In the same way, of course, humans have their limits because they are not reasoning like a large language model would. This is not necessarily a question of “which is better”, but more, “which is better for the task at hand”.

The success of AI tools is enhanced by the data you put in. Knowledge management is crucial in choosing and organising that data. Therefore, the role of knowledge management might become more focused on streamlining knowledge workflows and identifying the points where human knowledge is needed, diverting time away from exercises that are better performed by generative AI. For example, trawling through documents to spot trends and producing playbooks from contract examples might be best suited practical use case for AI and automation, perfectly equipped for sifting through large datasets. But AI is not replacing humans. It is acting in partnership. Anybody who thinks humans can delegate all knowledge activities to AI is not living in the real world.

On the flip side, strategic decisions will still need the human touch, leveraging knowledge that takes real-world experiences into account. The type of knowledge that cannot be reduced into neatly consumable documentary format for AI to ingest and regurgitate.

One final point – we are all very aware of the risks around AI “hallucinating” (i.e. making mistakes). Adding domain-specific data into the equations is one way of mitigating these risks. How do you deliver that domain-specific data? Knowledge management is a great way…

Knowledge management is changing, it’s not being eliminated. In fact, it’s becoming more important than ever.