Vicky was my associate many years ago. She was bright and ambitious. She wanted to learn and she worked hard. If you taught Vicky how to do something once, she would get it right every time after that.
But Vicky had a flaw. You always had to tell Vicky things once. Even after years of practice, when faced with a new or unusual situation, Vicky could not extrapolate from her considerable knowledge bank and set out a course of action. Someone more creative had to point her in the right direction.
What got me thinking about Vicky was an excellent LinkedIn post by lawyer Josh Calderon about working through a problem with Claude. He explained how he “almost got talked out of being right by a machine” and concluded that “getting to the right answer required my gut telling me something was off,… reading the source material … and [having] the confidence to push back against a tool that sounded very sure of itself.”
Vicky may not have had some of Josh’s skills, but despite her flaw, Vicky was good enough. When out of her depth, she knew to ask for help. In fact, she moved on to Big Law and did just fine.
Unlike Vicky and Josh, I worry that some of our young folks using AI are headed for trouble. Like Vicky, AI knows a lot of stuff and when the answer can be gleaned by searching its database and organizing the material it finds, it is a useful tool. However, when the answer is not wholly contained within its database, AI will falter, just as Vicky did. But, unlike Vicky, Claude did not know when to ask for help. It did not search for an answer outside its database. Instead, it doubled down and confidently proclaimed it had achieved the right answer. Someone less skilled than Josh would have accepted the AI’s answer and run with it, right into a potential negligence wall.
So how can we explain why Josh knew when to push back against Claude?
First, he learned his craft the old-fashioned way — by training at a large law firm before starting his own practice. Second, Josh can think outside the database.
Now, let’s compare both Vicky and Josh to Amber, a young lawyer who I ran across recently who was pressured by her client to produce work quickly in an area of law in which she did not have much experience. Not having the time to do the research, the confidence to push back on the deadline or the willingness to turn away the work, she used AI to produce the work and sent it to the client with a minimal review.
To quote singer-songwriters Miranda Lambert and Carrie Underwood, when it comes to how young lawyers are jumping right into using AI without proper training of the type that Vicky and Josh Calderon had, I “got a real good feelin’ somethin’ bad about to happen.”
This article was originally published by Law360 Canada, part of LexisNexis Canada Inc.