If I asked you, “When someone turns in a work assignment, how accurate is it? 80%, 90%, 95% or perhaps 100%?” We don’t think this way about coworkers’ spreadsheets. But we will probably think this way about AI & this will very likely change the way product managers on-board users. When was the last time you signed up for a SaaS & wondered : Would the data be accurate? Would the database corrupt my data? Would the report be correct? But today, with every AI software now tucking a disclaimer at the bottom of the page, we will be wondering. “Gemini may display inaccurate info, including about people, so double-check its responses” & “ChatGPT/Claude can make mistakes. Check important info” are two examples. In the early days of this epoch, mistakes will be common. Over time, less so, as accuracies improve. The more important the work, the greater peoples’ need to be confident the AI is correct. We will demand much better than human error rates. Self-driving cars provide an extreme example of this trust fall. Waymo & Cruise have published data arguing self-driving cars are 65-94% safer. Yet, 2/3 of Americans surveyed by the AAA fear them. We suffer from a cognitive bias : work performed by a human is likely more trustworthy because we understand the biases & the limitations. AIs are a Schrodinger’s cat stuffed in a black box. We don’t comprehend how the box works (yet), nor can we believe our eyes if the feline is dead or alive when we see it. New product on-boarding will need to mitigate this bias. One path may be starting with low-value tasks where the software-maker has tested exhaustively the potential inputs & outputs. Another tactic may be to provide a human-in-the-loop to check the AI’s work. Citations, references, & other forms of fact-checking will be a core part of the product experience. Independent testing might be another path. As with any new colleague, the first impressions & a series of small wins will determine the person’s trust. Severe errors in the future will erode confidence, that must be rebuilt - likely with the help of human support teams who will explain, develop tests for the future, & assure users. I recently asked a financial LLM to analyze NVIDIA’s annual report. A question about the company’s increase in dividend amount vaporized its credibility, raising the question : is it less work to do the analysis myself than to check the AI’s work? That will be the trust fall for AI. Will the software catch us if we trust it?
Henrik Fabrin
2mo
These two paragraphs are super important for understanding where we are today: 1) "In the early days of this epoch, mistakes will be common. Over time, less so, as accuracies improve." 2) "We suffer from a cognitive bias: work performed by a human is likely more trustworthy because we understand the biases & the limitations." P.S. I didn't know Schrodinger’s cat was so adorable.
Mostafa Megahid
2mo
Agreed 100%. It's going to be a long way to trust AI to be the main driver and decision maker rather than being a handy co-pilot.
I have also raised this question. Great discussions generate improvements in these systems!
Luke Shalom
2mo
We need good strategies to make sure AI works right and users trust it.
Yulia Pukhova
2mo
Love the parallel with humains - a great way to be more tolerant towards new AI technologies . Like any person starting a new job, one should expect errors, misses, under deliveries in the first months, then a learning curve occurs and the results can go up to 100% and over . Obviously the ramp up phase is quicker if the person has prior experiences. And like with a new colleague who is different from others, we all need to learn to work together in a productive way (balancing profit from time gain versus controlling time lost and AI cost)
Adam Smith
2mo
Which financial analysis AI did you use? I built a version for myself that has an accuracy linter and it works well.
James Meaden
2mo
"is it less work to do the analysis myself than to check the AI’s work?" - great point
Dan S.
2mo
Having the ability to 'double check' with a human when less sure vs always being confident will be a big milestone for AI systems building trust. People trust other people to admit when they don't know something important and to learn from their failures. We have spent hundreds of years building guard-rails preventing human caused catastrophic failure and have some well defined ways to verify a person being 'credible'. AI can fail at the scale of traditional computer systems but without the determinism of processes and data that make guard rails easy to implement. None of our human credibility signals work with an AI bot. And right now our AI systems don't even know how confident to be in their answers, so how could a human?
CEO | Applied | 2x VC backed founder
2moThe more risk in the use cases. The higher the accuracy needs to be. If it’s healthcare its 99.99%. If it’s deciding who gets the job, it should still be 99.9%. Hard to build unless systemised and trained for the sole purpose of those uses cases. And we all know algorithmic aversion is real. Really curious how this lands with users - ultimately will users care and by how much ?