How to think with AI?
A starting point - what are the core principles of my process
Number of toddler/baby interruptions while writing this: 3
As a lot of my colleagues in other professions increasingly worry about AI (there will be a post about that) - and as I observe industries that I think have achieved LLM product-market-fit (there will be a post about that) - it makes me think a lot about the societal implications of the current wave of AI build (there will be….you know what, there will be posts, let’s leave it at that).
I wanted to start, however, by outlining how I think a person should be thinking about AI1, what are some of the core principles that should help any business leader, person, enthusiast - think about AI applications, and help you understand whether you’re being sold a gimmick or a real thing. That last one I find increasingly important as the amount of AI companies proliferates, and grifters catch on to the wave.
It’s worth noting that I don’t pretend any of this is revolutionary. It’s the same stuff regurgitated over and over by AI thinkers far cleverer than me, but as I’m trying to do this within my own process - I find it important to state clearly the fundamentals.
1. Data
“AI lives downstream of your data” is the core principle I subscribe to. You’ve probably heard it described as “Garbage in - garbage out” many times - but I found that it doesn’t 100% click with people why that is.
Enter the human metaphor.
Humans don’t operate well when there’s no data. Imagine a human without any senses - no sight, no smell, no touch, no taste and no hearing. This human would be living a fairly limited reality - and their ability to affect the world would be extremely limited. AI is no different - AI systems are nothing but information processors - they need information to be able to make determinations.
That part is relatively trivial - but where people often trip up is that when data does exist - they think any data is sufficient. This is because the homo sapiens is an extremely efficient data cleanser. Our brains have evolved over thousands of years to filter unwanted sensory input and to clean signal towards the objectives our bodies need to achieve. Unfortunately, decades of anthropomorphising AI have ingrained, quite deeply, the expectation that AI systems do, in fact, behave like humans. They don’t.
Noise cleaning is one part of it - the other is adding additional context. Human knowledge is by definition interconnected in the brain - everything a human knows influences everything else a human knows and perceives. Data that we feed the AI should be the same - you can’t expect things that might seem obvious or common sense to have reached the AI - it will only be there if it was in the training data2.
This is still in the abstract, so let’s take it to a business example.
You might be pointing your AI system at the transactional data of users on your platform (let’s say, a bank) - and telling it to detect fraud. Your AI system happily works away and flags transactions that feel weird. And then it starts blocking all house purchases of your customers. It sees a large transaction - larger than anything previous on the account, and it just won’t let it through. A human might look at that and wonder what the problem is - it’s clear that it’s a house transaction, and house transactions are normal things that have been going on for decades.
But what the human is doing there is introducing extra context, that has not been in the training data. The training data was just a list of transactions - without annotations as to the different types of transactions, and so the system just learned that a large transaction that is far away from the average needs to be blocked, because it’s seen an equal amount of house purchases and fraudulent transactions3.
To have good AI - it needs good data, that is as informationally rich as possible for the objective it’s trying to achieve.
2. Objectives
I found this often conflated in the business world. It’s an easy one to conflate, but very damaging when conflated. A winning combo.
I find this whole problem to be equivalent to trying to explain to an entry-level intern that he needs to write copy in a way that will achieve the company goal of increasing revenue by 10% YoY. Wouldn’t that be a bit mental? Typically in organisations we have different goals at different levels and we trust that these components working in unison will deliver company goals. AI is no different.
AI are systems that pursue objectives - they get information on one end and they optimise against that information in order to achieve … something4. What that something is, is of extreme importance - as that is the only thing that determines what your AI system actually does5.
In case of the current AI darling in the out-there-sphere, the LLM6, the objective is to predict the next token in a string of text. That’s technical mumbo-jumbo that can be boiled down to - it predicts the next word in a sentence. It is remarkably good at that. And that, is remarkably similar to how the human brain works, but not quite7. If you believe that it is similar to the human brain - it might make you think that this thing that was created can just plug into processes designed for humans and achieve the same objectives as humans, just faster. And this is where people can trip up.
When you think about a successful human in an organisation there will be an objective that they’re trying to achieve. A marketer might need to achieve a good cost per acquisition; while an engineer might need to achieve a low cost to run their bit of infrastructure. Typically, ability to achieve the objective will be correlated with their overarching cognitive ability (let’s call it intelligence, for laughs). What that means is that people who have higher cognitive ability would be better at achieving those objectives.
But that is human cognitive ability. LLM doesn’t have cognitive ability, and applying a human lense to the LLM might trip us up to think it has. The reason we might think that an LLM has cognitive ability is that it is incredibly generic and can do many things because of it - which makes sense - because language is incredibly generic - and they are good at language. But really, they’re good at producing the next word in a sentence. The more your business problem objective is correlated to that specific thing they are good at - the more likely that the LLM will be able to bring you value in your use case8.
So to have good AI use cases - you need a very good understanding of your business objectives and how they relate to objectives you might point AI at.
3. Failure management
AI systems9 are non-deterministic complex systems, which is a fancy way of saying - you can’t easily describe how they are going to arrive to an outcome (complex) and a lot of them won’t arrive at the same result twice from the same set of inputs (non-deterministic). They’re not a series of IF…THEN statements, they’re not a decision tree or a flow diagram you can follow.
This means you have to be comfortable that every AI system will … fail. It might fail 1 every 100 times, or it might fail 1 every 1 000 000 000 times, but without doubt, it will fail. That shouldn’t scare you. Do you know what else is a complex and non-deterministic system that fails a lot? Homo Sapiens10.
Humans have been failing at stuff for so long, we’ve got a thousands of years old saying for it — “To err is human”. How do we deal with that in the division of human labour? We either build systems of control over it (limit decision-making ability) or we live with the failure (the penalty of failure is low). Typically, it’s the combination of the two that ensures that each human has a scope where we can tolerate the failure-rate.
Turns out - that’s exactly how you manage AI systems. Either you can live with the penalty (the failure rate) - in which case - great, go off and fly with it; or you introduce systems of control that allow the AI to operate within a margin of safety that the business is comfortable with.
The difference with AI and human failure is that AI fails at scale. If a human employee mislabels a mortgage application at a rate of 1/100 - over their work week there might be one risk exposure arising from it. If an AI that serves millions of customers does it - that’s potentially harmful at company-closing scale.
But as long as we know that AI fails - we can either control it, or live with it. For that to be possible - the system needs to either have a consistent and stable failure rate; or we need to monitor the failure rate at an ongoing basis11. We typically do that with either human quality assurance on a sample (where the “truth” isn’t directly available); or we read the truth from elsewhere in the system (for a marketing optimising AI, a sale could be an indicator of “success” - i.e. the truth).
Knowing that AI will fail, allows you to build it safely. And that’s your final component of a successful AI system.
4. Bonus Round: Allowing the human control
Danger!! We’re moving into product strategy territory!
There is another consideration that I think about deeply when thinking about how to get people to actually engage with and use AI systems; but I don’t think they are important for the AI system to be capable and useful. I think it’s important to be able to get demand and usage of the system - which is equally important to achieve your business objective - as the best AI system in the world is useless if your customers or your colleagues won’t come anywhere near it.
In my mind - that component is control. Humans are a tricky bunch, but what really grinds the gears of humans is someone telling them what they need to do and not allowing them to disagree. We’ve had all kinds of fun episodes in history of how that turns out.
It might go counter the objective that the AI is trying to achieve - but you have to give the humans the ability to disagree and take over - both because that failure rate will mean at some point the human will be right; but more fundamentally - because humans are stubborn animals. Creating friction reduces the probability of repeat usage - and tends to get people to remember that bad outcome; rather than all the value they got before.
Allowing the human control gives you great built in feedback - directly available as data as well - which you can use to further optimise your AI system - not for outcomes, but for usage - and in a business setting - it’s the usage that matters.
If you got to this point - hope you enjoyed that. It is pretty well circulated stuff - but it helps me position some of the thinking that will be coming in later posts.
I would appreciate some feedback - so hit me with how you found it in the poll above; and if you stumbled upon this because you found it somewhere in the interwebs - I’m sorry - and here’s a subscribe button on the off chance you want more.
I would recommend heavily the Citizen’s guide to Artificial Intelligence to prepare yourself for the world we’re rapidly rushing to; if you’re the kind of weirdo that still reads books (I am)
AI practitioners, please let’s leave emergence for another day
AI practitioners, bear with me
In AI parlance, we call this the objective function of the system
This, surprisingly, can get conflated even in the most tech-forward of businesses; with the most tech-forward of employees of those businesses; I’ve found
Large Language Model
More on why word prediction trained from language can’t represent the complexity of human thought prediction - later
And as it turns out, producing workable mediocre code is something that is fairly correlated to predicting another word in a sentence; and workable mediocre code at scale makes economic sense. But more on that another time
Or at least, AI systems in the sense of the word that the current media uses. AI is just autonomous decision-making, and that could be hard-coded through a series of IF…THEN statements.
Free-will vs. no free-will philosophers, don’t come at me, I’m illustrating a point
In ML-speak, we call this drift and quality monitoring