The hype surrounding AI chatbots like ChatGPT has grown rapidly over the last few months, but less attention has been paid to the flaws of AI systems. That changed last week when Google announced Bard, their version of ChatGPT, but failed to factcheck its output. This was a costly oversight, as their announcement tweet for Bard included a video where the chatbot gave an incorrect answer to a simple question. Soon after, $100bn was wiped off of Google’s market value.
Google Bard's big error. The first exoplanet was discovered in 1992, and more than 5300 have been identified since.
How did such a powerful piece of software make such an expensive error? Well, the cause is not unique to Google’s code, it’s a fundamental issue with all AIs that use the cutting edge techniques of the moment, Large Language Models (LLMs), and that includes Google’s competitors such as ChatGPT.
DR. SEUSS’S BRAIN TRAINING
In short, the way LLMs work is by first analyzing large amounts of text to see how often certain words or phrases are found near each other, and then using that information to make assumptions. Those assumptions are then tested against the original data to see how successful the AI was in mimicking the source. The AI then attempts to improve its assumptions using the knowledge it gained in the previous attempt, and this process is repeated until it is confident of its answer. This is known as “training”.
This story is part of our weekly briefing. Sign up to receive the FREE briefing to your inbox.
The hype surrounding AI chatbots like ChatGPT has grown rapidly over the last few months, but less attention has been paid to the flaws of AI systems. That changed last week when Google announced Bard, their version of ChatGPT, but failed to factcheck its output. This was a costly oversight, as their announcement tweet for Bard included a video where the chatbot gave an incorrect answer to a simple question. Soon after, $100bn was wiped off of Google’s market value.
Google Bard's big error. The first exoplanet was discovered in 1992, and more than 5300 have been identified since.
How did such a powerful piece of software make such an expensive error? Well, the cause is not unique to Google’s code, it’s a fundamental issue with all AIs that use the cutting edge techniques of the moment, Large Language Models (LLMs), and that includes Google’s competitors such as ChatGPT.
DR. SEUSS’S BRAIN TRAINING
In short, the way LLMs work is by first analyzing large amounts of text to see how often certain words or phrases are found near each other, and then using that information to make assumptions. Those assumptions are then tested against the original data to see how successful the AI was in mimicking the source. The AI then attempts to improve its assumptions using the knowledge it gained in the previous attempt, and this process is repeated until it is confident of its answer. This is known as “training”.
A quick example: An AI is tasked with guessing the final word of the phrase “The cat sat on the ___”. It has various techniques at its disposal to draw conclusions about the missing word, and its extensive knowledge of song lyrics makes it a dab hand at spotting rhyming pairs. It then thinks about the various words that rhyme with cat, seeing which - if any - are appropriate in this situation. After a brief moment wondering whether a cat might sit on a rat, it hits on the correct answer; this cat is sitting on a mat.
But what happens when the AI’s initial assumption is wrong? Perhaps its first port of call for a likely rhyming answer is not the canon of popular music lyrics, but the works of Dr Seuss? Well, now the cat is probably sitting on a hat, and a quick glance at the internet confirms that cats are likely to sit on just about any inanimate object, including hats. Our AI feels confident that it has found the right answer!
A psychedelic cat sitting - correctly - on a mat (created by the Midjourney image generation AI).
THE MURKY REALM BETWEEN INPUTS AND OUTPUTS
The above example is, of course, heavily simplified, but it is not too far removed from the way the autocomplete function on your phone’s keyboard works. For larger AI systems such as the LLMs that power ChatGPT and Bard, the process is substantially more complex, to the point where it is effectively impossible for a human to know precisely what’s happening within the confines of the AI’s “mind”.
This is by design, as these AIs have been purposefully created using what is known as a “black box model”, where the inner workings of the algorithm are unknowable, because they involve a large number of randomized calculations. As in the example above, an AI will frequently have to choose between two assumptions with similar chances of being correct, and these coinflips quickly become stacked on top of each other as the AI refines and improves its assumptions based on its testing of the results. Eventually, the AI should arrive at the correct conclusions, but exactly how it reached them remains a secret, sealed within the black box of its mind. For the human operators, this is not important; as long as the inputs (the data the model is given to train on) produce the desired output (the answer the AI gives), what happens in between is a secondary concern.
Black box models can be very powerful, and they are not exclusive to artificial intelligence; they are also used in finance, math, and many other walks of life. While powerful, the lack of transparency means that when the AI arrives confidently at a wrong answer - such as Google Bard’s expensive blunder - it’s not easy to understand how it happens, much less how to fix it. Scientists refer to these blunders as “hallucinations” and they have identified a great many potential causes - largely to do with the quality of the inputs - but understanding how these combine to cause hallucinations is a near-impossible task, and it’s only getting harder as the AIs grow more complex, and the datasets they are trained on grow larger.
TOWARDS A GLASS BOX MODEL
There are other issues with black box models besides hallucinations. One of the major concerns is that a lack of transparency makes it difficult to regulate biases that the AI picks up from its inputs. One of the most striking examples of this is the well known phenomenon of facial recognition AIs struggling to identify black faces (and black women in particular). Other problems of bias against minorities occur with AIs employed to filter job applicants, and similar tasks.
To tackle this problem, there is a concerted ongoing effort to produce AI systems where the inner workings of the AI’s algorithm are transparent and fully understandable. These “glass box” AIs are substantially less powerful than their black box counterparts, but it is also possible to examine their processes and eliminate discrimination and other forms of bias, making them suitable for limited tasks (such as filtering job applicants without opening employers up to potential lawsuits). How long it will take for these more transparent techniques to catch up with the black box AIs remains to be seen.
This story is part of our weekly briefing. Sign up to receive the FREE briefing to your inbox.
Gadgets & Gifts
Put your spy skills to work with these fabulous choices from secret notepads & invisible inks to Hacker hoodies & high-tech handbags. We also have an exceptional range of rare spy books, including many signed first editions.
We all have valuable spy skills - your mission is to discover yours. See if you have what it takes to be a secret agent, with our authentic spy skills evaluation* developed by a former Head of Training at British Intelligence. It's FREE so share & compare with friends now!
