Predicting the Next Word
A language model is trained on one task: given the words so far, predict the next word. The simplest version counts which word follows which in the training text (a bigram model). Chat assistants do the same thing at huge scale, with a neural network in place of counts.
Examplepython
Quick check
In this model, how is the next word chosen?
Your turn: write the code
Print the most likely word to follow 'cat'.
Finish the quick check (and any code exercise) to complete this lesson, or mark it done yourself.
Tip: use the left and right arrow keys to move between lessons.