Intention is a key concept in the philosophy of action, which in turn is a key concept in the concept of a person. In philosophy, only a person can perform an action, or, in philosophy, an action can only be performed by a person. Intention is a key part of this, because I can only intend to do something that I can do, or refrain from doing. (I don't normally intend to trip over my children's toys or lose my car keys.)
This profoundly affects how one might clarify whether LLMs can have intentions, and I'll have something to say about that later.
Intention is one of a number of concepts that are preliminaries of action. It looks to the future. Planning, preparing, practising, rehearsing, scouting, trying (though that typically involves failing, or being at risk of failure), (making a trial run is also a preliminary, but not the same as trying), learning a skill, exercising, are all other concepts of the same kind. However, these are perhaps most often also actions, but intending is not necessarily shown by any overt behaviour; in this, it is like deciding or evaluating or aspiring.
At first sight, one might be inclined to say that when I am carrying out an intention, I no longer intend it. But actions are complicated. When I walk (rather than drive or take the bus) to the neighbourhood shop to buy some beer, I may intend to buy beer, to spend money. I may be walking rather than taking a bus or driving to get some exercise and I may have intended that. But I am already walking and getting some exercise, so it seems inappropriate to say that I intend to do those things. I am already doing them. Yet, I am not unintentionally walking or getting some exercise. I have chosen to do that. Perhaps I should say that I am carrying out my intention to get some exercise by walking to the shop.
(That paragraph is an illustration of the complexity and subtlety of our concepts around actions.)
Intention is like introspective concepts, such as fear or happiness or pain in that I don't need evidence to know what my intentions are. What I say goes. Other people may attribute intentions to me, but get it wrong. But I cannot be wrong about what I intend (though I can, of course, mislead other people). Because of this, intending is, in some ways, like a promise or a commitment - though, of course, it does not involve any moral obligation. This is why intention is not, or not simply, a prediction. When I intend to do something, I will be actually bringing it about; predictions do not in any way suggest that.
All actions have unintended consequences, most of them trivial. When someone is affected by an action of mine, but I did not intend to affect them in that way, I can exculpate myself by pointing out that I did not intend that consequence. That's why the question of intention is often a focus of criminal law. That's why the question of intention is often a focus of criminal law.
Proving intention in a court of law is often difficult, even impossible. But there are ordinary ways in which we work out what someone is intending to do and, while they are not always correct, they are reasonably reliable. Preliminary actions, like planning or preparing are good evidence, and other clues, such as body posture, what is being attended to, and so forth are also helpful. Without going in to the whole questions of our knowledge of other minds, that's about as far as I can go.
On the question of LLMs, it should be clear now that the background that is needed to answer the question about intention is not in place. The concepts of person and of agency do not apply, and it is not meaningful to try to apply a single one - intention - without that background.
On the other hand, we do in some cases apply some of these concepts to machines. Machines are capable of saying things and even of some level of understanding of some of what people say, to some extent; they can weld things and drive vehicles. The counter-argument is that they have been built to imitate those activities as performed by human beings.
But that does mean that those descriptions do apply, though in an extended or metaphorical sense. So I would suggest that it is an over-simplification to say that they cannot perform at least some of the actions that people – human beings – carry out. It is more complicated than that.
But there is no ground for saying that they intend things.
Stanford Encyclopedia - Philosophy of action is a fairly comprehensive overview of this field. Wikipedia - Theory of action is shorter, but, for that reason, less helpful with intention specifically.