The Nature of Large Language Models (#LLM): Discovery or Invention?
Not just a Socratic debate, or philosophical discussion. If LLMs are an invention, then we really should know more about how they work, right? If Discovery, then what else do we need to uncover that is right here in front of our eyes?
Arguments for Discovery:
* LLMs leverage fundamental patterns and structures inherent in human language and knowledge, which we are uncovering rather than creating from scratch.
* The emergent capabilities of LLMs were not fully anticipated or designed, but rather discovered through experimentation. There is an entire discipline needed to communicate with LLMs called #PromptEngineering
Arguments for Invention:
* The specific architectures, training methods, and implementations of LLMs are human-designed , human considered, and prone to human errors.
* LLMs required significant engineering effort and novel technical approaches to create; yes, they are created.
Newton and Leibniz both developed calculus independently in the late 17th century. Was Calculus discovered (as a fundamental mathematical truth) or invented (as a novel mathematical system). A fun diversion to consider this one, no?
Understanding the answer will help us to chart the path forward. However, there may not be an answer, we need to consider that as well. Are LLMs on the path to #AGI ? Suggesting some reasons here, I will be 'out over my skis', and I am a good skier... That said,
* LLMs demonstrate strong zero-shot and few-shot learning abilities, indicating a capacity for generalization that's crucial for AGI.
* LLMs sometimes display unexpected capabilities not explicitly trained for, hinting at the potential for further emergent intelligence.
I am only scratching the surface, barely - I do think about such things, call me strange, bored, or innovative, not sure which...
For me, #GenerativeAI still required Human-in-the-Loop and we are still really early and only at the beginning. We all have a lot to learn. For me, I am going to enjoy the ride!
#Innovation #AI
1
7 Comments