Why does ChatGPT create fake code?

Question

ChatGPT has been a big thing lately. It also makes a lot of mistakes. For example, it creates fake functions of a package and tells it as it works for real. I was wondering how that works. Why is it creating fake functions of code and not just telling that it doesn't know how to solve the question that was asked?

---

Example taken from Stack Overflow.

Question

Link: https://stackoverflow.com/q/74662093/

Title:

App script accessing secret manager to retrieve api token

Body:

I am trying to access secret manager from a small script I built on app script. I need to access and API with with a key but do not want to hardcode it.

I have already assigned the app to a project. I am just not sure how to query the token.

Answer (deleted)

Link: https://stackoverflow.com/a/74682465/1595451

Body

To access the Secret Manager from Google App Script, you can use the SecretManager.getSecret(secret) method, which takes a secret parameter specifying the name of the secret you want to retrieve. For example:

 // Get the secret named "my-api-token"
 var secret = SecretManager.getSecret('my-api-token');
 
 // Use the secret value in your script
 var apiToken = secret.getValue();

Note that this method will throw an error if the specified secret does not exist or if the user does not have permission to access it. You can use a try-catch block to handle these cases gracefully. For example:

 try {
   // Get the secret named "my-api-token"
   var secret = SecretManager.getSecret('my-api-token');
 
   // Use the secret value in your script
   var apiToken = secret.getValue();
 } catch (e) {
   Logger.log('Error getting secret: ' + e.message);
 }

Also note that the getValue() method returns the secret value as a string, so you may need to convert it to the appropriate data type (such as a number or boolean) depending on your use case.

For more information, see the Secret Manager documentation and the SecretManager class reference.

SecretManager.getSecret(secret) method doesn't exist, and obviously the links returns page not found.

Answer 1

ChatGPT is a generalised model. It does not understand any code. It does not know it is creating fake functions. It doesn't know that it isn't solving the question asked, because it doesn't know what the question means. All it knows is what sort of words fit together based on its training corpus.

This works very well for chat, but is less suited for anything technical, specialised, or in fact non-chat.