ChatGPT works best with analytic prompts that respond with translations and not synthesis.
Meaning, if all the facts exist in the prompt then the likelihood of synthesizing fiction is diminished.
There are a number of ways to use the principle of analytic augmentation to add most or if not all of the facts required for a truthful response, ranging from simple “prompt engineering” to evaluating code to document embedding in latent space.
For example, if you use prompt engineering to k-shot a task to turn math word problems into executable JavaScript, meaning LLMs are only translators and the computations are done by a software interpreter, then the results are much more likely to be truthful.
Sampling from a number of variations on a prompt can lead to a more accurate outcome if say 1:10 times the translation attempt has a different answer.
Meaning, if all the facts exist in the prompt then the likelihood of synthesizing fiction is diminished.
There are a number of ways to use the principle of analytic augmentation to add most or if not all of the facts required for a truthful response, ranging from simple “prompt engineering” to evaluating code to document embedding in latent space.
For example, if you use prompt engineering to k-shot a task to turn math word problems into executable JavaScript, meaning LLMs are only translators and the computations are done by a software interpreter, then the results are much more likely to be truthful.
Sampling from a number of variations on a prompt can lead to a more accurate outcome if say 1:10 times the translation attempt has a different answer.