Do you sometimes feel like fighting against your LLM? 🥊 (yes of course you do) Let me introduce you to 𝐀𝐢𝐤𝐢𝐝𝐨–𝐩𝐫𝐨𝐦𝐩𝐭𝐢𝐧𝐠 (paradigm we use at https://jina.ai/). Don’t fight the bias, but use it to your advantage. During my last 4 years of prompt engineering, I sometimes found myself in a situation where the LLM did not want to follow a simple instruction. In the screenshot, you can see how I tell GPT 3.5 to only generate the section content in markdown format, but not the title. For some reason, it completely ignores my instruction and happily generates the title as well. It turns out that even after motivating the model a bit more (almost threatening it), sometimes the generations include a headline at the start. But I want you to remember: "𝐀𝐢𝐤𝐢𝐝𝐨 𝐢𝐬 𝐧𝐨𝐭 𝐚𝐛𝐨𝐮𝐭 𝐟𝐢𝐠𝐡𝐭𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 𝐨𝐫 𝐝𝐞𝐟𝐞𝐚𝐭𝐢𝐧𝐠 𝐭𝐡𝐞 𝐞𝐧𝐞𝐦𝐲; 𝐢𝐭 𝐢𝐬 𝐚𝐛𝐨𝐮𝐭 𝐫𝐞𝐜𝐨𝐧𝐜𝐢𝐥𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 𝐭𝐡𝐞𝐦." - 𝑀𝑜𝑟𝑖ℎ𝑒𝑖 𝑈𝑒𝑠ℎ𝑖𝑏𝑎, 𝑓𝑜𝑢𝑛𝑑𝑒𝑟 𝑜𝑓 𝐴𝑖𝑘𝑖𝑑𝑜 Thinking this way the solution was straightforward. I now explicitly instruct GPT 3.5 to generate the title and when parsing the output I just remove the first line. That way I get "deterministic" behavior from the LLM and handle everything it does not want to do. Another common example is code generation. You might have seen some fencing syntax like triple backticks (```) or some **filename.py** at the start of your generated code. Yes, you can go into endless prompt engineering to generate code without the extra syntax, but you can also just generate it in the way the LLM wants it and have your regex to clean it up in the post-processing step. Since there is a very limited number of instructions your model can follow at the same time, removing unnecessary ones is always a good idea. Let me know about your current fights you have with the LLMs and let's think together about how we can resolve them by thinking the Aikido way.
How do you balance between adapting to the LLM's tendencies and maintaining control over the output format? Are there cases where you've found it's better to "fight" rather than adapt?
Freelance AI Developer and Consultant | Passionate about building innovative AI-powered solutions | not perfect but versatile
1wAlthough you can use LLMs to generate candidates and try and optimize them, I agree it is a fight sometimes. Thanks for the akido prompting technique. It is sometimes difficult to get everything done via a single prompt.