Risky Context
Risky Context
February 11, 2025 at 01:50 AM
Here’s what Dylan Williams have to say (excerpt below from his LinkedIn post). I've been prompting the latest reasoning models (o3-mini, o1, DeepSeek R1) wrong and this was a game changer. After 18 months deep in prompt engineering, its tough to find out a lot of what we've learned is useless 😅 - while we are used to spending hours carefully crafting few-shot examples and chain-of-thought ("think step by step"), these techniques are not only irrelevant for the new models, its actually a hindrance! I wanted to share this because all we've known the past 2 years has been to do everything to get away from zero-shot, but the exact opposite is true now! The latest reasoning models (o3-mini, o1, DeepSeek R1) are not chat models (GPT-4o, Claude3.5). The built-in reasoning capabilities mean many traditional prompt engineering techniques are not applicable and we should treat them as "give me a detailed report" models. We're witnessing prompt engineering being "eaten by the model" - techniques we once explicitly engineered are now baked into the architecture. This is very exciting for a lot of cognitive architectures out there who have implemented techniques like decoupling planning from execution, or adding reflexion steps in their workflows. One constant remains: context is everything. As models evolve, the key challenge is still orchestrating precise, complete context at inference time. Just like as humans, we want all the information available to us at decision time. The future of "prompt engineering" promises to be fascinating. I wanted to share these resources because OpenAI's guide was pretty useless, but the ones below are super detailed and very helpful: 💡 o1 isn’t a chat model (and that’s the point): https://www.latent.space/p/o1-skill-issue 🌊 Prompt Engineering for OpenAI’s O1 and O3-mini Reasoning Models: https://techcommunity.microsoft.com/blog/azure-ai-services-blog/prompt-engineering-for-openai%E2%80%99s-o1-and-o3-mini-reasoning-models/4374010 🛠️ OpenAI reasoning models: Advice on prompting by Simon Willison: https://simonwillison.net/2025/Feb/2/openai-reasoning-models-advice-on-prompting/ 🔎 OpenAI's guide on prompting reasoning models: https://platform.openai.com/docs/guides/reasoning

Comments