I've been working on some exceptionally long LLM prompts for a couple of projects at work. I've noticed a fascinating phenomenon: A prompt that works well with one model can diverge in performance when applied to another. This presents switching costs for developers and businesses. You
AI agents have quickly evolved from obscure technical experiments to mainstream buzz. In the last 18 months, new automation capabilities have captured the imagination of entrepreneurs and established organizations alike. While it’s possible that we’re in another AI bubble, the early adopters of AI agents are reporting impressive
Last week I attended the Algorithmic Innovation & Entrepreneurship Summit on Open Problems for AI. That’s a bit of a mouthful but in essence, it was a 2-day conference covering the challenges and opportunities for AI over the next 5 years. Day 1 was focussed on algorithms and AI
An undeniable killer use case for large language models is code generation. It’s already helping software developers write code up to twice as fast and with more delight. As LLM models improve with each release, they can write better software and handle more complex architectures at a fraction of
AI agents have been getting a lot of attention but they can be expensive to run. This is because they use a lot of tokens and make multiple calls to LLMs. Some AI developers have beat performance benchmarks by using exceptionally expensive agentic runs. However, this is unrealistic because in
I switched one of my AI agents projects from GPT-4o to the new Claude 3.5 Sonnet model. The agents now run with a cost that’s roughly 85% cheaper and the final results arrive about 15% faster. To upgrade my AI agents project, all I had to do was
On the way home today I was listening this podcast — “Lessons from a Year of Building with LLMs“. The biggest takeaway? While everyone has been building lots of experimental projects with LLMs and often shipping to production scrappy builds of seemingly impressive demos (I’m guilty of this!), few people
AI Agents are a mixed bag. I tried this morning to get a group of CrewAI agents to complete some Youtube summaries and they kept failing or going off the rails. My deterministic version which I hard coded works significantly better than leaving things to a group of autonomous agents.
There’s an article in the Wall Street Journal I came across today with the headline below. The article ends with a quote from a Google Cloud Chief Evangelist, Richard Seroter: “If you don’t have your data house in order, AI is going to be less valuable than it
Claude AI is really good at coding. In one shot, I asked it to create streamlit app that I can use to chat to my PDFs with a local, offline, large language model. The first attempt had a roughly working version and with just 2 more follow-on prompts and refinement,
Just 18 months ago, OpenAI released GPT-3.5 Turbo which had double the input token context window of its predecessor, GPT-3. We went from 2048 tokens to 4096 tokens and that felt like a significant leap. But today, we are enjoying context windows of 128,000 tokens with GPT4o. How
A couple of experiments today: I completed a simple tutorial that helps you build a RAG app using local models. I also installed and tested https://openwebui.com/. This is a really cool open source project that gives you a ChatGPT-style interface to use with large language models that run