Tech
Just 18 months ago, OpenAI released GPT-3.5 Turbo which had double the input token context window of its predecessor, GPT-3. We went from 2048 tokens to 4096 tokens and that felt like a significant leap. But today, we are enjoying context windows of 128,000 tokens with GPT4o. How
Tech
A couple of experiments today: I completed a simple tutorial that helps you build a RAG app using local models. I also installed and tested https://openwebui.com/. This is a really cool open source project that gives you a ChatGPT-style interface to use with large language models that run
Tech
I’ve just finished listening to this podcast with the founders of https://www.factory.ai/. Their startup automates away the drudgery bits of work in software engineering. That includes writing tests, debugging, documentation, and migrations. I found the views of the founders compelling. For example, they don’t think
Tech
I finally have a Youtube video summarizer working! I’ve deployed the app with Streamlit.io. On the backend I’m using the Gemini 1.5 Flash LLM because it’s cheap and fast! The summaries are “chained” together using Langchain. A 20 minute video can be summarized in around
Tech
I’ve been testing out the new Claude LLM and it’s better at coding than GPT4o. I might actually start using it more given how fast and effective it is. You can see the benchmarks below that Anthropic provided. They probably have some bias as they are self-published but
Tech
Today I got a Youtube summarizer script working! I can input a Youtube URL and I get useful summary of the video using the Langchain “refine” documents chain. I really like this because iterates over portions of text and it refines the summary until something more thorough is generated. I’
Tech
This evening I experimented with Langchain’s summarisation frameworks. This is something LLMs are great at natively, but with Langchain, you can use even more sophisticated summarisation techniques. Here’s a GPT-generated summary of these techniques based on the Langchain documentation: From ChatGPT4o: 1. Stuff Method * Concept: Simply concatenate all
Tech
Given the note I wrote yesterday about the need to be “programmatically” lazy, I’m rewriting my Youtube reviewer system. I’m building it initially without agents, and I’ll only add an agentic layer in areas that are more unstructured. I’m not yet sure what areas will have
Tech
The temptation with any new powerful tool is to apply it in more places than is necessary. What’s that saying…”to a man with a hammer, every problem is a nail”? The same appears to be happening with large language model frameworks and AI agent frameworks. I’ve experienced
Tech
One of the pioneers of how large language models are trained and fine-tuned, Jeremy Howard, has a great article here on “What policy makers need to know about AI (and what goes wrong if they don’t)”. The article is actually very accessible to everyday readers. Here’s an excerpt
Tech
I tested out a few AI agents today with the goal of optimising on cost, but they remain expensive to run if you have tasks that involve multi-step reasoning with chunky bits of content. One group of tasks that I gave to GPT4o cost $0.15 and took 2 minutes
Tech
OpenAI launched a feature called “function calling” last year, but it’s a feature I haven’t used and didn’t understand until today. Part of my misunderstanding was because the feature is poorly named. It made me (and many other more technical people) expect that the capability means an