Tech

Day 63 of 100 Days of AI

Michael Tefula

May 27, 2024 — 1 min read

I’m now 69% of the way through the AI agents course, and I added a new idea to the list of side projects I’d like to build after the course. I look forward to this because I can already see how agents can automate both personal and work-related tasks.

Big Data is Dead?

I also came across this post today about big data. It’s written by an engineer who worked at BigQuery. The post argues that most organisations don’t need big data solutions (e.g. NoSQL databases like MongoDB or specialist data warehouses like Google BigQuery or Snowflake).

The illustration below from this engineer shows what he remembers from his time working with “big data”. He found that most customers had less than 1 terabyte of data and that among users who spent more than $1,000 a year on BigQuery, 90% of their queries involved less than 100 MB of data.

This begs the question of why many organisations bother investing in big data infrastructure. “Big Data” usually refers to data sizes in the range of hundreds of terabytes (TB) to petabytes (PB). Only a tiny fraction (the top 1%) of organisations actually handle data of this scale.

Open Problems (& Opportunities) for AI: Summary Notes from the Conference

Last week I attended the Algorithmic Innovation & Entrepreneurship Summit on Open Problems for AI. That’s a bit of a mouthful but in essence, it was a 2-day conference covering the challenges and opportunities for AI over the next 5 years. Day 1 was focussed on algorithms and AI

The Software Engineer is Dead, Long Live the Software Engineer!

An undeniable killer use case for large language models is code generation. It’s already helping software developers write code up to twice as fast and with more delight. As LLM models improve with each release, they can write better software and handle more complex architectures at a fraction of

Day 100 of 100 Days of AI 🎉

AI agents have been getting a lot of attention but they can be expensive to run. This is because they use a lot of tokens and make multiple calls to LLMs. Some AI developers have beat performance benchmarks by using exceptionally expensive agentic runs. However, this is unrealistic because in

Day 99 of 100 Days of AI

I switched one of my AI agents projects from GPT-4o to the new Claude 3.5 Sonnet model. The agents now run with a cost that’s roughly 85% cheaper and the final results arrive about 15% faster. To upgrade my AI agents project, all I had to do was

Read more

Open Problems (& Opportunities) for AI: Summary Notes from the Conference

The Software Engineer is Dead, Long Live the Software Engineer!

Day 100 of 100 Days of AI 🎉

Day 99 of 100 Days of AI