I spent today catching up on AI news, and reading about Meta’s new open-weights model Llama 3. Note the use of the word “open-weights” rather than “open-source”, since Llama 3 has some restrictions on use.
This is an impressive release by Meta. They trained the model on 24,000 GPUs using 15 trillion tokens of data (GPT3 was trained on 300bn) and some of the evaluation metrics look strong.
Tomorrow I’ll do some research on how to run open-weight models either locally or in the cloud.