Meta recently unveiled its latest artificial intelligence models, Llama 3 8B and 70B, which are part of the Large Language Model Meta AI series. These new models showcase improved capabilities over their predecessors, with the company implementing new training methods to enhance the efficiency of the models. Interestingly, Meta has increased the parameters in its large models significantly, with the new models containing more than 400 billion parameters compared to the previous largest model of 70B.
One of the key highlights of the launch is Meta’s community-first approach with the Llama 3 models. The company has made these new foundation models open source, similar to its previous models. This move aims to encourage collaboration and innovation within the AI community. Meta has also announced that the Llama 3 models will soon be available on various cloud, hosting, and hardware platforms, including AWS, Google Cloud, IBM WatsonX, and more. This broad availability makes it easier for enthusiasts to access and experiment with the AI models.
In addition to the open-source availability, Meta has integrated the Llama 3 models with its own Meta AI platform, allowing users to access the models via Facebook Messenger, Instagram, and WhatsApp in supported countries. The company has also shared benchmark scores for the Llama 3 models, showcasing their performance in both pre-trained and instruct models. The pre-trained models of Llama 3 70B outperformed Google’s Gemini 1.0 Pro in various benchmarks, highlighting the capabilities of the new AI models.
Meta has opted for a decoder-only transformer architecture for the Llama 3 models, making several improvements over the previous models. The new models feature a tokeniser with a vocabulary of 128K tokens and adopt grouped query attention (GQA) to enhance inference efficiency. GQA plays a crucial role in improving the attention of the AI models, ensuring they stay within the designated context when answering queries. Meta has pre-trained the Llama 3 models with over 15T tokens, sourced from publicly available data sources.
Overall, Meta’s introduction of the Llama 3 AI models represents a significant leap in the field of artificial intelligence. With enhanced capabilities, new training methods, and improved efficiency, these models are set to drive innovation and collaboration within the AI community. By adopting a community-first approach and making the models open source, Meta is paving the way for broader accessibility and experimentation with AI technology. The integration with Meta AI and the impressive benchmark scores further solidify the position of the Llama 3 models as a game-changer in the AI landscape.
Leave a Reply