Meta Unveils Improved AI Model, Llama 2, to Bolster Chatbot Technology
Meta has released Llama 2, its next-generation, open-source large language model, represented by 7 billion, 13 billion, and 70 billion parameter versions. The company has made this model accessible for research and commercial applications, fostering further development in the field.
Meta is charting new territories in the AI landscape, as it introduces Llama 2, the latest generation of artificial intelligence models set to power modern chatbots. This announcement signals a substantial leap from its predecessor, Llama, providing significant performance improvements and wider accessibility.
Unlike Llama, which was only available upon request, Llama 2 offers unrestricted availability for both research and commercial applications. Thanks to a broadened partnership with Microsoft and Qualcomm, it can now be fine-tuned on AWS, Azure, and Hugging Face's AI model hosting platform. The new model is optimized for Windows, and Qualcomm aims to bring it to Snapdragon devices by 2024.
Llama 2 comprises two variations – Llama 2 and Llama 2-Chat, the latter optimized for two-way conversations. Both iterations come in versions of varying sophistication, represented by 7 billion, 13 billion, and 70 billion parameters, highlighting their advanced learning capacity.
Training was conducted on two million tokens, almost twice as many as the original Llama. While Meta hasn’t revealed specific training data sources, it confirms that the information used is primarily factual, gathered from the web, and largely in English.
Despite an improved performance, Meta acknowledges Llama 2 lags behind competitors like GPT-4 and PaLM 2 in certain benchmarks, including computer programming. However, human evaluators found Llama 2 on par with ChatGPT in terms of helpfulness and safety.
Meta admits to the limitations and biases of Llama 2, such as favoring "he" pronouns over "she" due to imbalances in training data, a western skew, and not outperforming other models in toxicity benchmarks. To counteract, Meta is collaborating with Microsoft to use Azure AI Content Safety to reduce toxic Llama 2 outputs.
“We believe that openly sharing today’s large language models will support the development of helpful and safer generative AI too,” Meta writes in a blog post. “We look forward to seeing what the world builds with Llama 2.”
