Since the leak of Meta’s Large Language Model (LLM) Llama in February, it has become apparent that there are two paths for the future of AI. One path is taken by OpenAI, which offers AI models to businesses at a high cost, while the other is Meta’s approach. Meta aims to build a more user-friendly and accessible model, and offers Llama 2 for free.
Llama 2 is an improved version of Meta’s LLM, with three different model sizes: 7 billion, 13 billion, and 70 billion parameters. In comparison, Google’s chatbot, Bard, is based on the PaLM 2 model, which has been trained on 340 billion parameters.
Llama 2 is free for both research and commercial purposes, making it a great option for smaller entities and researchers. It has been trained on 40% more data than its predecessor, with a total of 2 trillion tokens. Tokens are important for training LLMs as they help predict the next word in a sentence.
In terms of performance, Llama 2’s 70-billion parameter model is comparable to GPT-3.5 in language capability and math solving benchmarks, but falls behind in coding parameters. Llama 2 also facilitates fine-tuning, making it easier for researchers to improve its coding capabilities.
However, it is worth noting that Llama 2 does not meet the technical definition of open-source according to the Open Source Initiative (OSI). Meta imposes restrictions on using Llama 2 to train other models, as well as a special license requirement for apps or services with more than 700 million monthly users. While this may contain competition, Meta presents itself as “open” to gain favor with independent researchers.
To address concerns about the environmental impact of LLMs, the Llama 2 paper includes a section on the carbon footprint cost of training, which is a first.
Overall, Meta’s Llama 2 offers a more accessible and transparent alternative to OpenAI and Google’s LLMs, with its free availability and easy fine-tuning capabilities.