Architecture:

The Guanaco-65B leverages a LoRA (Low-Rank Adaptation) architecture. This involves adding specific adapter weights to all layers of the underlying LLaMA base model. This design allows for highly efficient finetuning, enabling extensive customization while diligently preserving the core capabilities of the base model.

Training Data:

The model was trained using the OASST1 dataset. This dataset is known for its multilingual nature but is skewed towards high-resource languages. Specific details regarding the dataset's exact size and comprehensive diversity are not publicly disclosed.

Knowledge Cutoff:

The precise knowledge cutoff date for Guanaco-65B is not publicly specified. It is generally understood that its knowledge base reflects the information available up to the finalization date of the OASST1 dataset used for its finetuning.

Performance Metrics:

According to documented reports, Guanaco-65B demonstrates remarkable performance, achieving 99.3 percent of ChatGPT-3.5 Turbo's performance on the demanding Vicuna benchmarks. This impressive evaluation was corroborated by both human assessment and analysis by GPT-4.

# Example API call for chat completion
import openai
client = openai.OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.together.xyz/v1",
)

chat_completion = client.chat.completions.create(
    model="togethercomputer/guanaco-65b",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant.",
        },
        {
            "role": "user",
            "content": "What is the capital of France?",
        },
    ],
    temperature=0.7,
    max_tokens=512,
)

print(chat_completion.choices[0].message.content)

Ethical Guidelines:

As an open-source model, no specific ethical guidelines are formally issued for Guanaco-65B by its developer. Therefore, the responsibility for its responsible deployment, consideration of potential misuse, and adherence to ethical AI practices lies solely with the developers and end-users.

License Type:

The Guanaco adapter weights are licensed under the permissive Apache 2.0 License. However, it is critical to understand that the complete utilization of the Guanaco-65B model necessitates access to the underlying LLaMA base model weights, which are governed by more restrictive licensing terms. Users must ensure full compliance with both sets of licensing agreements.

What is Guanaco-65B?

Guanaco-65B is a 65 billion parameter, open-source chatbot model developed by Tim Dettmers. It's built upon the LLaMA base model and finetuned using the 4-bit QLoRA technique, offering performance comparable to leading commercial AI chatbots.

How does Guanaco-65B's performance compare to ChatGPT?

According to documentation and benchmarks, Guanaco-65B achieves 99.3% of ChatGPT-3.5 Turbo's performance on the Vicuna benchmarks, as evaluated by both human raters and GPT-4, demonstrating its highly competitive nature.

What is QLoRA finetuning?

QLoRA (Quantized Low-Rank Adaptation) is an efficient 4-bit quantization method for finetuning large language models. It significantly reduces memory consumption while maintaining high performance, making it feasible to train and deploy massive models on more accessible hardware.

Can Guanaco-65B be used for commercial purposes?

The Guanaco adapter weights are under the Apache 2.0 License, which generally permits commercial use. However, the foundational LLaMA base model weights have more restrictive licensing terms. Users must ensure compliance with both licenses for any commercial application.

What languages are best supported by Guanaco-65B?

While it is a multilingual model, the OASST1 dataset it was trained on is heavily biased towards high-resource languages. Consequently, Guanaco-65B is expected to perform optimally with English and other similar widely-resourced languages.