ChatGPT-like chatbot

ChatGPT-like chatbot:

• tloen/alpaca-lora
• Model Card: tloen/alpaca-lora-7b
• Hugging Face Demo: Alpaca-LoRA-Serve
• ChatGPT-style interface: Alpaca-LoRA-Serve
• Alpaca Dataset: AlpacaDataCleaned

Setup

Clone tloen/alpaca-lora

Install dependencies using pip install -r requirements.txt

Training

Fine-tune the script to run on the LLaMA model using the cleaned Stanford Alpaca model. You can look at the repository to tweak the hyper parameters for better performance.

python finetune.py \
    --base_model 'decapoda-research/llama-7b-hf' \
    --data_path 'yahma/alpaca-cleaned' \
    --output_dir './lora-alpaca'

Inference

The inference script reads the foundation LLaMA model from Hugging Face and loads LoRA weights to run a Gradio interface.

python generate.py \
    --load_8bit \
    --base_model 'decapoda-research/llama-7b-hf' \
    --lora_weights 'tloen/alpaca-lora-7b'

You can

• use alpaca.cpp for running alpaca models on CPU or 4GB RAM Raspberry Pi 4.
• use Alpaca-LoRA-Serve to create a ChatGPT-style interface.