Study to edit a 7b 7B of qlora using Axlotl with effective LLM training

nimda February 10, 2025

0 6 4 minutes read

Study to edit a 7b 7B of qlora using Axlotl with effective LLM training

In this lesson, we show that the Hard-7b Working Work Using the Qlora with AxlotlDisplays how to handle limited GPU resources while acting in customization of new jobs. We include Axlotl, create a small dataset for example, prepare a special Lora Hyperpareter, use a good layout process, and test the effectiveness of the result model.

Step 1: Prepare the Environment and Enter Axlototl

# 1. Check GPU availability
!nvidia-smi


# 2. Install git-lfs (for handling large model files)
!sudo apt-get -y install git-lfs
!git lfs install


# 3. Clone Axolotl and install from source
!git clone 
%cd axolotl
!pip install -e .


# (Optional) If you need a specific PyTorch version, install it BEFORE Axolotl:
# !pip install torch==2.0.1+cu118 --extra-index-url 


# Return to /content directory
%cd /content

First, we look at what is available to any GPU and how much memory is available. We then install GIT LFS for large files of models (such as 6b 7B) that is not treated properly. After that, we combine the estatil accommodation from GitHub and add it to the “Relevant” mode, which allows us to call its instructions from anywhere. The optional phase allows you to enter a specific pytroch version when needed. Finally, we go back to the bag / content to edit the following files and neat ways.

Step 2: Create a small sample dataset and Qlora Config of Marti 7b

import os


# Create a small JSONL dataset
os.makedirs("data", exist_ok=True)
with open("data/sample_instructions.jsonl", "w") as f:
    f.write('{"instruction": "Explain quantum computing in simple terms.", "input": "", "output": "Quantum computing uses qubits..."}n')
    f.write('{"instruction": "What is the capital of France?", "input": "", "output": "The capital of France is Paris."}n')


# Write a QLoRA config for Mistral 7B
config_text = """
base_model: mistralai/mistral-7b-v0.1
tokenizer: mistralai/mistral-7b-v0.1


# We'll use QLoRA to minimize memory usage
train_type: qlora
bits: 4
double_quant: true
quant_type: nf4


lora_r: 8
lora_alpha: 16
lora_dropout: 0.05
target_modules:
  - q_proj
  - k_proj
  - v_proj


data:
  datasets:
    - path: /content/data/sample_instructions.jsonl
  val_set_size: 0
  max_seq_length: 512
  cutoff_len: 512


training_arguments:
  output_dir: /content/mistral-7b-qlora-output
  num_train_epochs: 1
  per_device_train_batch_size: 1
  gradient_accumulation_steps: 4
  learning_rate: 0.0002
  fp16: true
  logging_steps: 10
  save_strategy: "epoch"
  evaluation_strategy: "no"


wandb:
  enabled: false
"""


with open("qlora_mistral_7b.yml", "w") as f:
    f.write(config_text)


print("Dataset and QLoRA config created.")

Here, we create a small JSON's dataset to point to the Yaml Dictionary pointing to the Mistral 7B model, Set the Qlora Periods of Good Remembrance, and explain training hyperparers such as batch size, learning rating, and length of order, and a length of chronology. We also say the lora settings such as dropout and sign and finally keep this program as qlora_mistsl_7b.yml.

Step 3: Finse-tune with a bartl

# This will download Mistral 7B (~13 GB) and start fine-tuning with QLoRA.
# If you encounter OOM (Out Of Memory) errors, reduce max_seq_length or LoRA rank.


!axolotl --config /content/qlora_mistral_7b.yml

Here, the anxoordl automatically downloads and fetching the unpleasant 7B weights (a large file) and begins the best qlora plan. The model is divided into 4 accuracy, which helps reduce the use of GPU memory. You will see the training logs that illustrate progress, including training losses, step by step.

Step 4: Explore a well-prepared model

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer


# Load the base Mistral 7B model
base_model_path = "mistralai/mistral-7b-v0.1"   #First establish access using your user account on HF then run this part
output_dir = "/content/mistral-7b-qlora-output"


print("nLoading base model and tokenizer...")
tokenizer = AutoTokenizer.from_pretrained(
    base_model_path,
    trust_remote_code=True
)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_path,
    device_map="auto",
    torch_dtype=torch.float16,
    trust_remote_code=True
)


print("nLoading QLoRA adapter...")
model = PeftModel.from_pretrained(
    base_model,
    output_dir,
    device_map="auto",
    torch_dtype=torch.float16
)
model.eval()


# Example prompt
prompt = "What are the main differences between classical and quantum computing?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")


print("nGenerating response...")
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128)


response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("n=== Model Output ===")
print(response)

Finally, we upload the basic 7B model and use the lora-new lora instruments. We build fast as soon as the difference between classical and quantum computers, convert us into the tokens, and produce feedback using a well-prepared model. This proves that our Qlora Training has begun working and that we can successfully move the updated model.

Snapshot of Supported Models with analatl

In conclusion, these above steps have shown you how to correct the environment, set up a small dataset, and run a special Lora-Tuning Session, and run the best Qlora Tuning Tuning Tuning Tuning. This approach indicates a well-efficient training process that is prepared for the limited areas. You can now grow the dataset, change hyperpares, or test with open source opening to continue analyzing and upgrading your beautiful pipe.

Download Colab Notebook HERE. All credit for this study goes to research for this project. Also, don't forget to follow Sane and join ours Telegraph station including LinkedIn Grtopic. Don't forget to join ours 75k + ml subreddit.

🚨 MarktechPost is shouting for companies / initializing / groups to cooperate with the coming magazines of AI the following 'Source Ai in production' and 'and' Agentic Ai '.

Asphazzaq is a Markteach Media Inc. According to a View Business and Developer, Asifi is committed to integrating a good social intelligence. His latest attempt is launched by the launch of the chemistrylife plan for an intelligence, MarktechPost, a devastating intimate practice of a machine learning and deep learning issues that are clearly and easily understood. The platform is adhering to more than two million moon visits, indicating its popularity between the audience.

✅ [Recommended] Join Our Telegraph Channel

Source link

nimda February 10, 2025

0 6 4 minutes read