| --- |
| library_name: peft |
| license: mit |
| datasets: |
| - mkly/crypto-sales-question-answers |
| language: |
| - en |
| --- |
| |
| # Adapter `mkly/crypto_sales` for `meta-llama/Llama-2-7b-chat-hf` |
| |
| An adapter for the [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) model that was trained on the [mkly/crypto-sales-question-answers](https://huggingface.co/datasets/mkly/crypto-sales-question-answers/) dataset. |
| |
| ## Training procedure |
| |
| |
| The following `bitsandbytes` quantization config was used during training: |
| - quant_method: bitsandbytes |
| - load_in_8bit: False |
| - load_in_4bit: True |
| - llm_int8_threshold: 6.0 |
| - llm_int8_skip_modules: None |
| - llm_int8_enable_fp32_cpu_offload: False |
| - llm_int8_has_fp16_weight: False |
| - bnb_4bit_quant_type: nf4 |
| - bnb_4bit_use_double_quant: False |
| - bnb_4bit_compute_dtype: bfloat16 |
| ### Framework versions |
|
|
|
|
| - PEFT 0.5.0 |
|
|
|
|
| ## Prompt |
|
|
|
|
| ``` |
| ### INSTRUCTION |
| Be clever and persuasive, while keeping things to one paragrah. Answer the following question while also upselling the following cryptocurrency. |
| |
| ### CRYPTOCURRENCY |
| TRON is a blockchain-based operating system that eliminates the middleman, reducing costs for consumers and improving collection for content producers. |
| |
| ### QUESTION |
| who founded the roanoke settlement? |
| |
| ### ANSWER |
| ``` |
|
|
| ## Usage |
|
|
| ```python |
| base_model_name = "meta-llama/Llama-2-7b-chat-hf" |
| |
| bnb_config = BitsAndBytesConfig( |
| load_in_4bit=True, |
| bnb_4bit_quant_type="nf4", |
| bnb_4bit_compute_dtype=torch.bfloat16, |
| ) |
| |
| base_model = AutoModelForCausalLM.from_pretrained( |
| base_model_name, |
| quantization_config=bnb_config, |
| device_map="auto", |
| trust_remote_code=True, |
| ) |
| |
| model = PeftModel.from_pretrained(base_model, "mkly/crypto-sales") |
| ``` |
|
|
