A general model like Qwen or Llama knows a lot about the world and nothing about your company. Your price list, your SOPs, your customers and your tone of voice are not in it. There are three ways to fix that, and the heaviest one, fine-tuning, is often the first one people ask about. Usually it should be the last.
Three ways to teach a model about your company
1. A better prompt: the rule sheet
A prompt is the text you send with each question. A system prompt is a rule sheet sent with every question: "Answer in short formal Bahasa Indonesia. Sign as the sales team. Never promise a delivery date." It is free, it works instantly, and you can change it in a minute. It is the right tool for tone, format and short rules. It is the wrong tool for a 200-page manual, because the rule sheet has to fit in the model's context window, the amount of text it can read at once.
2. RAG: the open-book exam
RAG, retrieval-augmented generation, works like an open-book exam. Your files are cut into pieces and indexed. When someone asks a question, the system looks up the most relevant pieces and hands them to the model together with the question. The model answers from those pages and can point to them.
RAG is the right tool for facts that change: prices, stock, SOPs, contracts, HR policies, product manuals. When a document changes, you update the index, and the next answer is current. Nothing is retrained.
The index uses an embedding model, a small model that turns text into numbers so that similar meanings sit close together. A multilingual one such as BGE-M3 handles more than 100 languages, including Bahasa Indonesia and English, and reads up to 8,192 tokens at a time. Indexing is light work. Answering is not: the chat model runs on the GPU like any normal conversation, with the retrieved pages added to the context, so plan memory the same way you would for chat.
3. Fine-tuning: the training course
Fine-tuning changes the model itself a little by training it on your examples. It is a training course, not a reference book. It is the right tool for a fixed output format, a house writing style, sorting incoming messages into categories, or reading one kind of form the same way every time.
It is a poor tool for facts that change. The model memorises what it was trained on, and it goes stale the day your price list changes. It can also state old facts with confidence.
LoRA and QLoRA in plain words
Training all of a model's billions of weights needs a huge amount of memory. LoRA (low-rank adaptation) avoids that. The base model stays frozen, and you train a small add-on, an adapter, that sits on top of it. The adapter is a small file, easy to store and to swap.
QLoRA goes one step further. It loads the frozen base model in 4-bit form (a quant, a compressed copy of the weights) and trains the adapter on top. 4-bit weights take about a quarter of the memory of 16-bit ones, so a model that would not fit in a 32 GB card for 16-bit LoRA can fit for QLoRA. For narrow jobs like the ones above, QLoRA is the usual way to fine-tune on a single card.
How much memory fine-tuning needs
| Model | QLoRA, 4-bit base | LoRA, 16-bit |
|---|---|---|
| Llama 3.1 8B | 6.0 | 22.1 |
| Qwen3.5 9B | 6.8 | 25.6 |
| Qwen3.8 27B | 22.6 | 65.9 |
| Llama 3.3 70B | 41.4 | 165.5 |
| gpt-oss-120b | 68.7 | 274.8 |
Unsloth, the training toolkit behind these figures, calls them the absolute minimum. Plan for some headroom on top.
Which EMD Lab machine fits
- 32 GB (Studio): the Studio workstation with an RTX PRO 4500 or an RTX 5090 covers QLoRA up to about 27 to 31B models, and 16-bit LoRA on small 8 to 9B models.
- 96 GB (one RTX PRO 6000): covers QLoRA of a 70B model (41.4 GB in the table) with room to spare, QLoRA of gpt-oss-120b (68.7 GB), and 16-bit LoRA of models up to about 27B.
- 2 x RTX PRO 6000 Max-Q (192 GB): the AI workstation is where 16-bit LoRA of large models starts, for example 70B at 165.5 GB, with training software that splits the model across both cards. Larger 16-bit jobs need more than that.
Fine-tuning is a batch job: it runs, it finishes, and the machine goes back to serving chat or RAG for your staff.
Check the licence before you resell
Open models come with different licences. Qwen3.8 27B and gpt-oss-120b are Apache 2.0. Llama (and models built on Gemma 2, which uses Google's Gemma Terms of Use) carry their own conditions, for example on acceptable use and naming. Using a fine-tuned model inside your company is a different question from reselling one to customers. Read the licence on the model page before you build a product on it, and ask your legal adviser if it matters to you.
Which one should you start with?
Start at the top and move down only when you must. Write a good system prompt. If staff still need answers from your documents, add RAG. Consider fine-tuning only when you have hundreds of good examples, a clear and repeatable job, and someone to run and test the training. Many offices stop at RAG and are happy.
The short version
- A prompt is a free rule sheet for tone and short rules.
- RAG is an open-book exam. Use it for facts that change: prices, stock, SOPs, contracts.
- Fine-tuning teaches format, style and repeatable tasks. It goes stale on facts.
- QLoRA trains a small add-on on a 4-bit base, so 32 GB handles models up to about 27 to 31B and one 96 GB card handles 70B and gpt-oss-120b.
- Most offices need RAG first. Check the model licence before reselling anything fine-tuned.
The memory figures above are estimates. See how a model fits your hardware in the model finder.