> For the complete documentation index, see [llms.txt](https://openledger.gitbook.io/openledger/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://openledger.gitbook.io/openledger/openlora/workflow.md).

# Workflow

1. **Base Model Initialization:**

* A foundational model (e.g., Llama 3, Mistral, or Falcon) is loaded into GPU memory.

2. **Dynamic LoRA Adapter Retrieval:**

* When a request specifies a fine-tuned adapter, the system dynamically loads it from Hugging Face, Predibase, or a local directory.
* The adapter is merged with the base model in real-time.

3. **Merging & Activation:**

* LoRA adapters are merged into the base model using optimized kernel operations.
* Multiple adapters can be combined for ensemble inference.

4. **Inference Execution & Token Streaming:**

* The merged model generates responses with token streaming for low-latency output.
* Quantization techniques ensure memory efficiency while maintaining accuracy.

5. **Request Completion & Adapter Eviction:**

* Once inference is complete, the adapter is unloaded to free GPU memory.
* This process allows for serving thousands of fine-tuned models without memory bottlenecks.