> For the complete documentation index, see [llms.txt](https://openledger.gitbook.io/openledger/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://openledger.gitbook.io/openledger/openlora/optimizations-and-performance-enhancements.md).

# Optimizations & Performance Enhancements

#### Dynamic Adapter Loading

* Unlike traditional methods where all fine-tuned models are preloaded, Open LoRA loads adapters dynamically, reducing GPU memory usage.
* JIT (Just-in-Time) adapter loading ensures only the necessary adapters are in memory.

#### Parallel Processing & Merging

* **Tensor Parallelism:** Spreads computations across multiple GPU cores to accelerate inference.
* **Paged Attention:** Handles longer sequences efficiently, reducing memory fragmentation.
* **Multi-Adapter Merging:** Supports inference using multiple LoRA adapters simultaneously for ensemble generation.

#### CUDA & Low-Level Optimizations

* **Flash Attention:** Reduces memory bandwidth usage by computing attention efficiently.
* **Precompiled CUDA Kernels:** Optimized for low-latency execution, minimizing computation overhead.
* **Quantization (FP8/INT8):** Reduces model size without significant loss in accuracy, improving inference speed.