Running Local GGUF Models with Ollama (GPU Enabled)

1. Install & Start Ollama curl -fsSL https://ollama.com/install.sh | sh systemctl start ollama ollama --version 2. Verify GPU Detection NVIDIA nvidia-smi AMD rocm-smi 3. Set Up Model Directory mkdir -p ~/Documents/LLM cd ~/Documents/LLM # Copy your .gguf file here 4. Create a Modelfile vim Modelfile Vim quick reference: i — enter insert mode (start typing) Esc — exit insert mode :wq — save and quit...