tim/stack
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add mini/mid/gaming hardware profiles and always-on full stack

Browse source 
Profiles select CPU vs Vulkan and Bonsai-27B vs Qwen3.5-9B Abliterated
Q4_K_M. OpenHands is no longer optional; compose deploys every service.
download-models.sh fetches both GGUF families as the profile requests.
This commit is contained in:
tim 2026-07-22 19:00:59 -07:00
parent 3bf1726a99
commit a729ff14e9
11 changed files with 333 additions and 171 deletions
Download patch file Download diff file Expand all files Collapse all files

24
profiles/gaming.env Normal file
View file

@ -0,0 +1,24 @@
# Profile: gaming — ~16GB AMD GPU + 64GB system RAM
# Vulkan; Ternary-Bonsai 27B primary, Qwen 9B also downloaded for light loads.
STACK_PROFILE=gaming
BONSAI_BACKEND=vulkan
BONSAI_NGL=99
BONSAI_CTX=32768
BONSAI_KV4=0
BONSAI_THREADS=16
BONSAI_TEMP=0.7
BONSAI_TOP_P=0.95
BONSAI_TOP_K=20
LLM_ID=bonsai27
BONSAI_HF_REPO=prism-ml/Ternary-Bonsai-27B-gguf
BONSAI_MAIN_FILE=Ternary-Bonsai-27B-Q2_0.gguf
BONSAI_MODEL_PATH=/models/Ternary-Bonsai-27B-Q2_0.gguf
OPENWEBUI_DEFAULT_MODEL=openai/Ternary-Bonsai-27B-Q2_0.gguf
OPENHANDS_MODEL=openai/Ternary-Bonsai-27B-Q2_0.gguf
DOWNLOAD_MODEL=true
DOWNLOAD_BONSAI27=true
DOWNLOAD_QWEN=true
DOWNLOAD_VISION=true

25
profiles/mid.env Normal file
View file

@ -0,0 +1,25 @@
# Profile: mid — ~8GB AMD GPU + 16GB system RAM
# Vulkan offload; use Qwen3.5-9B Abliterated Q4_K_M (~5.6GB) for VRAM fit.
STACK_PROFILE=mid
BONSAI_BACKEND=vulkan
BONSAI_NGL=99
BONSAI_CTX=8192
BONSAI_KV4=1
BONSAI_THREADS=8
BONSAI_TEMP=0.7
BONSAI_TOP_P=0.95
BONSAI_TOP_K=20
# Qwen3.5-9B Abliterated Q4_K_M (standard GGUF; works on Prism llama-server)
LLM_ID=qwen9
BONSAI_HF_REPO=mradermacher/Qwen3.5-9B-abliterated-v2-MAX-GGUF
BONSAI_MAIN_FILE=Qwen3.5-9B-abliterated-v2-MAX.Q4_K_M.gguf
BONSAI_MODEL_PATH=/models/Qwen3.5-9B-abliterated-v2-MAX.Q4_K_M.gguf
OPENWEBUI_DEFAULT_MODEL=openai/Qwen3.5-9B-abliterated-v2-MAX.Q4_K_M.gguf
OPENHANDS_MODEL=openai/Qwen3.5-9B-abliterated-v2-MAX.Q4_K_M.gguf
DOWNLOAD_MODEL=true
DOWNLOAD_QWEN=true
DOWNLOAD_BONSAI27=false
DOWNLOAD_VISION=false

26
profiles/mini.env Normal file
View file

@ -0,0 +1,26 @@
# Profile: mini — Beelink / SER5-class mini PC
# Always CPU for inference (stable, no iGPU hassle). Full stack always deploys.
STACK_PROFILE=mini
# Ternary-Bonsai 27B fits in system RAM; keep CPU
BONSAI_BACKEND=cpu
BONSAI_NGL=0
BONSAI_CTX=16384
BONSAI_KV4=0
BONSAI_THREADS=12
BONSAI_TEMP=0.7
BONSAI_TOP_P=0.95
BONSAI_TOP_K=20
# Primary model (Prism Q2_0 — needs Prism llama-server image)
LLM_ID=bonsai27
BONSAI_HF_REPO=prism-ml/Ternary-Bonsai-27B-gguf
BONSAI_MAIN_FILE=Ternary-Bonsai-27B-Q2_0.gguf
BONSAI_MODEL_PATH=/models/Ternary-Bonsai-27B-Q2_0.gguf
OPENWEBUI_DEFAULT_MODEL=openai/Ternary-Bonsai-27B-Q2_0.gguf
OPENHANDS_MODEL=openai/Ternary-Bonsai-27B-Q2_0.gguf
# Also download mid-tier Qwen so you can switch without re-bootstrap
DOWNLOAD_MODEL=true
DOWNLOAD_QWEN=true
DOWNLOAD_VISION=true