Commit Beelink mini defaults: CPU inference, 32k context
Stop forcing Vulkan NGL in compose override (that overrode .env). Raise mini profile context to 32768 for ~28–32GB RAM hosts.
This commit is contained in:
parent
84a0945b10
commit
7e5444a177
2 changed files with 7 additions and 17 deletions
|
|
@ -1,22 +1,11 @@
|
||||||
# GPU access for Vulkan (bonsai) + VA-API (jellyfin).
|
# Default override for mini / Beelink-class hosts.
|
||||||
|
# - bonsai: CPU only (do NOT force NGL/devices here — that overrides .env)
|
||||||
|
# - jellyfin: /dev/dri for VA-API when available
|
||||||
#
|
#
|
||||||
# Do NOT use group names like "video"/"render" here — Docker resolves them
|
# mid/gaming: re-add bonsai devices + set BONSAI_NGL in .env via apply-profile.sh
|
||||||
# inside the container image, which often has no "render" group and fails with:
|
# Do NOT use group names (video/render) — Docker resolves them inside the image.
|
||||||
# unable to find group render: no matching entries in group file
|
|
||||||
#
|
|
||||||
# Options:
|
|
||||||
# A) Rely on host /dev/dri being 0666 (Ansible users role udev rule) — no group_add.
|
|
||||||
# B) Use numeric host GIDs: getent group video render
|
|
||||||
#
|
|
||||||
# Ansible regenerates this file with host GIDs when you run the stack role.
|
|
||||||
|
|
||||||
services:
|
services:
|
||||||
bonsai:
|
|
||||||
devices:
|
|
||||||
- /dev/dri:/dev/dri
|
|
||||||
environment:
|
|
||||||
NGL: "99"
|
|
||||||
|
|
||||||
jellyfin:
|
jellyfin:
|
||||||
devices:
|
devices:
|
||||||
- /dev/dri:/dev/dri
|
- /dev/dri:/dev/dri
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,8 @@ STACK_PROFILE=mini
|
||||||
# Ternary-Bonsai 27B fits in system RAM; keep CPU
|
# Ternary-Bonsai 27B fits in system RAM; keep CPU
|
||||||
BONSAI_BACKEND=cpu
|
BONSAI_BACKEND=cpu
|
||||||
BONSAI_NGL=0
|
BONSAI_NGL=0
|
||||||
BONSAI_CTX=16384
|
# 32k uses free system RAM well on ~32GB mini PCs (weights ~7GB)
|
||||||
|
BONSAI_CTX=32768
|
||||||
BONSAI_KV4=0
|
BONSAI_KV4=0
|
||||||
BONSAI_THREADS=12
|
BONSAI_THREADS=12
|
||||||
BONSAI_TEMP=0.7
|
BONSAI_TEMP=0.7
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue