[VicPiMakers Projects] Running the new llamafile (llama.cpp) app

[Craig Miller]

Hi Deid,

Looking at the gguf models on HuggingFace:

https://huggingface.co/models?library=gguf

There were a couple of parameters I was looking at:

 1. Not too big, somewhere between 5 and 10 GB in size
 2. Relatively recent
 3. Doesn't core dump right away

I had the best luck at running the Qwen models. I am running 
Qwen2.5-VL-7B-Instruct-abliterated.Q4_K_M.gguf on my PN-50, and it seems 
to run reasonably fast. Some of the other models were quite slow on the 
PN-50.

Have fun!

Craig...

On 4/26/26 07:13, Deid Reimer wrote:
> Hey Craig,
>
> Why did you pick that particular LLM?
>
> Deid   VA7REI
> On Apr 25, 2026, at 8:32 a.m., Craig Miller <cvmiller at gmail.com> wrote:
>
>     Hi All,
>
>     We were chatting before the most recent NetSIG about the new
>     Llamafile app, which has excellent support for IPv6. The app runs
>     a webserver (which is IPv6 accessible).  The new llamafile app
>     takes a -m parameter which points to the gguf LLM model.
>
>     * Old way*
>          ./google_gemma-3-4b-it-Q6_K.llamafile --server -v2 --host
>     lxcllama.example.com
>     *New way*
>          llamafile -m model.gguf --server --port 8080
>
>     Find the new llamafile at:
>
>     https://github.com/mozilla-ai/llamafile/releases/tag/0.10.0
>
>     You can find gguf (LLM models) at:
>
>     https://huggingface.co/models?library=gguf
>
>     I start my llamafile using this command:
>
>         ./llamafile-0.10.0 -m Qwen3.5-9B.Q4_K_M.gguf --server --port
>     8080 --host lxcllama.example.com
>
>     This way any webbrowser at my house, can access the LLM.
>
>     Happy LLM-ing,
>
>     Craig...
>
>     -- 
>     IPv6 is the future, the future is here
>     http://ipv6hawaii.org/
>
>     -- 
>     Projects mailing list
>     Projects at vicpimakers.ca
>     http://vicpimakers.ca/mailman/listinfo/projects_vicpimakers.ca
>
>
-- 
IPv6 is the future, the future is here
http://ipv6hawaii.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://vicpimakers.ca/pipermail/projects_vicpimakers.ca/attachments/20260426/56ba9e55/attachment-0001.htm>