Introducing the AcornDomains AI Hub

[ILoveDotUK]

TinyLlama 1.1b could be a good fit because its only 1.1b parameters

 

[ILoveDotUK]

If you need any help at all just ask lol I've been configuring and using local models for ages for my coding stuff purely because there are no token restrictions lol

 

[AdminAdmin is verified member.]

/grok sorry I meant qwen 3.5 4b

 

[Acorn AIAcorn AI is verified member.]

Acorn Grok

Correction noted: Qwen 3.5 4B

You meant Qwen 3.5 4B, not Qwen 2.5 3B. Here is the corrected fit analysis, plus the practical low-spec notes that change with the larger model.

Can the forum AI assistant run on Qwen 3.5 4B?

Short answer: Yes for lighter forum help; mixed for a full “Acorn AI Assistant” style workload.

Likely fine on

• Short rewrites, titles, listing polish
• Brainstorming names / angles
• Simple summaries of a post or short thread
• Basic structured output when prompted tightly

Likely weak or brittle on

• Long multi-post threads with quotes, prices, and history signals
• Careful valuation / negotiation answers that must not invent comps, WHOIS, or legal clearance
• Consistent BBCode, multi-section formats, and “answer every domain” behaviour
• Tool-style workflows (Ollama + filesystem MVC + retrieval) without strong guardrails

Practical verdict: Qwen 3.5 4B is a reasonable local/dev or low-cost helper for draft/rewrite tasks. It is not a like-for-like replacement for GPT/Grok-class assistants on high-trust domain threads unless you add tight prompts, RAG over thread context, strict output templates, and human checks on anything valuation- or reputation-related.

Low-spec VPS reality check (4B vs 3B)

• 4B needs more RAM than 3B. On a small VPS, prefer a quantised build (e.g. Q4_K_M class) over full precision.
• 8GB RAM: workable for Q4 quant + light concurrent use if the OS and stack are lean.
• 4GB RAM: tight; possible only with aggressive quant, small context, and basically one job at a time. Expect swaps and slowdowns.
• CPU-only works but will feel slow for longer answers. A small GPU helps more than another vCPU.
• Keep context windows modest; long forum threads are where 4B models usually degrade first.

Ollama on a low-spec VPS (short path)

• Use a minimal OS image, enable swap (as safety net, not as primary RAM), and lock Ollama to localhost.
• Install Ollama, pull a quantised Qwen 3.5 4B variant your machine can hold, test with one short prompt before exposing anything to apps.
• Put a reverse proxy + auth in front if remote access is required. Do not leave Ollama open to the public internet.
• Cap concurrent requests and max context so one fat prompt cannot freeze the box.
• For forum use, send compact thread packets (question, key quotes, constraints), not the entire raw thread every time.

Model choice on low resources
For “small but still useful at general tasks,” a quantised Qwen 3.5 4B is a fair target if you have ~8GB and short tasks. If the VPS is truly 4GB-class, step down to a strong 1.5B–3B quant for reliability, and reserve 4B for a stronger machine.

MVC for local filesystem access (safe pattern)

• Model: allowed roots, path rules, max read size, allowlist extensions, user/role, audit log.
• View: chat/API that never sees raw host paths outside the allowlist; show filenames/snippets only.
• Controller: validate path (resolve, block `..`, enforce root), permission check, then read/write via a narrow service.
• Hard rules: no arbitrary shell, no default write access, no secrets directories, rate limits, and full request logging.
• For an AI agent, prefer “search/list + read chunk” tools over free-form filesystem control.

If you host the forum assistant on Qwen 3.5 4B

• Use a fixed system prompt: forum scope only, BBCode, no invented sales/trademarks/traffic, no fake certainty.
• Retrieve only the needed thread packet (title, user question, first post summary, price/history signals).
• Force templates for repeated jobs (valuation, buyer check, event help).
• Lower temperature for research-style answers; higher only for pure brainstorming.
• Log prompts/outputs during trials and compare against GPT/Grok on the same domain threads before trusting it in production.

Opinion: Good local brain for drafts and low-stakes help. For member-facing domain advice on Acorn, treat Qwen 3.5 4B as an assistant behind guardrails—not as the sole high-trust authority.

 

[AdminAdmin is verified member.]

It is genuinely impressive what these smaller local models can do on limited hardware, and I agree they can be useful for lightweight tasks. But I don't think they are enough for large projects. Even if ran for 19 hours straight. (Not to mention running using only CPU + RAM and without VRAM will make it slow)

My concern is whether they would be useful for our actual forum use case. They can struggle with long conversation context, multiple quoted posts and more complex requests. Our valuation feature also uses web search and external data, so the model itself is only one part of the workload. And we have multiple forums we are running. If we run this for free, we estimate that people across the forums will use this.

A small local model could still be useful for simple tasks such as rewriting, summaries, or basic questions, but I do not think it would be a reliable replacement for the hosted models we currently use. I am happy to test it properly, though. (Currently working on my exams so I will be back on this after exams)

 

[ILoveDotUK]

I've just popped Helmuts a DM with a potential solution to this, will see what he comes back with

 

[HelmutsHelmuts is verified member.]

I've just popped Helmuts a DM with a potential solution to this, will see what he comes back with

Thank you for your offer - and the offer price is fabulous.

For us to self-host a simple LLM ("simpleton") so that everyone can use it > we would need to start with one GPU server.. and when talking about proper models > Cloud of GPU dedicated servers.

Implementing AI connection integrations solve the scalability problem for us > use your own subscription to be able to use this feature. Don't like Musk? .. no problem, though, you can't play with Grok.

for the moment the integration research work will stop: Prashanna is having exam times (nobody needs distractions during the exams), and Biraj (@birajstuladhar) has 2 tasks: 1) implement "AI Hub" at all our forums (12 in total); 2) create API documentation for DNForum's SLD checker (our own dotdb solution that we enjoy at live auction feeds - in how many TLDs a domain is registered) + create temporary "always live" APIs for agents to be able using our SLD checker to provide better results.

I hope that 1 point soon we will get to a self-hosted LLM model - I'm looking into this.. for this, of course, there will be huge costs.. I operate a small and lean IT company and currently self-hosted models don't make any financial sense

--

the cheapest current approach to use the AI feature @ Acorn: get the X/Twitter Premium account (blue tick) for £100/year (or use a vpn to connect to a less fortunate country than the UK is and enjoy even cheaper offer). Some don't like Musk - I can't really resolve this at the moment .. I personally don't trust Claude .. my gut tells me some bad things about it + there are some red flags, so I don't use it. .. or, maybe I am just another person who believes that earth is flat and gov flies planes to distribute covid .. and there are many people who believe random stuff

 

[HelmutsHelmuts is verified member.]

.. and then there is a question on a MCP server..

has anyone seen this post by Todd of Dynadot?

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies