I tested a local LLM on 5 everyday tasks—only one was actually worth doing

I use local LLMs on a daily basis, but in my personal setup, they’re only really useful for a limited selection of tasks. I wanted to see how a local LLM could handle everyday tasks. My most powerful device for running LLMs is my M2 MacBook Air with only 8GB of RAM, so while more powerful hardware could do much more, these results should broadly match what you can do on similar hardware.

Answering broad questions

The responses are painfully slow and often wrong

The easiest mistake to fall into with a local LLM is treating it like ChatGPT, Claude, or Gemini. With these powerful models running on super-fast hardware, you can afford to ask vague and open-ended questions, and the models can infer your meaning and provide useful answers that start to appear almost instantly.

This is far from the case with a local LLM on my hardware. I used the Qwen3.5 4B model running in Ollama for all of these tests, as it’s a fairly capable model but is still small enough to run on my limited hardware. I gave it a fairly broad question: Explain IPv6. Not only was the response incredibly slow, taking more than 30 seconds to complete, but it also contained factual inaccuracies, such as stating that there are around 10^58 IPv6 addresses rather than the actual value of roughly 10^38.

When it comes to answering broad questions like this, the local LLM simply wasn’t up to the job. I could try different models, but while smaller models could respond more quickly, they are also even more likely to make factual mistakes.

Writing complete articles

I can write one faster than fixing what I was given

One thing that’s always concerned me as a writer is that AI may put me out of a job. LLMs can write text, and they have plenty of examples of similar articles in their training data. I wanted to see if my local LLM could make a reasonable stab at writing a full 800-word article.

I gave the local LLM the following prompt:

Write an approximately 800-word technology article with the title:

"5 Things Every New Home Assistant User Should Do First"

Requirements:

- Write in an engaging, conversational style.
- Include an introduction.
- Include five H2 section headings.
- Each section should explain one practical tip with examples where appropriate.
- Finish with a conclusion.
- Do not use bullet points except where absolutely necessary.
- Aim for around 800 words.

The response was actually generated impressively quickly. It took just over a minute to produce the article, which overshot the 800-word limit by about 200 words. The quality, however, left a lot to be desired.

As well as being too long, it ignored some formatting instructions, completely omitted the requested conclusion, was very repetitive, and included several factual errors. The writing was also far too wordy and sounded like it had been written by AI, unsurprisingly. It would take me longer to fix all the problems than it would to write the whole thing myself from scratch.

Summarizing long documents

The local LLM actually did a reasonable job

ChatGPT and other cloud-based chatbots are great at taking long documents and accurately summarizing them. It always amazes me how a chatbot can start answering a question about an enormous file almost instantly, making it feel as though it’s already “read” the information, like a robot flipping through a dictionary at high speed in a 1980s movie and then announcing that it now speaks French.

I used the following prompt, and pasted a page of documentation that was about 3,000 words long:

Summarize the following Home Assistant documentation.

Your response must include:

1. A one-paragraph executive summary.
2. Five key takeaways.
3. The most important security warnings or risks.
4. Any important conclusions or recommendations.

Keep the complete response under 600 words. Base the summary only on the supplied document. Do not add information that is not present in the document. Clearly distinguish between configuring Home Assistant's HTTP server and creating HTTP-based sensors or binary sensors.

DOCUMENT:

<<>>

The local LLM actually did a reasonable job of summarizing a long documentation file. It pulled out the key topics, followed the instructions, and found the most important security implications. It got a little verbose and hit the output limit, but with a little tweaking, it could give useful results.

The biggest problem was that it took just under a minute to generate the finished summary. If you’re not in a rush, however, then even a relatively small local LLM may be able to do a reasonable job of summarizing long documents.

Acting as a smart home voice assistant

The response time is way too slow

This was something that I really hoped a local LLM running on my hardware would be able to do, because it would allow me to have a fully local voice assistant that could rival what Alexa or other cloud-based voice assistants could do.

Home Assistant has a voice assistant called Assist that can match built-in sentence patterns to predefined intents without needing an LLM. It’s not perfect, however. If I ask Assist to turn off the light, it happens almost instantly, because that’s one of the predefined commands. If I then say, “Turn it back on again,” it fails, because it has no idea which device I’m talking about.

You can make the system much smarter by hooking it up to an AI service such as OpenAI, which allows you to use natural language and let the LLM figure out what it is that you’re intending with your voice commands. By connecting Assist to a cloud-based LLM, it becomes much more capable, but commands handled by that LLM are sent to third-party servers, which goes against the privacy-first nature of Home Assistant. A local LLM could solve this problem.

I hooked up my local LLM to Home Assistant using the Ollama integration, and set it up as a conversation agent for Assist. I asked Assist to turn off the study light, which it did. I then asked, “Turn it back on again,” and the light did eventually turn back on, but it took 21 seconds for anything to happen.

In other words, the local LLM took a third of a minute to figure out that when I said, “Turn it back on again,” I was referring to the light that it had literally just turned off. This is far too long to be useful in a real smart home; a voice command that takes 20 seconds to complete is no use to anyone, and more complicated requests would be likely to take even longer.

A coding assistant

The LLM argued itself out of a good response

Powerful tools such as Codex and Claude Code have made writing code accessible to almost anyone. I wanted to see if a local LLM could work as a useful tool for helping with coding, so I gave it a made-up Python error to see if it could spot the problem.

It turns out that I was the problem. The error message that I stated I’d received wasn’t possible with the code I pasted into the prompt. The LLM initially misdiagnosed the error, but then spotted that the error I claimed I’d received wasn’t possible.

However, instead of then asking for clarification or explaining what the true error should have been and how to fix it, the model just got stuck in a loop of second-guessing itself until it ran out of tokens.

It took 40 seconds to get the response, and it was completely useless. I won’t be using my local LLM to help with coding any time soon.

A local LLM can do the right jobs well

In many of these examples, the local LLM was either too slow or too inaccurate to be of any use. That’s not to say that an LLM can’t do a good job. There are plenty of cases where the slow speed isn’t a major issue, such as generating my spoken morning briefing, which I run during the early hours each morning. For the right tasks, even a small local LLM can do a job.