Ten days with my home server: how I made it fast, stable and able to work alone

Ten days ago I had a powerful small computer at home. It ran language models, and it answered my requests. So I believed that it worked well. I was wrong.

The computer has an AMD Ryzen AI MAX+ 395 processor and 128 GB of memory. The CPU and the built-in GPU share this memory. I call the computer yserver. This is the story of how I improved it, and what I learned.

Step 1: I checked the real state first

Before I changed anything, I collected every fact about the computer in one file: the hardware, the installed software, the running services, and the measured behaviour of the models.

The result surprised me.

• The models did not use the GPU at all. The Linux kernel was too old for this GPU, so every model ran on the CPU at 42 tokens per second.
• Linux could use only 62 GB of the 128 GB. The other half was reserved for the GPU, and the GPU was not in use.
• The context was only 4,096 tokens when a program did not ask for more. The context is the amount of text a model can read at one time. There was no warning.
• The CPU was set to save power, not to work fast.

Nothing was broken. But almost everything worked worse than I thought.

Step 2: I made the GPU work, with a safe way back

The GPU needed a much newer Linux kernel. This was a risk, because I reach the computer only over the network. A failed start could lock me out.

So I prepared the change carefully. The computer started the new kernel once. The old kernel stayed the default. The new kernel became the default only after a good start, with a working GPU and a working network. After a bad start, the computer would return to the old kernel by itself.

The start was good. The same model now ran at 94 tokens per second instead of 42. The reserved half of the memory became 64 GB of GPU memory. This made a much larger model possible. My main coding model now has 80 billion parameters and needs 52 GB. It runs at 50 tokens per second. The context is 64,000 tokens instead of 4,096. In one test the model read a prompt of 44,000 tokens completely.

Step 3: I built one entrance for all programs

Several programs used the model server directly. Each program asked for a different model. Only one large model fits in memory, so every change of model took 13 seconds. Two programs could force such changes all day.

I wrote a gateway. A gateway is a small service that receives every request first. Mine understands both the Anthropic API and the OpenAI API. Programs ask for names they already know. The gateway sends each request to the right local model, and one coding model serves most requests.

The effect was clear. A short request from one of my tools took 7.5 seconds before. With the gateway it took 0.6 seconds, because the model stayed in memory. The gateway also puts requests in a queue. It rejects a prompt that is too long instead of cutting it silently. It never stores the text of a prompt.

Step 4: I let the computer do the work

I did not type most of this myself. I wrote my plans as a list of numbered steps on the computer. An AI coding agent carried out the steps. After each step, an automatic check decided if the step was really done. The agent could not influence this check. The agent had no administrator rights, and there was a spending limit. When a step needed administrator rights, it was a script that first printed what it would do, and I ran it myself.

After 34 steps the cost was about 215 dollars. The agent had built the gateway, a program that runs long tasks, speech input and output, image generation on the GPU, and the services for my other project.

Step 5: I compared my system with Claude

Next I tested my system against the Claude models with a set of graded tasks. I wrote every weakness into a table: the evidence, how serious it is, the fix, and the status. This table became my most useful document.

Some results:

• My AI tools could not search the web. I installed my own search service behind the gateway. A search takes one second. Reading a page takes 0.13 seconds.
• The model for code completion in my editor was too small. I measured three sizes. A model with 3 billion parameters was correct, and a larger one was not better.
• I found the most serious error of the project. When a prompt was too long, the server cut it without a message. Then it returned the answer to the previous request, word for word, with a success status. I reproduced the error without my own software. I reported it to the project, and I installed a fixed version. My gateway now rejects such prompts.

My own check then caused a new error. It counted an image as text, and the vision model failed for one day. I found this with a test that sends a real request to every model. This test now runs every day.

Step 6: the difficult week

Then the computer started to fail.

The first problem was memory. On this computer the CPU and GPU share one memory. When the GPU memory is full, the driver moves data into normal memory. This ignores the configured limit. The memory is not assigned to any program, and no report shows it. One evening an image job, a large model and several test jobs ran at the same time. In 97 minutes the system stopped programs 234 times to free memory. Most of the time it chose the wrong programs. In the end I had to switch the computer off and on again.

One setting did not solve this. I needed four things together:

• An upper limit for this kind of memory.
• A Linux feature that protects the GPU memory of each workload.
• A rule: every heavy job must ask for permission before it uses the GPU.
• A small monitor that checks GPU memory every two seconds.

The second problem came the next night. The computer stopped completely for 8 hours and 22 minutes. No log explains why it stopped. But I know why it took so long. Some days before, I had configured a hardware watchdog. A watchdog is a timer inside the computer's chipset. The system must reset this timer every few seconds. If the system stops responding, nobody resets the timer, and the timer restarts the computer. A program cannot do this job, because a program stops together with the computer. My script had reported that the watchdog was active. After the next restart it was not active any more. The Linux distribution blocks this driver by default, and the system skips a blocked driver without any message. My alerts ran on the same computer, so they stopped too.

During the investigation I found three more hidden problems. The system backup had failed every night for three days, because one folder was missing. The copy of the backups on my laptop was four days old, because the copy job ran while the laptop was asleep. One service restarted itself 150 times per hour. Some days later I found a fourth problem. My GPU memory log had recorded the value 0.0 for four days, because it read the wrong device.

All these problems had the same cause. A setting said "yes", but the real state said "no". Now every check that I write reads the real state. Is the watchdog timer really running? Is the last backup really new? Does the logged value really match the GPU? I published these tools as open source: github.com/YauhenBichel/silent-failures

The mistakes in my own work

The fix for the long stop was finished and tested. It waited for my review in a temporary folder. Then the folder was cleaned, and all the work was lost. The fix was rebuilt in one evening from my work log. In this log I record everything that is done on the computer, with times and with corrections. Since that day, work that waits for a review is always saved in the code repository.

There was a second mistake. I reported an error in my own gateway after I had only read a code change. I even used this "error" to judge how well local models review code. Then I wrote a test for it, and the test showed that the error did not exist. The models had reviewed the code badly: two reviews only praised it, one did not finish, and one described errors that were not there. But my own review had been wrong too. A finding is only an assumption until a test or a run confirms it.

Last night

Yesterday I gave the computer two tasks. Then I switched off my laptop and went to sleep.

The computer tested its own speed with a standard tool. First it waited 46 minutes, because another job was using the GPU, and the task said that it must not disturb other work. In the morning I had a message on my phone and a report: 132 requests, 0 failures, the first token after 0.64 seconds, 50 tokens per second, and a prompt of 8,000 tokens read in 13.7 seconds.

Ten days ago the computer answered, and I believed that it worked. Today it tells me when something does not work.

What I learned

1. Check the real state of a machine before you build on it. My models were running on the wrong processor.
2. Prepare a safe way back for every risky change.
3. Use one protected entrance, and close everything else.
4. Compare your system with the one you want to replace. Keep every weakness in a table with its status.
5. Do not trust a setting. Check the real state.
6. A monitor must not run only on the machine that it monitors.
7. Keep a work log with times, and include your own mistakes. It is a backup of the work itself.