I was going through this process about a year ago. I ended up settling on the Protectli V1410 (Intel N5105 and 4 Intel I226-V 2.5Gbps ports) because: a) I wanted coreboot, b) I was a little wary of lesser-known Ali Express sellers c) Protectli seemed like a decent company and I felt good about supporting people investing in an open source firmware (minus proprietary blobs which you can’t do much about). None of that was strictly necessary, and you probably would be fine just going with a cheaper NUC or mini PC. I would definitely second some of the other comments on avoiding Realtek NICs though.

Thanks for this! I guess that settles my case deliberation at least. But man, getting all the things you mentioned for $170 on eBay is crazy good!!! I must just really need to dig in there a bit deeper (it’s gonna be too easy to go broke with home networking as a hobby otherwise lol). I guess so long as I am memtesting my RAM and burning in drives right after getting them I should be okay. I think I also am overindexing on trying to get my power draw as low as possible instead of really doing the math out and seeing how long I’d need to be running something 24/7 before the power savings ends up outweighing the price differential. But thanks again!

Thank you! The case rec is super helpful actually, I was considering the Fractal Design Define R5 (though I wasn’t sure how to feel about potentially spending the same amount on my motherboard as my case, but people seem to swear by it), so it’s great to have confirmation they’re actually worth it. I had given PC Part Picker a shot, but it seems like it doesn’t quite catch the ECC memory support for some of the AMD systems (maybe because they’re unofficial?) I will give it another shot though, and thanks again!

Thanks for this! It’s a good impetus for me to think a little more holistically about my network security versus overfocusing on the router. I’ll have to do some more reading on overall networking best practices.

No I think we’re aligned! I am not trying to say the “build literally everything” from scratch is a viable alternative. You could go all the way down the rabbit hole of building a compiler, your own programming language, a smelter to refine the metals you need to try to cobble together your own hardware. But of course that is not realistic, which was what I was trying to get at in my comment. Basically, given that it is not feasible to do everything by yourself, at some point it seems you have to decide to trust something to be a functional human and not devolve into solipsism. So the question I am asking is, what are your own evaluations of what is trustworthy? Do you trust coreboot more than AMI? Protectli versus Qotom? It seems to me that we have to make these sorts of evaluations, versus believing that because there is some risk to everything that those risks are all equal. Apologies if I am not being clear though.

Thanks for sharing! Did you end up staying with the stock firmware?

Thanks for this! Agree that coreboot is definitely the requirement that, if dropped, would open up the most other options. So far it sounds like folks are mostly willing to have some faith in stock firmware, which is great as a sanity check for me. Appreciate your response!

Sorry, imprecise wording on my part, I meant build as in build/code from scratch, not build from source!

Thanks so much! I’d seen Hunsn mentioned in a few places as well, so glad to hear that it’s working well (and thanks especially for the memory ballooning tip, I’ll try to remember that when I inevitably run into issues later).

Thanks so much for sharing this! I think reading through it helps refocus the question I guess I should have asked, which is “Which vendors do people trust more in practice, recognizing that at some point recursive paranoia has to end unless one has the time and skill to try to build literally everything on their own?” And as a question of probabilities, it feels a bit more manageable to try to make a call and move on. I’m sort of thinking of this thread as a way for me to calibrate my current probability estimates with people who know more than I do and have likely thought about this question more than I have. But the reminder that there isn’t really going to be any certainty regardless of what I decide is well-taken.

(But if you don’t mind my asking, what machine did you end up with from CN? How did you approach firmware?)

Yes, I’m US-based, and you make a great point that it’s not as though US brands are inherently trustworthy either. That’s why I’m leaning towards an open source (or as open source as possible) firmware, with the understanding that we’re stuck with some proprietary blobs at the moment. I suppose I am thinking about it more from a harm reduction lens versus trying to find a bullet-proof solution.

Thanks for sharing! Will probably try to go this route once I get the NAS squared away and turn back to localLLMs. Out of curiosity, are you using the q4_k_m quantization type?

This is exactly the sort of tradeoff I was wondering about, thank you so much for mentioning this. I think ultimately I would probably align with you in prioritizing answer quality over context length (but it sure would be nice to have both!!) I think my plan for now based on some of the other comments is to go ahead with the NAS build and keep my eyes peeled for any GPU deals in the meantime (though honestly I am not holding my breath). Once I’ve proved to myself I can something stable without burning the house down, I’ll on something more powerful for the localLLM. Thanks again for sharing!

Wow, that sounds amazing! I think that GPU alone would probably exceed my budget for the whole build lol. Thanks for sharing!

This is definitely good advice. I tend to run my laptops into the ground before I replace them, but a lot of the feedback here has made me think experimenting with something much less expensive first is probably the right move instead of trying to do everything all at once (so that when I inevitably screw up, it at least won’t be a $4k screw up.) But thanks for the sanity check!

Thank you so much for all of this! I think you’re definitely right that probably starting smaller and trying a few things out is more sensible. At least for now I think I am going to focus on putting something together for the lower-hanging fruit by focusing on the NAS build first and then build up to local AI once I have something stable (but I’ll definitely be keeping an eye out for GPU deals in the meantime, so thanks for mentioning the B580 variant, it wasn’t on my radar at all as an option). But I think the thread has definitely given me confidence that splitting things out that way makes sense as a strategy (I had been concerned when I first wrote it out that not planning out everything all at once was going to cause me to miss some major efficiency, but I feel like it turns out that self-hosting is more like gardening than I thought in that it sort of seems to grow organically with one’s interest and resources over time; sort of sounds obvious in retrospect, but I was definitely approaching this more rigidly initially). And thank you for the HDD rec! I think the Exos are the level above the Ironwolf Pro I mentioned, so will definitely consider them (especially if they come back online for a reasonable price at serverpartdeals or elsewhere). Just out of curiosity, what are you using for admin on your MC server? I had heard of Pterodactyl previously, but another commenter mentioned CraftyController as a bit easier to work with. Thank you again for writing all of this up, it’s super helpful!

You are both totally right. I think I anchored high here just because of the LLM stuff I am trying to get running at around a GPT4 level (which is what I think it will take for folks in my family to actually use it vs. continuing to pass all their data to OpenAI) and it felt like it was tough to get there without spending an arm and a leg on GPUs alone. But I think my plan is now to start with the NAS build, which I should be able to accomplish without spending a crazy amount and then building out iteratively from there. As you say, I’d prefer to screw up and make a $500 mistake vs. a multiple thousand dollar one. Thanks for the sanity check!

Woah, this is big news!! I’d been following some of the older articles talking about this being pending, but had no idea it just released, thanks for sharing! Will just need to figure out how much of a datahoarder I’m likely to become, but it might be nice to start with fewer than 6 of the 8TB drives and expand up (though I think 4 drives is the minimum that makes sense; my understanding is also that energy consumption is roughly linear with number of drives, though that could be very wrong, so maybe I’ve even start with 4x a 10-12TB drive if I can find them for a reasonable price). But thanks for flagging this!

Amazing, thanks again for all for all of this! I’ll start keeping my eyes peeled for any good deals on 3090s that pop up (though will probably end up prioritizing the NAS build first just to get my feet wet before diving straight into the localLLM world). But thanks again for taking the time to share!