Home Server Build

• 

  AndyC1 on 05/04/2022 - 17:24

  With your budget you will probably end up with a CPU with embedded graphics.

  With docker and containers and VM's you need as many real CPU cores and ram as you can afford to get the best benefit, but you seem to want to concentrate on the storage/hdd/ssd etc.

  You may want to think what is the highest priority out of the two as $1K will not support both IMHO:
  docker & VM's
  drive storage configuration

  The RAM config IMHO is as a MIN:
  1) Host OS 8GB
  2) Each VM - 4 GM
  3) Each docker 1 to 4G depending on what they are running (if it's Oracle or MSSQL then up it to 8GB)

  The CPU config IMHO is as a MIN:
  1) Host - 2 CPU cores (plus 2 HT)
  2) Each VM - 2 cores
  3) Docker 1 core or for 2 for SQL/NoSQL per container

  • 

    nodii on 05/04/2022 - 17:28

    Hi AndyC1 thanks for your reply! Let's pretend I didn't say $1k… what would you recommend?

    • 

      mskeggs on 05/04/2022 - 18:24

      You can buy a lot of cloud for $1k.
      Especially if lots of the time they are idle or low load.
      Is offloading some of the images possible?

    • 

      AndyC1 on 05/04/2022 - 18:57

      I would get:
      * 64GB of DDR4 3600Mhz ram
      * 1 x Samsung 980 Pro PCIe 4.0 M2 for OS
      * 1 x 8GB HDD for data storage (docker & VM images and data)
      * 1 x USB SATA adapter
      * 1 x 8+GB for backups (using the USB SATA so the backups are not in the PC, unless being backed up or restored).

      Then comms the CPU hard part:
      AMD Ryzen 7 5700G
      Intel i7-12700

      If you did have no budget as you wanted to run heaps of Docker and VM's then get a 16 core xx thread CPU and cheapest graphics card you can get.

      One issue with the cloud is that you pay for transactions, so if you need to upload or download data for backups

• 

  idjces on 05/04/2022 - 18:23

  I love building everything on an am4 platform. The market of compatible parts is huge. You can go from a 4 core budget build for home office type stuff, up to 16 cores for a home server running VM's and the like.

  You could pick up a used 3900x, that's 12 cores with really decent performance for ~$400? Maybe even a 2700x/3600 if you want to cut down on the cpu price.

  You could build something really decent with used parts for your budget. It's only when you start going with 10-18TB drives, latest gen cpu's etc, that you start blowing out the budget.

  For a reference idea, here is a couple of mine https://i.imgur.com/AmwgfDh.png
  Left is pretty budget, runs a laundry list of blockchain nodes so see's heavy processor use. Right is my main server which rarely see's heavy use except for occasional transcoding and compression. It could also run games and stream to a TV.

• 

  elusive on 05/04/2022 - 19:45

  +3

  At the end of the day, what you'll need very much depends on what you're running. But here's a few notes:

  CPU

  • You're currently running an Ivy Bridge CPU on your main server. You'll see significant gains in virtualisation performance just by stepping past Haswell, not to mention general IPC gains.
  • Should have at least 4 cores. Not hard these days.
    
    • If you do lots of CPU-intensive stuff, consider 6, 8, or more. Alder Lake Intel is a good option for lots of power-efficient cores, although you may want to tune power limits as the defaults go into fairly inefficient territory.
  • On a server, there's not much point in getting a discrete GPU. Problem is a lot of the better AMD CPUs don't have an onboard GPU. If you don't already have an extra video card lying around, you will need to consider this — and either get an AMD APU (ends in G) or an Intel non-F CPU.
    
    • If you don't have any video out capability, you will find setup and troubleshooting more difficult — you'll need to move the OS drives to a different machine if you ever lose network and a reboot doesn't fix it.

  RAM

  • Stick to DDR4 RAM. It has the best capacity:price ratio right now.
  • Don't worry too much about RAM speeds and latency, in a home server the ability to run more things at once and keep data cached in RAM matters more.
  • 32 GB (in 2x 16 GB) is a good starting point. Not much more expensive than 16 GB, not quite as overkill as 64 GB.
  • ECC RAM is good but maybe hard to find a board supporting it. AMD is possibly your only option if you want this.

  Storage

  • Avoid SMR drives. They are entirely unsuitable for RAID of any sort.
  • You're best off getting either Seagate Ironwolf or WD Red Plus drives, as they are CMR. Or any Toshiba desktop drive (X300, N300). If this is always-on, go for the NAS variants.
  • Avoid attaching storage by USB. Pick a motherboard with sufficient SATA ports, and a case that can hold enough 3.5" drives. Unfortunately this is less common these days.
  • Budget for backup drives. RAID won't protect you from accidental deletion, ransomware, house fires…

  Software

  • If you run anything Windows, that raises the requirements floor considerably. Each concurrently running Windows VM should be allocated 1-2 GB RAM for the OS itself, whereas Linux VMs only need 256-512 MB RAM for the OS. Same goes for storage requirements, 20-40 GB for OS vs 2-5 GB.
  • Especially if you're in a Linux environment, consider running system containers rather than full fat VMs. This will let you minimise resource usage, and low-usage containers don't sit there reserving a chunk of your RAM. I like LXD as a container manager; if you're familiar with Proxmox they also provide their own LXC-based containers.
    
    • System containers are in contrast to application containers like Docker. Each system container is basically a lightweight VM that shares the host kernel.

  ---

  Some vague recs based on my last server build:

  • The Cooler Master N300 case has lots of 3.5" bays. It can fit a 165mm tower cooler and run pretty quiet. The motherboard tray on it is recessed slightly so any ports pointing out the side can be a bit painful to access without a 90deg connector.

  Other:

  • For a super-budget option, the Skylake-based Dell Optiplexes can be had for ~$300 second-hand. The biggest problem is they don't really have enough internal space or ports for proper storage expansion, so you're either back to USB, or you'll need to fit a SATA/SAS HBA card and possibly expand out of the case to attach more HDDs properly.
• 

  clippers on 05/04/2022 - 22:54

  I'd second the AMD Ryzen 7 5700G. 8 cores with iGPU for $450 - bargain! Start with 2x16GB RAM.

  If it's just for tinkering, Proxmox can prob run as a VM under Unraid.. nested virtualisation.

• 

  bigmicka on 08/04/2022 - 13:01

  I don't know why I am giving this idea away.

  If power is of no concern buy ones of these.

  https://www.reboot-it.com.au/p/Used-Computers/Xeon/Lenovo-Th… $599 (note they also sell on Ebay for coupons)

  You will get 12 Cores / 24 threads out of the box.
  Ram is low but you can buy DDR4 EEC cheap of Aliexpress.

  Then if you need more cores replace the CPU's again off Aliexpress

  https://psref.lenovo.com/syspool/Sys/PDF/ThinkStation/ThinkS…

  two of these will run you $180 aud and give you 28 cores / 56 threads.

  https://www.aliexpress.com/item/32989189857.html?spm=a2g0o.p…

  If you want to build a homelab that is crazy value.

  • 

    elusive on 09/04/2022 - 05:46

    It's 12 cores, but those are Haswell cores at 2.4 GHz base / 3.2 GHz boost. It'll actually be slightly lower single-thread perf than OP's current Ivy Bridge CPU, and somewhere around half the single-thread perf of a modern (Alder Lake, Zen 3) CPU.

    Of course, some server workloads are highly scalable and don't care that much about single-thread perf. Would depend on what exactly OP is doing. But you can also skip that whole issue with a modern machine with higher single-thread perf in fewer cores. Keep in mind power consumption (bills!) may also be relevant in a home server.

    Getting an old workstation is an interesting option if you want to play around, but I suspect it's not the most budget-friendly option. Although it does have the advantage of ECC that you won't often find in desktop boards, if that's a desirable feature.

    Side note, the E5-2695 v2 you linked from AliExpress is an Ivy Bridge part, FCLGA2011 socket. That's incompatible with the FCLGA2011-3 socket used by the original Haswell Xeon. I assume you meant to find a E5-2695 v3?

    • 

      bigmicka on 09/04/2022 - 06:59

      Yes, I did mention power as being an issue.

      You are right about being an old architect. If you are running more than a few VM's and want them running all the time it is a good option. It also depends on their workload. Are they running at capacity over that time frame?

      The ram is also more scalable allowing up to 768gb.

      I did screw up the aliexpress link. Here is a v3 version https://www.aliexpress.com/item/32832278819.html?spm=a2g0o.p…

      It is not the right use case for everyone. But if you want a home lab that you can build on and your requirements are more for Cores/Threads instead of single-core performance it is not bad.

      I personally do both old and new.

• 

  forged on 14/04/2022 - 12:38

  Would recommend looking at used business / enterprise equipment, it’s better value than building new.

  I bought a $650 base spec HP ML350 Gen9 a while back, and added 2x E5-2678v3 (cheap) and 128GB DDR3 (expensive). Total cost a little over $1k excluding disks. 24 Haswell cores is great value for the money imo.