Project TinyMiniMicro is something I have wanted to look at for some time. There are a lot of home/ SMB server users out there who need large amounts of storage. Others simply need a server to run a WiFi AP controller, a VoIP solution, and a few other services/ development VMs. While there are some that need massive compute and memory footprints in single machines, there are others that prefer building smaller clusters. Project TinyMiniMicro is not for everyone, but many of our users are going to be wowed by the results. TinyMiniMicro comes from the fact that Lenovo, HP, and Dell are the primary manufacturers of these nodes and those are the tags they use respectively to indicate they are small form factor workstations.
Video Version
Since this has been a major undertaking at STH, we have a discussion of the units both in article format as well as a video version:
Of course, we have a bit more detail below that is harder to get all of the detail into a video.
What and Why Tiny/ Mini/ Micro PCs?
There are a lot of technologies out there that one would want several physical nodes. For example, one may want Proxmox VE deployed across more nodes for HA style setups. One option is to use nested virtualization, but sometimes this is not what one would want. Further, sometimes you may have a big node, and want two smaller nodes simply for offload purposes when the main node needs to go down for service.
This may seem crazy to use these PCs as servers, but maybe it is not. Chick-Fil-A came out and stated that they are using Intel NUC platforms for in-store Kubernetes clusters. You can read about that here. Intel NUCs are very hip and very popular, but they are also quite costly for what they deliver. The major PC OEMs each have platforms that follow a similar style, huge volumes, great quality, and excellent support. Frankly, while NUCs are trendy, TinyMiniMicro is the segment that drives volume and has vendor diversity.
My thought on the nodes is that basically:
- There are a ton of these corporate mini/ micro desktops
- Customers for them do regular upgrade/ refresh cycles plus there are events such as bankruptcies that lead to a constant supply on secondary markets
- They are likely to be sold by the large vendors in most countries due to their lower cost
- The build quality of these devices is fairly good and they are easy to service
- Allegedly, some even come with active onsite warranties. New units can be purchased with onsite warranties
When we look at cluster builds with the Xeon E-2200 series, such as with theHPE ProLiant MicroServer Gen10 Plus, something we noticed was that we either wanted more low power compute nodes, or we wanted smaller and lower cost nodes without 3.5″ bay support.
There is a lot to like here, but let us get to the obvious question: why not a Raspberry Pi 4 now that they are 8GB capable?
How Raspberry Pi 4 8GB Spawned this Project
For anyone that is thinking of building a small cluster of nodes, the Raspberry Pi ecosystem is awesome. They tend to be fairly inexpensive, small, and low power which makes integration extremely easy. When I heard that the Raspberry Pi 4 8GB model was launched, I immediately went to go order some. Part of that was seeing how it would impact our AoA Analysis Marvell ThunderX2 Equals 190 Raspberry Pi 4.
I ordered two kits from CanaKit, one was a barebones starter, one had a bit more functionality. Although the RPi 4 is advertised as a $75 device, the starter kit with the 8GB RPi 4 was around $140 before tax and shipping. I know you can get the components less expensively, but I like supporting the ecosystem and frankly having the components picked for me was worth a few extra dollars.
These units took a long time to arrive. When they did, I had kits of parts, not completed systems. So for just over $300, I had 8 low-power cores, 16GB of memory, in two nodes that I had to assemble and then had to set up power, add networking for, and so forth.
While I was waiting for them to arrive, I had the thought that perhaps I was struggling down a mentally inept path for what I was trying to accomplish. That is when I started looking at the TinyMiniMicro (TMM) nodes. What I found was an interesting comparison.
Here are some notes on the comparison:
- Raspberry Pi 4 8GB models may be $75, but once one buys all of the accessories to make a single node, they end up being $120-150.
- Some of the TinyMiniMicro units I have been buying in the $250 range have at least 4 threads and 8GB or 16GB of memory (upgradeable.) Usually, I have been getting 256GB SSDs with the units that are higher quality than the 32GB SD cards for the RPi’s.
- Expansion on TMM units usually allows a 2.5″ and a m.2 SSD
- While RPi’s are way better for IoT and such, as a lab, where power is a concern, but perhaps not the only concern, a 16GB/ 256GB x86 device at $250 is not that much of a worse value proposition than an 8GB/ 32GB (SD Card) RPi 4
- They are designed to be silent except under heavy loads
- They tend to use lower-power processors
- They are small and stackable
- A negative is that WiFi is optional where it is a core feature of the RPi 4
The more I looked into it, the more buying a TinyMiniMicro PC made sense. These had many features similar to the Raspberry Pi 4 8GB but effectively traded power consumption and a bit more size for a massively more robust platform.
By the time the Raspberry Pi 4 8GB kits arrived, I already had bought the first batch of TinyMiniMicro nodes and started iterating.
Project TinyMiniMicro Nodes
The first nodes I purchased were HP EliteDesk 705 G3 Mini units with the AMD Pro A6-8570E chips. These had 16GB of memory each and 256GB SSDs for around $225 after tax and shipping. Those are not great processors by any means. My thought was that I am usually more RAM than CPU limited, so it would be fine. This made me realize a few things. First, these TMM PCs have great build quality. The chassis on these inexpensive nodes has a single thumbscrew to get into and it is made mostly of metal giving it a surprisingly good quality feel. The units were small. The RAM used standard SODIMMs so that could be easily varied. There was room for a 2.5″ SSD, M.2 SSD, and even an extra M.2 slot for WiFi. The CPUs are actually socketed but HP has a nice cooling solution that runs fairly quiet. It even had multiple display outputs which makes sense as a corporate desktop.
There was a lot to like, and then the killer feature Windows 10 Pro. These corporate desktops have their Windows licenses tied to the motherboard. As a result, this $225 device has a Windows 10 Pro license which is worth $100 alone. With Windows 10 Pro one can join these nodes to a domain but also get remote desktop capabilities among other features. While some may scoff at Windows, Microsoft is doing an awesome job providing features such as WSL2 so you can run popular Linux distributions alongside Windows 10. That means you can have a Windows desktop that also is a lab node. Or if you are running Linux on it today, you can convert it into a Windows web browsing machine in the future.
While the cost is around $70-100 more than a RPi 4 8GB, you get a lot more machine for your money and in some cases more than two RPi 4 8GB models for less.
The processors on those nodes were anemic, so I then found two HP EliteDesk 705 G3 Mini nodes with AMD Pro A10-8770E CPUs for only around $200 each. 8GB SODIMMs sell for around $27-30 each so if you get an 8GB unit with 1x 8GB SODIMM, it is relatively inexpensive to get to 16GB.
Installing Linux on these machines was straightforward (although sometimes requires disabling security features meant to prevent this in corporate scenarios.) Within about an hour, I already had Kubernetes installed.
At that point, I had a 5-node cluster and realized, this was absolutely awesome which is how Project TinyMiniMicro was born.
Initial Nodes In Our Population
We have been working with about 15 different TinyMiniMicro nodes and 18 of the machines in total for more than a month. Here is the quick list:
Lenovo Tiny
- Lenovo ThinkCentre M710q Tiny
- Lenovo ThinkCentre M715q Tiny
- Note STH reviewed theM720q Tiny previously
HP Mini
- EliteDesk 800 G2 Mini
- EliteDesk 705 G3 Mini (AMD A6)
- EliteDesk 705 G3 Mini (AMD A10)
- EliteDesk 800 G3 Mini
- EliteDesk 705 G4 Mini
- EliteDesk 800 G4 Mini
- ProDesk 600 G4 Mini
- ProDesk 400 G4 Mini (was being sold as an EliteDesk 800 G4)
- ProDesk 405 G5 Mini
Dell Micro
- OptiPlex 7070 Micro
- OptiPlex 7060 Micro
- OptiPlex 5050 Micro
- OptiPlex 3050 Micro
All of these nodes are a similar size, use external power bricks usually shared with their respective company’s notebook lines. There are usually internal spots for 2.5″ and NVMe SSDs as well. Almost all of them also have the ability to customize the display outputs. All have DisplayPort, some have a mix of HDMI and VGA as well. All of our test units fall in the 10W or so idle to 64-68W maximum power consumption range even while spanning 2-6 cores. Performance ranges from somewhere in the Atom C3000 range to the lower-middle of the Xeon E-2200 range.
Having this many different units makes it abundantly clear that this is a highly competitive segment where the companies are investing heavily in design and features. All have excellent documentation. Most consumer-level and server-focused SFF cases would not be competitive in this market.
Management Details
While typical servers have built-in IPMI and Redfish management, along with features such as BMCs, some of the TinyMiniMicro PCs have additional management features.
Many of these nodes have Intel vPro and AMD DASH. Both let you do remote power on/ off. Intel vPro is probably the more robust solution. Neither is as good as a standard server BMC, however, there are some benefits, namely lower idle power consumption.
The management situation is one we are going to look into more deeply with the nodes. Depending on the chipset used, and the markets that the particular units can be exported to, one may or may not have full Intel AMT/ vPro features.
Final Words
If you purchase one of these units, especially second-hand, you may end up with a great deal, but there can be disasters as well. For example, one of the HP models arrived and it was a similar yet different model. Some were missing parts as well. Others had very interesting default BIOS settings.
By getting a relatively large population, we were able to see many of the configuration differences and purchasing pitfalls beyond just what one may see on a single listing. We have had forum users working with these nodes for years, so we decided to purchase a large set of nodes (and they are still arriving) so we can share in-depth insights into each model along with some of the general trends and potential pitfalls of the space.
Perhaps the most important aspect of this project is that these nodes can be purchased new and deployed similar to how Chick-fil-a does with the Intel NUCs, with the added benefit of getting on-site support to further bolster warranty and remote service and support. While we are discussing lower-cost options here, there are many organizations that can deploy these edge compute solutions easily since they already purchase the Lenovo, Dell, and HP units with volume discounts. The fluidity of being able to use the nodes as desktops or servers is something that the Raspberry Pi lab folks often touted as an advantage.
This was just a quick piece to introduce the new project now that we have over a month of testing this large population. Expect more to come on what we have learned from testing all of these units for our Project TinyMiniMicro home lab revolution.
