So I recently (a couple months ago) moved my fragmented docker-on-raspberry-pi architecture over to a Proxmox cluster. I'm running it on a pair of HP DL360 G6s, and I couldn't be happier.
Except, well, I could be happier with just one more thing: high availability.
In particular, I want HA for my OPNsense firewall/router, but eventually for more of the workloads my family are depending on for life in general - Home Assistant, Plex, Overseerr, Immich, etc etc.
My current storage setup is a couple ratty old ARM-based NASes - an ancient Netgear ReadyNAS and an even more ancient Qnap TS-410. They're both populated with 4 x 4TB (max raw size they can take) using RAID5, so I get about 22TB usable across the pair of them. They mostly store media for my Plex setup, but also support my 2N+C backup strategy for stuff like Immich, Paperless, and other important data.
My high-level plan is to grab another DL360, so I can have a quorum, then introduce a new storage system that:
provides an iSCSI target for my Proxmox cluster; and
can eventually grow to replace my old NASes.
The two solutions I'm pondering are:
Build a TrueNAS setup from scratch - mini ITX case, board - the lot
Pickup something tried, true and proven in the market, like a Synology
Up front cost is a consideration - I have a family to feed, so I can't just run out and buy an 8-bay enclosure and fully populate it with 16TB disks.
Whatever I get, I'm likely to want to start with, say, 3 disks and grow it over time.
So, I guess this is a call out to the community to share any and all successes, war stories, and other advice. The more technical, the better. I want to make a sound, data-based decision here, and anecdotes from others who think like me are the best way to set my compass.
High availability storage is what the Ceph integration is for.
Edit: though it's kind of a pain to set up and probably way overkill. A separate TrueNAS or similar appliance with a 10 gig uplink will be easier and probably just as reliable for your use.
I did look at Ceph as well, but balked at the 10gig requirement. I only have a couple of SFP slots free on my switch, which otherwise only provides 1Gbps sockets. Could be a future upgrade path, though, if I want to get to play with full HCI.