[Brad]

It finally happened. I was hit hard by two drive failures over the last month, and now I'm figuring out a better plan going forward.

At home, I have a tower PC running in the corner of a room that I use as a very poor man's NAS; it's actually just an old Hackintosh with its drives shared over SMB. Its boot drive is an SSD, which is fine, but I've been using a 4TB HDD as its main shared document and media storage for many years. That HDD started to fail a few weeks ago, and I immediately scrambled to transfer whatever I could onto another external HDD that I've used for occasional backups but never bothered automating fully.

Well, fast forward two weeks, and that second external drive started failing. When it rains, it pours! Fortunately, I had another new-in-box drive that I had been planning to use as the middle part of the classic 3-2-1 backup strategy. I've been able to get everything from the second drive onto that third drive, but now I'm back to only having one reliable copy of everything.

Oof.

I've had this idea for a long time to build/repurpose a PC as a dedicated NAS with more fault-tolerant software and hardware for the "live" copy of the data (I know I'd still need to automate external backups), and this latest crisis has prompted me to take action. ZFS has been at the top of my list, but there's a lot to learn about proper management. I'll want to give it multiple HDDs so it can do full parity, and my understanding is that I'll need to give its system lots of RAM, preferably the more expensive ECC kind, for best performance.

I could (okay, I should) take the time to learn all the ins and outs of ZFS zpool and vdev management right now, but I kind of want to just get something up and running quickly and learn more about management best practices along the way. That's what led me to TrueNAS (formerly FreeNAS). It's a Linux distro that's built to get you running with ZFS and various helper and monitoring tools out of the box, and it's supposedly tuned and locked down specifically for storage management. A lot of folks in the /r/datahoarders community use and recommend that distro especially for folks just getting started running their own NAS.

Does anyone here have any experience with ZFS, TrueNAS, or similar high-reliability data storage solutions? I know that I could buy a dedicated NAS from a company like Synology, but I don't like the idea of it just being an appliance. I want to be able to get in there and learn, tinker, troubleshoot, and further optimize things.

What do you use for shared storage around the home network? Any other ideas I might not have considered?

[turtle]

So I'm the guy who is gonna say Synology because I'm tried of having to do my job at home and just don't want to mess with it.

Synology is not the answer if you want to tinker and learn though. For me, it means I don't have to and I have redundancy. I have two NASs running with some cross copy. The critical items are actually backed up in iCloud Drive really. I'm not following the full proper backup strategy I would/do for work, because this isn't work and I have a higher fault tolerance.

I did look at FreeNAS back when I moved to Synology, but the convenience, stability and feature base just really stood out as the best for my situation.

[kscherer]

Hmmm … seems like you're overcomplicating the storage/backup solution.

But, you're Brad, and you know things/want to learn things.

Otherwise, I'd recommend an OWC enclosure that you can build out the way you want. Buy the box, make it RAID 5, and install your own drives. They're reliable and fast. With some of them you do also have to get a SoftRAID license, although this one comes with it (these are the units that we sell and customize for our clients). I think I've built out about ten of these in the last 2 or 3 years. They're easy to deal with, and SoftRAID works with both Mac and Windows. It isn't "Tinker Town" stuff, but SoftRAID gives you a lot of options as far as RAID monitoring and notifications, etc.

And, yeah, your "oops" are yours!

[chucker]

I'll preface this by saying I'm sorry you've had drive failures, buuuuuuuuuut.

Originally Posted by Brad I know that I could buy a dedicated NAS from a company like Synology, but I don't like the idea of it just being an appliance. I want to be able to get in there and learn, tinker, troubleshoot, and further optimize things.

nonononono stahp

Buy a NAS. You're not a teenager any more. You have real data, important data on there, and you don't have the time to play janitor for some custom-rolled community junk.

Install Docker on Synology if you need something to tinker around with. Or rent a VPS. But don't roll your own NAS.

Use SHR on top of btrfs and worry about things that actually matter in life.

[drewprops]

I want in on this party!!!!

I am an idiot!!!!!!

This may be my guide...

https://www.nytimes.com/wirecutter/r...ached-storage/

...

[chucker]

I would go four-bay if you can wing it, not two-bay.

[kscherer]

I agree with Chucker. A 4-bay gives you a solid RAID 5. Redundancy and speed for the win!

[turtle]

Yeah, I'm all for larger from the start since I always go diskless. This allows me more room to expand. I started with a DS413j back in the day. I now have a "normal" middle range (not j or + series) and a "+", which is 6 bay.

SHR Is not the same as RAID 5 or any of the others. Somewhere I read a great comparison, but I'm not going to dig it up now. I went with SHR and haven't looked back. I've had disks fail and no issues. I simply remove the bad disk and replace it with a new one and normally a larger one giving me more storage than I had before. It syncs the changes rebuilding the array with the new capacity and I'm back up and running.

It is slow on the start though, rebuilding and recovering from a disk failure will take time with larger disks in the unit.

[drewprops]

Are these wireless now, or must you be physically connected to the drive at all times?


...

[chucker]

Originally Posted by drewprops Are these wireless now, or must you be physically connected to the drive at all times? ...

Just plug them into your router via Ethernet, then access them via Wi-Fi. (Time Machine will work across Wi-Fi, too.)

[Brad]

Originally Posted by turtle So I'm the guy who is gonna say Synology because I'm tried of having to do my job at home and just don't want to mess with it.

Excellent! Then maybe you can help answer some of my questions about Synology that aren't easily found in their marketing materials.

Two main features I would use if I build my own NAS are 1) mirror or full parity across multiple drives and 2) the ability to add larger drives, migrate data off of the smaller ones, and remove the smaller ones to free up slots for future larger drives. The former to protect against hardware failures and bit rot which is actually what happened to the first of my two drives; it has a couple of catastrophically bad sectors that are impossible to read now no matter what tools I use (the drive physically stops responding). The latter to facilitate easier expansion over time. So, I'd want a PC or a NAS with at least 3 or 4 drive bays, though probably only two bays would be occupied most of the time.

It looks like SHR would fit the bill for at least the first case. How well does it copy or rebalance when adding new drives and removing old ones? From what I can tell, Synology doesn't support ZFS, which itself isn't a deal breaker, but I know ZFS has commands to add/remove devices and copy/rebalance data pretty seamlessly.

Also, since ZFS checksums everything, when it periodically scrubs to verify its checksums, it can use the parity data to automatically fix most damage. Do you know if SHR has any similar automatic data loss detection and recovery process?

Originally Posted by turtle The critical items are actually backed up in iCloud Drive really.

My most critical documents are periodically archived and encrypted using borg and synced to Google Drive with rclone. I have 1TB of free storage in Google Drive since I have Google Fiber, and I'm using about 185 GB for my encrypted archive, but ideally I'd commit several TB in a proper 3-2-1 strategy. Going forward I'm thinking about getting a Backblaze account and shoving everything in the cloud and not relying on Google's infra.

Originally Posted by kscherer Hmmm … seems like you're overcomplicating

You should see me in the kitchen trying to decide what to make for dinner!

Originally Posted by chucker nonononono stahp Buy a NAS. You're not a teenager any more. You have real data, important data on there, and you don't have the time to play janitor for some custom-rolled community junk.

I hear you. Really, I do! But some of the best software out there is "custom-rolled community junk"! Every wifi router I've owned for at least the last decade has gotten upgraded with custom firmware because the stock firmware was slow, anemic, and late (if ever!) to get security patches. I figure these NASes are all just running Linux with a bunch of open-source tools anyway with a shiny branded UI on top. So, why not go straight to the source and trim the fat?

That's my first line of thinking, of course. Yes, buying a prefab appliance means a much lower investment in setup time and maybe some kind of official support channel if things go wrong. I don't relish the idea of maintaining vdevs and zpools and all that stuff. It'd be fun to learn, but I don't want to spend more time troubleshooting than I have to. But I also don't want to drop a bunch of cash on an appliance only to be gobsmacked with buyer's remorse, realizing I could have done it better myself. I really do want to hear your experiences with Synology to teach me what I probably need to know.

As for tinkering, I don't mean that I'd be tinkering on it as a live system. Even if I go with TrueNAS, I'd be running through its setup process in a VM and getting comfortable there long before installing it on my "production" hardware.

Originally Posted by chucker Install Docker on Synology

Ah, good! You've preempted another one of my questions about Synology devices. Now that I'm looking that up on their site, I'm glad to see that virtualization and Docker in particular are headline features, but I'm a little concerned that they're only available on certain models. I'll have to be very careful choosing one because the ~$370 DS418 I was originally looking at doesn't appear to support virtualization, but the nearly identical ~$500 DS420+ does. Weird.

Originally Posted by chucker btrfs

In my on-again, off-again research (I've been quietly reading about this stuff for months), I came across a lot more horror stories involving btrfs than ZFS. I read about how Red Hat abandoned it a few years ago because maintaining it was a huge pain, and it supposedly has had several serious (but arguably edge-case) bugs, and that all gives me pause. Do you use btrfs? On what kind of scale/load?

Originally Posted by drewprops I want in on this party!!!! I am an idiot!!!!!!

Captain on deck! One of us. One of us!

[Brad]

Originally Posted by turtle SHR Is not the same as RAID 5 or any of the others. Somewhere I read a great comparison, but I'm not going to dig it up now. I went with SHR and haven't looked back. I've had disks fail and no issues. I simply remove the bad disk and replace it with a new one and normally a larger one giving me more storage than I had before. It syncs the changes rebuilding the array with the new capacity and I'm back up and running. It is slow on the start though, rebuilding and recovering from a disk failure will take time with larger disks in the unit.

Well, I started typing that last post before dinner, walked away, came back hours later, and hit submit without checking for new replies. It looks like this might answer some of my questions. Thanks for the link!

[turtle]

Originally Posted by Brad Excellent! Then maybe you can help answer some of my questions about Synology that aren't easily found in their marketing materials. Two main features I would use if I build my own NAS are 1) mirror or full parity across multiple drives and 2) the ability to add larger drives, migrate data off of the smaller ones, and remove the smaller ones to free up slots for future larger drives. The former to protect against hardware failures and bit rot which is actually what happened to the first of my two drives; it has a couple of catastrophically bad sectors that are impossible to read now no matter what tools I use (the drive physically stops responding). The latter to facilitate easier expansion over time. So, I'd want a PC or a NAS with at least 3 or 4 drive bays, though probably only two bays would be occupied most of the time. It looks like SHR would fit the bill for at least the first case. How well does it copy or rebalance when adding new drives and removing old ones? From what I can tell, Synology doesn't support ZFS, which itself isn't a deal breaker, but I know ZFS has commands to add/remove devices and copy/rebalance data pretty seamlessly. Also, since ZFS checksums everything, when it periodically scrubs to verify its checksums, it can use the parity data to automatically fix most damage. Do you know if SHR has any similar automatic data loss detection and recovery process?

So the short version of the long answer is SHR does all the thinking for you. It will alert you if there is an issue (assuming you configure alerting methods) and let you know if a volume is in trouble. If you want to replace a drive, for data capacity reasons or failure, just pull the drive and replace it. You will be walked through initializing the drive and then it will rebuild. You will be good to go, as long as you don't lose another drive during that process. If you want to expand the volume and have an open bay, just add a drive to the NAS. It has some basic steps to initialize it and then rebuild and increase capacity.

My most critical documents are periodically archived and encrypted using borg and synced to Google Drive with rclone. I have 1TB of free storage in Google Drive since I have Google Fiber, and I'm using about 185 GB for my encrypted archive, but ideally I'd commit several TB in a proper 3-2-1 strategy. Going forward I'm thinking about getting a Backblaze account and shoving everything in the cloud and not relying on Google's infra.

I should encrypt and then store the encrypted package, but I'm not there yet. It just isn't worth the effort to me and I'm "trusting" Apple to keep it safe. I do however retain a copy on my NAS as part of my normal replication process. I also used Synology Drive which is a private version of Dropbox but stored on your NAS. You can even share public links to files hosted in Synology Drive.

I hear you. Really, I do! But some of the best software out there is "custom-rolled community junk"! Every wifi router I've owned for at least the last decade has gotten upgraded with custom firmware because the stock firmware was slow, anemic, and late (if ever!) to get security patches. I figure these NASes are all just running Linux with a bunch of open-source tools anyway with a shiny branded UI on top. So, why not go straight to the source and trim the fat? That's my first line of thinking, of course. Yes, buying a prefab appliance means a much lower investment in setup time and maybe some kind of official support channel if things go wrong. I don't relish the idea of maintaining vdevs and zpools and all that stuff. It'd be fun to learn, but I don't want to spend more time troubleshooting than I have to. But I also don't want to drop a bunch of cash on an appliance only to be gobsmacked with buyer's remorse, realizing I could have done it better myself. I really do want to hear your experiences with Synology to teach me what I probably need to know. As for tinkering, I don't mean that I'd be tinkering on it as a live system. Even if I go with TrueNAS, I'd be running through its setup process in a VM and getting comfortable there long before installing it on my "production" hardware. Ah, good! You've preempted another one of my questions about Synology devices. Now that I'm looking that up on their site, I'm glad to see that virtualization and Docker in particular are headline features, but I'm a little concerned that they're only available on certain models. I'll have to be very careful choosing one because the ~$370 DS418 I was originally looking at doesn't appear to support virtualization, but the nearly identical ~$500 DS420+ does. Weird.

This has a lot more to do with the CPU used vs other specs. My + is capable of running docker containers. In fact my TT-RSS feed reader is running on my NAS in docker. So is my primary Homebridge instance. I even had a WordPress blog running there for a little bit. Very capable though the CPU is only so much. I put most on a purpose ESXi host that the ANMC server is running on.

In my on-again, off-again research (I've been quietly reading about this stuff for months), I came across a lot more horror stories involving btrfs than ZFS. I read about how Red Hat abandoned it a few years ago because maintaining it was a huge pain, and it supposedly has had several serious (but arguably edge-case) bugs, and that all gives me pause. Do you use btrfs? On what kind of scale/load?

I've NEVER had issues related to the file system. I don't recall when Synology moved to btrfs but I've been on it from the time either I've has a NAS or it was part of the DSM package. I've had drives (begin to) fail but never a failure of the file system itself. One volume is ~38TB and the other is ~16TB. No issues at all with them. I even host the ANMC Maps on these NAS's so the inode management can handle thousands of tiny files just fine. The NAS also holds my family media. I have a large family and we have many different drives streaming (via Plex) from the NAS stored media.

If you were to ask me what you should get, based of off what I know of you then this is what I would recommend: No lower than the "Value series" and a minimum of 4 bay. These are the ones without the "j" or "+". If you want to be able to do more, then I would really recommend a "+" with same minimum of 4 bays. Mine has six and I'm feeling cramped. Go diskless and shuck drives. Ideally you would use NAS rated drives, but the cheapest large externals being shucked have always served me well. No need for "red" drives when most externals are "black" and flawless anyway.

If you have other questions about Synology or how their tech works day to day feel free to ask. I use mine for many many wonderful things!

[drewprops]

Where do Y'ALL buy your NAS? Your drives? (I'm assuming you have to fill them yourself)


...

[Brad]

Originally Posted by turtle (lots of really good answers and info)

Thanks again, T! This really puts my fears at ease and nudges me closer to the "just buy the damn appliance" camp.

Originally Posted by turtle Go diskless and shuck drives.

I'm so glad you used the term "shuck" here. In my circle of IRL friends and colleagues, I think I'm the only person ever to use it this way!

Originally Posted by drewprops Where do Y'ALL buy your NAS? Your drives? (I'm assuming you have to fill them yourself) ...

Yeah, a "diskless NAS" has no hard drives, and you have to fill it with your own. That's generally the way to go because bundled drives are always going to be overpriced.

Speaking of shucking drives, see https://shucks.top

(Alternatively https://diskprices.com/ for many more options)

The term is borrowed from oyster shucking. Many high-capacity external drives actually have really good hardware inside them, and you have to pry the delicious innards out of the hard plastic case. Inside the plastic is just a normal desktop hard drive that you can install into a PC or a NAS.

Here's a video explaining how to non-destructively shuck a Western Digital Easystore external drive. I followed these steps for another one of my drives (not one of the ones that failed) several months ago: https://youtu.be/pMudTWoMvsY

[drewprops]

I want to follow in your footsteps, could you please keep this conversation going as you implement?

I'm certain that I do not have the overhead to learn how to customize the OS of a NAS - and must trust that the stock setup will work great for someone simply looking for a solid setup.

Heck, I still need to figure out a clean routine for backing up my 5TB external road warrior drive.

But please keep posting!


...

[Brad]

👍👍👍

Once I'm done shuffling data between these three drives and get back into my previously normal routine, I'll come back and also share my borg+rclone setup for backing up and uploading my most critical data to the cloud.

Whether I end up with a Synology NAS or something else, I'll definitely share my unboxing and setup experiences.

[Ebby]

Originally Posted by turtle I've NEVER had issues related to the file system. I don't recall when Synology moved to btrfs but I've been on it from the time either I've has a NAS or it was part of the DSM package. I've had drives (begin to) fail but never a failure of the file system itself.

I also run a couple Synology units for ages but have had hiccups. I've had the OS fail requiring reinstall but all MY data/drives were restored easily using their Hyper Backup app. However, system settings and most app configs were not backed up as I had hoped and I did spend a day redoing all that annoying stuff.

BTRFS was an upgrade/default right before DSM version 7 IIR and I see it as a good upgrade. Snapshots and stuff make for easy backups especially on my VM's. The only bug I've experienced is with BTRFS and Plex in a Docker container. The tmp folder and btrfs didn't behave together and created a 5TB container that was a bit hard to track down because you have to SSH with admin to access the entire directory. After finding the rogue data, it was a simple fix.

Docker has a few quirks, but it is getting better. The DSM 6 version didn't have IPv6, but DSM 7 does. I consider it beta at this point because updates change/break things occasionally. Still, you get lots of functionality the official app store doesn't. There is no docker compose, and lots of crap doesn't install/work with the GUI tools. I've had the most success installing by script. Plex in a container works better than the package center version which is always out of date. Portainer seems to work better for me than the native app. Oh, even have had a minecraft container running for a few years.

Virtualization does take a bunch or resources. The CPU isn't... well it works for a NAS but loading an OS loading an OS takes some sweeeeet time. I run DSM 6 running DSM 7 in a VM. There are some app features I like that are no longer supported in DSM 7 and others that didn't exist in DSM 6. But once its up and running, Its nice and snappy. I upped the RAM myself from factory. It uses pretty available modules. There are CPU limitations so watch out for those. You can install more RAM than the CPU can address (if memory serves) and once it hits that threshold, weird things happen.

The Hyper backup app also has rsync destination capability along with cloud services like google too. Encrypted optional.

I used to use Cloudstation to sync data, but they converted to a app called Drive which didn't behave as I needed. I swapped everything to Syncthing and find that superior for syncing.

Hope this helps!

[drewprops]

Originally Posted by Ebby Hope this helps!

You just scared me away again


...

[Brad]

Originally Posted by Ebby I also run a couple Synology units for ages but have had hiccups. I've had the OS fail requiring reinstall but all MY data/drives were restored easily using their Hyper Backup app. However, system settings and most app configs were not backed up as I had hoped and I did spend a day redoing all that annoying stuff.

How do OS upgrades on the Synology normally happen? How would you "factory reset" and reinstall it from scratch? Is there a way to back up its settings, or are you just stuck redoing everything manually like you experienced?

Also, are you able to get a root shell (ssh?) into its system (not your files)? Or are you limited to what you can poke in their GUI?

Originally Posted by Ebby The only bug I've experienced is with BTRFS and Plex in a Docker container. The tmp folder and btrfs didn't behave together and created a 5TB container that was a bit hard to track down because you have to SSH with admin to access the entire directory. After finding the rogue data, it was a simple fix.

I'll have to be on the lookout for that. I use jellyfin which is an open source product very similar to Plex, and I would likely run it on the NAS in Docker like this too.

Originally Posted by Ebby There is no docker compose, and lots of crap doesn't install/work with the GUI tools. I've had the most success installing by script.

Maybe this is answering my earlier question about root shell access, but could you install docker-compose somewhere via command line? Or are you limited to what the GUI apps provide?

Originally Posted by Ebby Virtualization does take a bunch or resources. The CPU isn't... well it works for a NAS but loading an OS loading an OS takes some sweeeeet time. I run DSM 6 running DSM 7 in a VM. There are some app features I like that are no longer supported in DSM 7 and others that didn't exist in DSM 6. But once its up and running, Its nice and snappy. I upped the RAM myself from factory. It uses pretty available modules. There are CPU limitations so watch out for those. You can install more RAM than the CPU can address (if memory serves) and once it hits that threshold, weird things happen.

This was something I was concerned about too. I see that all Synology products have the CPU soldered to the board, and the "mid range" DS920+ I was just looking at has a Quad-Core 2GHz Celeron J4125. At least I could upgrade the stock 4GB RAM to 8GB. Totally fine for the storage side of NAS, like you said, but I'm surprised it would run well with demanding services like Plex (I assume you're not transcoding anything in HD?) and Minecraft (just a couple LAN players?). If I were to build something, of course I'd have the freedom to throw any chip in there, and I'd start with something way better than a Celeron (at least an i3 or i5).

Originally Posted by Ebby Hope this helps!

It does! Thank you!

[turtle]

Ebby's story does remind my of why I moved from the DS413j to the DS418, it got wet! Remember me posting about it raining on my network rack a while ago? That is what killed my DS413j:

The logic board was swimming. The amazing part, my data was all retained. I literally got my new NAS and put the drives into the slots in the same order I pulled them out of the old one. Booted and I had all my volumes intact. I did have to reconfigure the DSM with the move, but I didn't lose the data.

Originally Posted by Brad How do OS upgrades on the Synology normally happen? How would you "factory reset" and reinstall it from scratch? Is there a way to back up its settings, or are you just stuck redoing everything manually like you experienced?

It is a normal software update like anything else. A few minutes to download, process and reboot. It can take up to 30 minutes and there isn't really any progress indicators of it, but it works well. I've done it from DSM 4 to being on 7 now.

Also, are you able to get a root shell (ssh?) into its system (not your files)? Or are you limited to what you can poke in their GUI?

You create an admin user and shell in with that user. You can then sudo if needed. Of course, your primary use might be an admin too, but I have a specific user that is my main admin for the NAS that isn't limited at all. I sudo to root as needed/wanted from there. Really though, there is little that is needing root in shell. Scripting can all be done by normal users and kicked off with "Scheduled Tasks" in the DSM control panel.

Maybe this is answering my earlier question about root shell access, but could you install docker-compose somewhere via command line? Or are you limited to what the GUI apps provide?

Yes! I use docker-compose for my tt-rss install in fact! I will caution this though, paths are not the same in the NAS as they would be in a *nix docker install. You will have to work through the paths for certain things. Might not be a problem for you, but I'm not very good with docker so it is a massive learning curve for me.

MyAdmin@borg:/volume1/docker/ttrss/ttrss-docker$ ls -lahtotal 24Kdr-xr-xr-x+ 1 root    root   178 Jul 20 08:54 .drwxr-sr-x  1    1000  1000   86 Jan 31 17:24 ..dr-xr-xr-x+ 1 root    root    84 Jul 20 08:38 appdr-xr-xr-x+ 1 root    root    54 Jan 30 21:04 config.d-r-xr-xr-x  1 root    root  1.4K Jul 20 08:54 docker-compose.yml-rwxr-xr-x  1 root    root   883 Jan 14  2022 .env-r-xr-xr-x+ 1 root    root  1.8K Jul 20 08:38 .env-distdr-xr-xr-x+ 1 root    root   234 Jul 20 08:50 .git-r-xr-xr-x+ 1 root    root   117 Jul 20 08:38 .gitignore-r-xr-xr-x+ 1 root    root  5.7K Jul 20 08:38 README.mddr-xr-xr-x+ 1 root    root    60 Jul 20 08:38 srcdr-xr-xr-x+ 1 MyAdmin users   36 Feb  4 18:43 themes.localdr-xr-xr-x+ 1 root    root    64 Jan 30 21:06 web-nginxMyAdmin@borg:/volume1/docker/ttrss/ttrss-docker$

If you have to rebuild:

root@borg:/volume1/docker/ttrss/ttrss-docker# docker-compose down && docker-compose rmroot@borg:/volume1/docker/ttrss/ttrss-docker# docker-compose pull && docker-compose up -d

This was something I was concerned about too. I see that all Synology products have the CPU soldered to the board, and the "mid range" DS920+ I was just looking at has a Quad-Core 2GHz Celeron J4125. At least I could upgrade the stock 4GB RAM to 8GB. Totally fine for the storage side of NAS, like you said, but I'm surprised it would run well with demanding services like Plex (I assume you're not transcoding anything in HD?) and Minecraft (just a couple LAN players?). If I were to build something, of course I'd have the freedom to throw any chip in there, and I'd start with something way better than a Celeron (at least an i3 or i5).

I had to remind myself that this is a NAS, not a full on server. Yes it can do many things, but in the end it is a NAS. I will (and do) have a server for heavy compute tasks. This is why my Plex server is now on dedicated hardware. My VMs run on ESXi host and such. Things like Homebridge are lightweight and EASILY handled by the NAS with docker, so there it sits. I will say, buying bigger is better since you can't upgrade the hardware but in the end you really have to know what purpose you're buying the hardware for and go from there. If you plan to run media on the NAS, you REALLY want a CPU that can do the hardware transcoding. It makes a huge difference.

[turtle]

Oh, one more thing: certain folders on the NAS are volatile so don't store anything in "root's" home. Always store scripts and such in your user (or service account user's) home folder.

[Brad]

Ruminating about Synology and the possibility of a DS920+…

I've been reading and watching lots of reviews for Synology's hardware and software, and I'm definitely not a fan of what I've seen of their management software, but hopefully I wouldn't have to interact with it very often once things are in place. Too many years of working closer to bare metal with command-line Linux sysadmin stuff makes me feel like their Windows-style GUI is a few too many levels abstracted away from what I want to do. As for the hardware, I've learned that most of their products are on a two-year refresh cycle as can be seen by the last two digits in their models, and the DS9xx+ series is due for an update any day now.



2016=DS916+… 2018=DS918+… 2020=DS920+… so, maybe 2022=DS922+?

Some of their other products have been updated this year, and I imagine the buyer's remorse would be pretty intense since I'm already on the fence and considering building a much more powerful NAS from scratch for the same price. I'm also more than a little disappointed by their insistence on using Synology-branded or Synology-approved products for everything potentially user-serviceable (there's even a list of approved third-party hard drives its software checks) or else you may void the warranty and/or have to work around some settings and warnings. I guess that's not too surprising considering their business model, but sheesh.

On the bright side, I'm a bit less worried about btrfs after reading up on recent experiences from many folks. I think I'd still prefer ZFS, but btrfs+SHR will do fine if it's the next best option.

I also have a different problem of figuring out where I'd put this thing. We just use wifi for everything in our house, and this has me thinking it may finally be time to hire someone to run ethernet cables to a few strategic locations. If I'm going to do that, then I'm certainly in no hurry to drop a thousand bucks on hardware since I won't have a good place to use it. Have any of you run cables through residential walls from a ground floor to a second floor? (We have a small cape cod style house.)

Also…

Originally Posted by turtle I had to remind myself that this is a NAS, not a full on server. Yes it can do many things, but in the end it is a NAS. I will (and do) have a server for heavy compute tasks. This is why my Plex server is now on dedicated hardware. My VMs run on ESXi host and such. Things like Homebridge are lightweight and EASILY handled by the NAS with docker, so there it sits. I will say, buying bigger is better since you can't upgrade the hardware but in the end you really have to know what purpose you're buying the hardware for and go from there. If you plan to run media on the NAS, you REALLY want a CPU that can do the hardware transcoding. It makes a huge difference.

Thanks for the examples and details around your workflow, shell, docker, etc. Yeah, I think if I get a Synology NAS, I'm pretty sure I'd end up putting a PC server right next to it to do the heavy lifting for things like transcoding.

[turtle]

I never saw that graphic with the naming convention. I kinda knew it after years of looking at them and working with them though. It starts to stand out after a while. I would wait for the DS9XX+ to upgrade if that is you desired model though. It should be due for an upgrade release in the near future. Looking at the specs for that one (DS920+) it does support hardware decoding. Also, while looking for other things I found this article that covers a few points for you, like Docker Compose! Granted, it is for Plex, but it clearly walks though the process for using compose that can be applied to other things.

For the location, you could just put the NAS near your router. The ones I have a near silent. Not full silent, but pretty quiet. Granted, mine now live in the basement away from all of use, but even in Va Beach where it was in my utility room it was just near the router.

One time I did have to run a wire from a lower floor to an upper floor, I went up through a closet into another closet and tucked the cable between the baseboard and the tack strip of the carpet. This hid the cable and still got me networking where I needed it. I did it close to the door of the closet so you would have to stick your head in and look back at the door frame to see it. Not the most professional, but it worked very well!

As you get closer to where you are looking to buy something, feel free to ask questions from me. I'd be happy to help with what I can.

I will say though, you can buy a high end Synology NAS that is really running full server grade hardware and able to combine everything under one appliance. I lean away from that personally because compute power needs grow faster than I'm willing to buy storage. So a mid/high end NAS will last for YEARS to come where you might want to upgrade the compute server hardware more frequently.

[Ebby]

Sorry for the delay, I was camping this weekend. Completely offgrid.

Originally Posted by Brad How do OS upgrades on the Synology normally happen? How would you "factory reset" and reinstall it from scratch? Is there a way to back up its settings, or are you just stuck redoing everything manually like you experienced? Also, are you able to get a root shell (ssh?) into its system (not your files)? Or are you limited to what you can poke in their GUI?

As mentioned, updates are easy via webpage and never really had a problem with them. Synology is always patching for bugs which is good. They even support the old DSM 6 a bit too.

You can get root access if I recall but have to jump through a hoop or two. I usually just sudo my user and that's usually enough. From what I remember, the OS is sandboxed from the data volume. There is a reset button on the back and you have to press the button in the right sequence to wipe the OS and start from scratch. It boots to a webpage that asks for a DSM image. That process was easy, but the settings took forever. Buuuut yeah, you can download most of the OS settings as a text file I believe. I just didn't before problems popped up

This was something I was concerned about too. I see that all Synology products have the CPU soldered to the board, and the "mid range" DS920+ I was just looking at has a Quad-Core 2GHz Celeron J4125. At least I could upgrade the stock 4GB RAM to 8GB. Totally fine for the storage side of NAS, like you said, but I'm surprised it would run well with demanding services like Plex (I assume you're not transcoding anything in HD?) and Minecraft (just a couple LAN players?).

I have a DS218+ with 6GB and a DS1618+ with 8GB ram. While the DS1618 has a faster CPU, it struggles with more than 3 transcodes. (I do watch parties over the web, so they are not HD and limited by my upload speed) Its fun, and good but not the best quality. The DS218 has a slower CPU, but can handle transcoding HD better because it has hardware acceleration. Many cheap-ola Synology models have hardware encoding and Plex can use that.

The DS920+ you mentioned looks like it has hardware encoding support if your apps work with it. In docker, you have to do a little extra wizardry to allow the container access to hardware, but I heard success stories. However, my DS218+ is not powerful enough to handle bigger things like that minecraft server or virtualization.

[drewprops]

I never appreciated what network area storage entailed until this thread. These boxes aren't just multi-bay hard drives with an Ethernet connection, they're flipping computers, with options for different OSes and unique disk formats. Even media machines.

Turtle, they may not be full blown servers, but I'm buffaloed by the technical acumen required to make these perform.

Maybe I could get one and run it full vanilla, getting accustomed to the device, but there's a lot of new knowledge to acquire, and while I know that I'm capable, I simply don't have the bandwidth for that right now.

But this is still fascinating to follow.

...

[turtle]

Drew, they really are server but generally running super low end CPUs. More than sufficient for general tasking and data management. The CPUs just fail to handle heavy load for real intensive compute tasking. Some of the Synology servers are in fact "server grade" with the "xs" series. I just lean away from those because that is too much all in one for my taste. I want more flexibility with my compute resources.

For you, I would totally recommend you get a 4 bay "Value Series" (no "j" or "+" on the end) and plug it into your router. Start with three disks so you have a greater fault tolerance and get familiar with the system.

These Synology NASs running DSM can be your email server, web server, docker host, DNS, DHCP, etc. DSM is really built to be able to do it all and do it well. I don't utilize many of the features available for many reasons, namely I have purpose built components to handle specific tasking. I also grew into this, I didn't start where I am now. Using the Synology Drive is awesome because it negates the need for Google or Dropbox from managing my data. However, I have one of my NASs connected to all my cloud services making backups of them live syncing just like desktop clients would. This is for pretty much all of them, MS, Google, Dropbox and even Box!

[chucker]

Originally Posted by drewprops I never appreciated what network area storage entailed until this thread. These boxes aren't just multi-bay hard drives with an Ethernet connection, they're flipping computers, with options for different OSes and unique disk formats. Even media machines.

Yes. Think of it like a Mac mini with a more specialized OS, but hard disk bays inside.

Or like a Time Capsule but with more capabilities.

[Ebby]

Originally Posted by chucker Or like a Time Capsule but with more capabilities.

That's one of the first reasons I got mine. Time Capsule backups were soooo slow. But I also had a RAID5 array of 8 external drives spewing out of my G5 which was on 24/7. Those power bills were horrendous! I took 4 of those drives, slapped them into a Synology unit to replace my Time Capsule, and ran most of my server stuff on like, 25 watts.

[Brad]

Since I've decided to wait and see if there's a 2022 version of the DS920+ coming soon, in the mean time, I'm wiping my old unused Hackintosh/PC and installing TrueNAS CORE on it as an experiment. This old box has a 12-year-old motherboard and CPU (i5-750 2.66 GHz 4-core), but it has an SSD for it's boot drive and 16 GB of RAM. Buuuut it's effectively free, and I have it right now. I still need to buy a couple new drives, but I'd have to buy new drives anyway for the Synology.

I've been reading up on the history and state of TrueNAS, and now I realize I was mistaken in my original post when I described it as a community-run Linux distro. It's actually based on FreeBSD, which arguably makes it closer to macOS's roots since Darwin was also based on FreeBSD, and it's the product of a company called iXsystems that has maintained it for ~11 years (they've been doing BSD stuff since the '90s) as a part of their business for selling hardware and professional support to enterprise customers. The relationship feels a bit like Fedora/CentOS to Red Hat in that regard but with a sharp focus specifically on storage.

My plan is to use this old machine as my NAS and to run a long ethernet cable (I didn't realize we're up to Category 8 now!) around the perimeter of our downstairs room that has our Google Fiber jack into a nearby closet until I have a better wiring solution. I'll need roughly 30ft/10m of cable which should be plenty safe within the stable distance (100m, IIRC).

Since this is going to be an experimental system, I plan to adhere more stringently to the 3-2-1 strategy. The NAS will be my primary live storage, but I'll periodically back up its entire contents to an external drive (I'm nowhere close to filling 14 TB yet), and I'll periodically back up the most critical files to Google Drive like I do today (no Backblaze account yet, but maybe one day).

I'll keep sharing my experiences here since it seems like I'm the only one going down this kind of DIY route. In a few months, assuming there's news about the next generation Synology DS9xx+, I'll decide then whether I'll buy a proper appliance or budget the same amount of money to replace this old PC with modern parts.