Storage Spaces in Windows 10

[suttonbg]

I'm about to upgrade to Windows 10. In current and previous versions of Windows, I have tried to provide a measure of data security with RAID 1 HDDs and Macrium image backups. This results in a fairly complicated system both in terms of hardware (3 disks for each drive: 2 for RAID and 1 for Macrium backup) and discipline, keeping backups current and so on.

I was wondering if anyone had experience with Storage Spaces under Windows 10? I'm asking that in the most general sense, as I am aware, if not totally understanding, all sorts of possbilities with REFS file systems, tiered storage pools etc. If I don't have to be a pioneer, that would be wonderful!

Regards,

Bruce

[Mario]

Many motherboards these days have on-board RAID controllers. I prefer them over a software-based RAID.

Here is why: I tried Windows 10 built-in RAID, but it's not working for me. My PC sometimes reboots (developer stuff, and sometimes caused by buggy graphic card drivers etc.) and every time Windows 10 rebooted (cleanly) from such a hard reboot, it considered the software RAID as potentially broken and started to check/rebuilt it. This takes about 6 hours (!) for a RAID-1 with 2 very fast 2 TB hybrid disks. Not really good.

I switched to the Intel RAID support integrated into the chipset on my motherboard and this worked like a charm for the past six month. Despite many hard reboots / bug checks.
A motherboard with on-board RAID costs maybe 20$ more, but that's worth it. Another 10 $ to get an 'military-grade' board with some more expensive and more reliable components, and you're good to go for 5 years.

I had to restore from backup and rebuilt it once, though.
After installing a MSI BIOS update with a fix for a long-standing memory bug I needed. Unfortunately this BIOS update forced all settings back to default and also broke the RAID. I had to re-create the RAID, loosing all data on both disks. The BIOS offers no option to "keep" one disk and create the other one from it. The free Intel RAID software for Windows has this option. Lesson learned, just too late.

Thanks to my up-to-date Macrium backup from before the BIOS update I could recover the broken RAID in about 1 hour. Macrium is really worth the money. Like IMatch 

[suttonbg]

Dear Mario,

I think we have shared similar experiences with RAID and Windows. I would normally stay with  this sort of system, but one of my motherboards is just old enough that Intel will not provide a Windows 10 version of the RAID software. When I upgraded to Windows 10, the RAID drive was there and running, but with absolutely no way to manage it. I didn't stay in that configuration long enough to see if it would automatically rebuild, but not being able to replace a drive left me feeling very exposed. Back to Win 7!

I understand Intel's strategy for the future is to have a pretty short lifecycle for supporting such software, perhaps to drive us to frequent upgrades of motherboards and chipsets. I wondered if Storage Spaces was a viable, albeit software-based, alternative. The other approach, I guess, is to build a dedicated, Linux-based NAS box to store all data except working files. From some of your previous posts, I gained the impression that you have this sort of system too. Is that relatively straightforward to maintain and are you happy with its reliability?

Regards,

Bruce

[Mario]

I'm not sure about the Intel life cycle, but usually RAID configuration is done by the on-board BIOS. No Windows software needed. The feature set depends on the motherboard of course.
A typical motherboard has a life of 3 to 5 years. Less if you buy a el-cheapo discounter box. Longer if you buy a Dell or HP workstation. These are designed for 24/7 operation, have extended life cycles of 3 to 5 years with support etc. They are expensive, but for a reason.

Maintaining a Linux system requires a different skill set than maintaining Windows. A Linux system must also be updated regularly, software may not work, drivers may not work etc. All this requires fiddling and, in general, Linux is much more 'nerdy' than Windows. I use both, but work on Windows.
Setting up SAMBA on Linux to make the drives available for Windows computers can be challenging. Depends on many factors.

Most professionals and studios I know spend a few grand on a pro-grade NAS, with a specialized Linux variant that is properly maintained by the NAS vendor (e.g., Synology or QNAP), on-board RAID, fast Gigabit Ethernet, USB 3 for external backups etc. Expensive, but reliable. The cheaper SOHO NAS boxes often lack in many areas (speed, reliability, update support, network performance, ...)

We now have now quite fast internet connections in Germany and I backup more and more into 'the cloud'. Locally encrypted before uploading of course  With ~ 1 MB/sec upload I can backup quite a lot in an hour. I use an 'imaging backup' program which keeps the daily backup volume to a few hundred MB. Since I get 5 TB cloud space with my Office subscription, I can upload and keep lots.

[sinus]

We now have now quite fast internet connections in Germany and I backup more and more into 'the cloud'. Locally encrypted before uploading of course  With ~ 1 MB/sec upload I can backup quite a lot in an hour. I use an 'imaging backup' program which keeps the daily backup volume to a few hundred MB. Since I get 5 TB cloud space with my Office subscription, I can upload and keep lots.

Sorry, if I take an older thread, but your answer, Mario, is interesting for me.

I have not worked a lot with cloud, hence I have no experience or knowledge.

If I understand you correctly, you mean:

- locally (means on my computer) first encrypt the datas (like photos) and then upload?!

You mean, with this "security" it is quite safe to upload in a cloud like Microsoft or Google (in Germany maybe Strato or 1&1)?
And you mean, you do upload also backups, photos and so on?

I am afraid, clouds are more and more coming, hence we must live with them and take the best out of it.

Thanks for any informatioins about this ....
I like it to ask here in the IMatch-forum, because I do "know" a lot of people here, hence I do more trust some information here then in another forum. 

[Mario]

I use Macrium Reflect for backup. Macrium can encrypt the backups with strong encryption.
If you now use a local OneDrive or Dropbox drive as the target for the backup, the backed up file remains on your local computer, but is also uploaded into the cloud. In encrypted form!
I use OneDrive because I have an Office subscription (5 users). Each user has 1 TB (!) of storage in the MS cloud - and all that for ~ 70€ per year - including the full Office suite.

I use external storage (2.5" 2 TB WD disks) for the full images taken every day. I move about 20 GB of data every day and that's too much to upload into the cloud (takes to long).
I also do several backups per day of critical and fast changing data, and these go right into the cloud, using Reflect. This gives me multiple backups per day, and I retain these for several weeks.
I also do weekly and monthly backups on separate disks.

I also move data into the cloud. For this, I use a software called BoxCryptor, made by a German company: https://www.boxcryptor.com/de
BoxCryptor basically creates a virtual drive on your system. It shows up in Windows Explorer and you can use it like any other drive. But everything you store on that drive is encrypted. BoxCryptor works directly with most cloud services.

I would never store unencrypted data, not even photos, in a cloud without having a local copy (or multiple) and proper encryption. I don't trust any of the cloud providers.

[sinus]

I use Macrium Reflect for backup. Macrium can encrypt the backups with strong encryption.

Fine. I have this tool also, it seems to be really a good backup-tool.

If you now use a local OneDrive or Dropbox drive as the target for the backup, the backed up file remains on your local computer, but is also uploaded into the cloud. In encrypted form!
I use OneDrive because I have an Office subscription (5 users). Each user has 1 TB (!) of storage in the MS cloud - and all that for ~ 70€ per year - including the full Office suite.

Oh, that sounds great. I have to check this also.

I use external storage (2.5" 2 TB WD disks) for the full images taken every day. I move about 20 GB of data every day and that's too much to upload into the cloud (takes to long).
I also do several backups per day of critical and fast changing data, and these go right into the cloud, using Reflect. This gives me multiple backups per day, and I retain these for several weeks.
I also do weekly and monthly backups on separate disks.

I also move data into the cloud. For this, I use a software called BoxCryptor, made by a German company: https://www.boxcryptor.com/de
BoxCryptor basically creates a virtual drive on your system. It shows up in Windows Explorer and you can use it like any other drive. But everything you store on that drive is encrypted. BoxCryptor works directly with most cloud services.

Ah, I understand. I will also have a look at this German company.

I would never store unencrypted data, not even photos, in a cloud without having a local copy (or multiple) and proper encryption. I don't trust any of the cloud providers.

Hm, yes, I have spotted over the years, that you are very restricting and careful with your data. And this is, I think, a very wise thinking.
What I should do more, although I am also quite careful.

Thanks, Mario, for your answer, very appreciated!

[Mario]

What I should do more, although I am also quite careful.

Note:

Since you run a business, make sure you understand all your local privacy rules. Uploading client data (photos, personal data etc.) into a cloud requires to consider many legal issues. Since most clouds are hosted outside your country, you transfer information to servers not under your countries jurisdiction. If the data you transfer to the servers in foreign countries is customer data, personal information about your customers, business documents etc. you usually are required by law to enter into a special contract with the cloud vendor. It's all very complicated, best to ask a specialized lawyer. If you only make backups of your own stuff, this is not required.

Especially German law is very restrictive in this matter. Good. We all should learn from what Snowden discovered. We consider privacy to be very important.
Germany is the first country (I'm aware of) where Microsoft opened data centers which are run under control of a German company (Deutsche Telekom). This allows businesses in Germany to host their data in the Microsoft cloud running on these servers - this is backed by German privacy and data security laws.

I don't think that many people outside IT are sensitive to these matters. Thea spread their personal data all over the place, sometimes even without proper encryption. As long as it's easy and comfortable and free...<face palm>