Fastest Online Backup / Upload?

Jingo · February 03, 2025, 02:44:58 PM

Hi all... I am looking for some advice/help in figuring out how to speed up online backup and your experiences with various services/methods.

I have a robust local backup for all my photos, files and OS. However, the recent CA fires got me thinking that putting all this data someplace other than my local systems should be priority #1. So, I installed Cryptomator and started the process:

1 - Use Cryptomator to encrypt a backup of all my processed Photos.
2 - Upload this encrypted folder to Google Drive via drag/drop

It is working.. but a 750GB folder is going to take 60 hrs to complete... this is with my robust and very fast 1 gigabit upload speeds.

I'm curious if others have a better (and faster) method of using cloud services for encrypted backup storage? I used Google Drive because I already have a business account there so lots of available space.

I probably need to rethink the entire process as well because future images would need to be manually encrypted, uploaded, etc each time I want to back up new items... but I at least wanted to get the bulk of my files safe and secure in a remote backup location.

I appreciate the advice/help and services you are using! Thx - Andy.

axel.hennig · February 03, 2025, 02:57:58 PM

Hi Andy,

I'm using restic for this.

It can deal with Google Drive (maybe rclone is needed)
Encrypted
Within the 2nd upload (and of course all following ones) only differences to last backup are uploaded
Open-source (also rclone)
Compression before uploading

Jingo · February 03, 2025, 04:31:02 PM

Quote from: axel.hennig on February 03, 2025, 02:57:58 PMHi Andy,

I'm using restic for this.

Cool.. thx Axel.. I'll check it out!

Mario · February 03, 2025, 04:34:53 PM

I assume the 750GB is one single file, since you mentioned Cryptomator.
This is of course always problematic, unless you use a cloud storage provider that supports incremental or differential backups. Google e.g. for Google Delta Sync.

Jingo · February 03, 2025, 04:40:40 PM

It looks like Cryptomator encrypts every single file within the folder so they are individual files rather than one large chunk... the Google uploader does show it is uploading 108,000 files.. but man oh man is it slow! Obviously, I need a better method than this..

Mario · February 03, 2025, 05:17:47 PM

Sounds like it's the Google end when you have a fast upload connection.

I do "mass volume" backups via 2TB mobile SSDs and place them at the house of relatives once a week.

medgeek · February 05, 2025, 11:42:20 PM

Quote from: Mario on February 03, 2025, 05:17:47 PMSounds like it's the Google end when you have a fast upload connection.

I do "mass volume" backups via 2TB mobile SSDs and place them at the house of relatives once a week.

Same! Faster than any internet connection, including time in the car...

Mario · February 06, 2025, 12:10:25 AM

We're having movie night or games night or pizza night anyway, so swapping a disk at the same time is just a useful side-effect

Jingo · February 06, 2025, 02:51:09 PM

Quote from: medgeek on February 05, 2025, 11:42:20 PM
Quote from: Mario on February 03, 2025, 05:17:47 PMSounds like it's the Google end when you have a fast upload connection.

I do "mass volume" backups via 2TB mobile SSDs and place them at the house of relatives once a week.
Same! Faster than any internet connection, including time in the car...

I need to get some more friends and my family lives too far away...