Looking at storage options for some of my wife's photographic files i.e. an offline archive.
I had someone recommend I use something called an M-Disk - old school optical drive, but it seems setup cost might be prohibitive, and max storage size per disk is 100Gb?
Wondering if I should use a good old USB hard drive, USB flash drive, or an SD Card?
Thoughts?
How much do you need to store? Aprox 1Tb in total
Whats your budget? 300-400
What do you have them stored on now? NAS
NAS is not a backup. If the NAS dies don’t assume you can just rip the drives and put them in another device. You’ve now got 2 points of failure instead of one. Same goes for any kind of raid setup.
@LFO: RAID is NOT a backup! There's so many things that can take out all the disks in an array, regardless of how many disks you are using as parity etc.
Serious backup means following the 3-2-1 rule. That is:
@photonbuddy: I think op means for home data storage, nothing too expensive to do some hardcore like that.
I think op means for home data storage, nothing too expensive to do some hardcore like that.
OP is talking about ~1tb of data. There's no need for it to be expensive at all.
The $300-$400 budget OP has listed is more than enough to meet the 3-2-1 rule for such a reasonable amount of data.
@photonbuddy: What about unraid? Ive have numerous components die in mine over the years. (Motherboard, parity drive, cache drive etc.)
Never lost a byte of data
Never lost a byte of data
A backup does more than protect against hardware failure. For example if you're hit by ransomware, you'll just have a very reliable encrypted copy of your files.
If there's a lighting strike, power surge, fire, or burglary, you could lose all your drives in one go.
That is why RAID is not a backup. One copy should be kept offsite.
@eug: That's why I have a docker that backs up incrementally to the cloud daily (only the important stuff, not the 100tb of Linux isos).
Also have dual parity.
I think if you are that unlucky that you somehow lose the physical copy, cloud copy etc all in one go you'd just have to laugh at your luck and move on.
I feel very confident backing up anything to my unraid server, after all it isn't a traditional raid setup.
Google Photos is probably the safest backup you can have. Can't see that going anywhere.
That's why I have a docker that backs up incrementally to the cloud daily
You only mentioned multiple hardware failures earlier.
@eug: Correct I had multiple hardware failures recently (power surge).
The beauty of unraid over a traditional raid setup (Like a consumer NAS) is that you aren't reliant on using the same hardware in the case of companies like Synology, QNAP etc.
It could worthwhile/comforting to some people to have another physical backup stored off site but I think one physical and one cloud copy is sufficient in most cases.
Correct I had multiple hardware failures recently (power surge).
The point being discussed is that RAID is not backup.
It isn't - that's why you also back up to the cloud.
It could worthwhile/comforting to some people to have another physical backup stored off site but I think one physical and one cloud copy is sufficient in most cases.
Offsite simply means not in the same building.
That includes cloud storage as it's well… offsite. :)
@creesy: How do you know if you've never lost a single byte- are you doing regular CRC hash comparisons on entire filesystems to ensure that you're not getting the occasional flipped bit?
I've been archiving photos for a couple of decades on RAID systems and because jpeg files have no redundancy inside them, a single flipped bit will corrupt the entire image in an instantly recognisable way. This has happened numberous times that I've seen.
There are not many file systems that protect against this. The only one that occurs to me is zfs.
Your comment about dual parity on unraid… That just provides additional uptime protection. It's nothing to do with backed up data.
@rumblytangara: Last year unraid added ZFS filesystems as a feature and since then everything important except media (Linux isos) has been in ZFS pools.
I can assure you it's very safe my unraid server has been around in different configurations for 15 years now and I've never lost a file. I'd say for consumer file storage that's pretty good.
Of course dual parity isn't an extra protection it's just so that if there is a problem with the parity I don't have to wait a fortnite to access my files while it rebuilds.
We can talk about doomsday scenarios all we want, if it happens it happens. I've done nearly everything I can to protect anything important.
If OP is really worried then spend the $500 dollars for an m-disc setup make multiple copies pay for storage somewhere secure etc etc.
@creesy: Time for me to look at unraid again, then. I can't remember the zfs settings to get protection against bitrot either.
But same here, I don't "lose files" as such. But I have noticed flipped bits in images.
RAID is NOT a backup!
Someone literally setup a website about this: https://www.raidisnotabackup.com
@PhilosophicalTurtle: You can use backup software. e.g. CrashPlan's cloud backup software can handle versioning and deduplication. It will back up your data to the cloud as well as local storage, e.g. a mechanical hard drive and SSD. It can backup to a NAS too although that requires some custom config.
There are also other programs like ArqBackup that handles versioning and can back up to any destination as well.
@eug: I do use Arq and backup to Google Drive using CyberDuck. Pretty happy with the service myself.
@LFO: RAID provides for the availability. A NAS and RAID provide high availability not DR. for instance a corrupt driver, failed controller, malware, corrupt data copy can all completely wipe out a NAS and RAID will do SFA to help you. seen more than a few enterprises and government departments that have made the mistake of thinking their NAS/SAN are backup/DR, it does not end well.
I got a Samsung 1TB portable SSD for about $90 from Officeworks - it's quite slim and tiny and much faster than hard disk for backing up many GB - like seconds. Looks like $151 from Amazon - I'd look for a sale price - https://www.amazon.com.au/Portable-External-Reliable-Student…
I think the consensus is blu-ray is best of consumer grade optical, dvd/cd-rom inks fade.
SD cards and flash drives lose charge over time, and need to be accessed to restore state. Not sure if SSDs have this issue.
Spinning hard drives retain files a long time, but can suffer from mechanical issues like lubricants drying out.
My conclusion is the cheapest way to store 1Tb for 10 years plus with no ongoing work would be two portable hard drives of different brands, the redundancy aiming to reduce the chance of a manufacturing fail.
SSD can fail too within years, so offline storage and cloud storage are important for important files.
Backblaze company has shared valuable info about their HDD and SSD failure rate/history from their data center ;-)
Agree with cloud also being in the mix.
Offline cold storage is great, but it is human nature to leave it too long between backups and not to test restores.
Online can be automated.
I was wondering about that. How much power does an SD card need? You might be able to rig one up with a battery
There are also very different qualities of blu-ray media. I have lost photos that I thought were safely archived when I stupidly used cheaper media :(
https://www.tomshardware.com/news/pioneer-new-blu-ray-record…
Archive grade DVD media, note the price …
https://www.amazon.com.au/Verbatim-96320-DVD-R-ULTRALIFE-Arc…
Bluray archiving tech
https://www.disc-group.com/technology/blu-ray-technology/
M-Disk.. another archive format. Burners that support it seem more available (I have one)
https://en.wikipedia.org/wiki/M-DISC
Mdisk media
https://www.amazon.com.au/Verbatim-Japan-VBR520YMDP5V1-Long-…
Thanks for that, I learn smth new today. The Pioneer BDR-WX01DM is very pricey though
yeah - remember when Optical Disk CDs were gonna last forever ?
or until they got scratched, when they became unreadable
and consigned to the dustbin of history
I would like to see an optical drive that can last 100 years.
Consumer-grade Blurays are being discontinued just FYI, so stock up on these if you need them
If needs to be accessible, then RAID5/6 with medium of choice (HDD are cheap, SSDs are fast) …
If can be stored, multiple copies …
(for example, 1 x HDD online, 1 x HDD offline, 1 x HDD at work, 1 x BD/CD/DVD written per month/year depending on updates) …
I still have text and word documents from the 1980s that were transferred from HDD to new HDD over the years!
The Data CDs that were written on the first CD writers are unreadable (they were not my only copies, just backups) …
Floppy disks
Too bendy. How about a 12" 10MB glass HDD from just before storage became bendy?
OP still have NAS, hopefully also have cloud storage.
Good backup method is both offline and online backup. If NAS burned down along with house burn or natural disaster, cloud storage is still available …
I have cheap second hand OG Pixel XL that can backup photos and videos at original quality unlimited ;-) Another way is using custom Pixel experience ROM for certain phones that can also have unlimited original quality backup just like the OG Pixel.
I'm no data storage expert, but it seems to be putting it on a SSD then plugging it in once every 6-12 months to maintain charge should last an extremely long time. The less used a SSD is the better it tends to retain data and it'll be significantly more reliable than flash drives/SD cards at a reasonable price per GB.
That said, m-disks are about $10-20 a pop for 100GB and require no real effort to maintain.
On top of that, you should have NAS, Cloud (Office 365 family account will get you 6TB for $100 a year) and the local storage itself. That's more than enough.
If only photos and videos, unlimited Google Photos backup at original quality from the OG Pixel phone or certain phones with Pixel experience custom rom is much cheaper.
I don't want to pay $100 every year for photos and videos online backup. Everyone has different need of how many GB or TB cloud backup.
my iPhone is currently showing iCloud+ backup for 2TB at $15pm or $160pa
I pay Google storage about A$28pa for 100GB online storage
but if I kick the bucket they would probably disappear - so not great for beneficiary family memories after that event
Better buy lifetime cloud storage rather than paying every year. Much cheaper and other certain cloud storage services can be safer than Google cloud storage.
@neoleo: Is lifetime cloud storage a sustainable business model? What if you paid for it, upload a few TB, then the business realises it's not sustainable then just shuts down and deletes everything?
Google storage is pretty secure, but if you want to go the extra mile you could switch to Google's Advanced Protection Program. Stronger security means less convenience though.
You can also encrypt everything before storing it on your cloud storage provider with a program like Cryptomator.
@eug: No of course not, 'lifetime' is normally lifetime of the company and reputable providers don't really offer lifetime plans. It would only be sustainable if the offer was really expensive.
@ldd-mn: storage on the internet is forever, or three years, whichever comes first - unless you stop paying, or die - in which case it's inaccessible anywaze … ?
https://wasabi.com/ is what I use and it is great.
Only use it for files backup, you would have to download stuff to view it.
Good solution for also backing up a NAS as it is an S3 solution. It can mirror your local NAS.
Can use a FTP connection as well for making it easy to upload/download files quick without a web panel.
I think it is about same same. I like Wasabi because it is a like for like with S3 comptability. Seems like Backblaze do now offer this.
Yes, they've offered it for years
Wasabi has a Sydney presence :)
chatgpt told me consumer-grade SSDs are rated for a 1 year data retention period
What’s your wife into?
Photography, I'm guessing.
Could be taken on holiday. A nudge is as good as a wink to a blind bat.
Dropbox with smart sync. Files live on the cloud but the client will show thumbnails in your Finder/Explorer window. When opening, it will download to disk. So it feels like local storage but no hassles
Dropbox is $184/year for 2TB, might as well go with Google Drive at $125/year.
Agreed, it is expensive but I suggested it for smart sync. Even cheaper is S3 Glacier or similar if access is going to be infrequent.
Agreed, it is expensive but I suggested it for smart sync.
How does it differ to Google Drive's "online only" and OneDrive's Files On Demand?
I use iDrive, normally US$99.50/year for 5TB but they offer the first year at a discount.
- multiple device backup, (computers, phones, tablets, servers, etc), I've used it for backing up VPS's as well as home/mobile stuff
- default encrypted, can specify your own encryption key but if you forget it your data is lost
- can access backup data via web account or device app, (e.g. access computer backup from phone app)
- versioning (last 30 versions)
- Cloud Drive, separate storage that is synced between devices, e.g. a 5TB iDrive backup account comes with a 5TB Cloud Drive - it doesn't count towards your backup storage
- you can seed your backup, they'll send out a HDD, you fill it up and send it back, (you pay postage)
- etc, etc
That's a backup service which has different features to Google Drive, e.g. you don't get Google Photos' useful photo search, the ability to share albums with others, automatic phone photo backup, Gmail storage, etc. It does do pure backup well though.
Other options are CrashPlan and BackBlaze. CrashPlan provides unlimited storage for US$88/year for 2 devices, and the app also allows you to do a file-level backup to a local drive with versioning.
@eug: Fwiw, I don't use Google Photos, I run an Immich server which gives me what I need, [search, share albums (which I don't anyway), phone photo backup], and it gets backed up to iDrive.
Plus it doesn't eat into my Google storage limit.
BackBlaze - used it, more trouble than it was worth, (I don't want my whole computer backed up, I want important data backed up without wasting time and bandwidth on unimportant things like the OS/programs)
CrashPlan - limited number of devices, (currently have 8 on iDrive, their program also allows file level local backup with encryption on their free tier)
I run an Immich server which gives me what I need
That's great. Judging from OP's question though, I probably wouldn't recommend that route for them.
BackBlaze - used it, more trouble than it was worth, (I don't want my whole computer backed up, I want important data backed up without wasting time and bandwidth on unimportant things like the OS/programs)
BackBlaze does not back up your OS and programs.
CrashPlan - limited number of devices, (currently have 8 on iDrive, their program also allows file level local backup with encryption on their free tier)
It doesn't sound like OP is wanting to back up multiple devices, he just mentioned backing up his wife's photos.
Thoughts?
Any medium will decay over time, so you will need multiple copies of the data on different types of mediums to protect from this.
If it was me, I would be going a standard HDD for one copy, and a cloud copy. Google drive does 2TB for $125/year.
I have 20 years of photos and storage backed up on Google Drive, a separate hard drive on my computer and also a third copy on my wife's computer. The reason I have it on Google Drive is if my house burns down (yes it does happen) and I only have physical copies, everything is lost. I am also able to access them when travelling if need be. The small amount of money spent on cloud storage will be worth it compared to explaining to your wife that it is all gone and not coming back..
I would get an enterprise disk drive, i still have old 2 tb western digital re3s working close to 20 years later used as cold storage.
Costly drives back then when storage was about $1 per GB.
The often overlooked part of these long term storage efforts is how the data would be retrieved when required. What events are you insuring against? Would the technology required to read the data be available in 5 years or 30 years? Are you archiving that technology with the data?
Remember to format shift people! Keep up with the times!
I am happy to bet any amount a JPG on an exfat or NTFS disk will be easily readable in 30 years.
A JPG on a fat32 partition created 30 years ago is easily readable now, perhaps needing a $20 adapter between pata and sata.
I'm very confident anything that plugs in via USB will be even easier.
Define "easily readable" and by whom? Are your grandkids going to be able to do that with their extensive knowledge of antiquated technology?
I use a couple of enterprise-grade HDDs on-site and backup to Backblaze offsite.
The problem with cloud storage is that it leaves your files wide open to hackers and who knows what they will do your data and photos once they gained access to them.
You don't have to store them as-is, you can use a program like the free opensource Cryptomator which seamlessly encrypts your data before storing it on your cloud storage provider. It just appears as a normal drive on your computer.
More chance of a HDD being stolen from your house.
Iron Wolf Pro hard drive is what i would recommend. Don't go SSD for long term storage.
For myself I have a NAS that gets backed up once a week to a portable hard disk, and to Google Drive, Microsoft Onedrive, and a second old NAS.
Overkill I know but take your pick of ANY combination of at least two backup methods (suggest one physical and one cloud) in addition to the main NAS.
Too many sad stories from my customers when their HDD has to go to a lab to get get some data retrieved starting at $800 and only going up from there. Photos are precious and moments in time you cant get back.
Blueray discs would be the cheapest, as the disc them selves are reasonable priced for 25GB each and $100 for a writer.
NVMe
I use 2 external HDDs for my backups.
The first one sits near my PC and I sync each month using robocopy.
The second one is kept at my parents place.
Every few months I swap them around.
PCloud
I recently pulled out some CDRs I burned in 1997 on gold coloured media. They all work 100% fine.
Cloud storage is just storing your files on someone else's computer. They can choose to delete them.
I suggest multiple backup media, in multiple locations. Also avoid flash media. It has a known limited lifespan.
Backblaze B2.
They're also in the Bandwidth Alliance, if that helps, e.g.:
https://www.reddit.com/r/backblaze/comments/1d5ddug/question…
LTO tape stored in Iron Mountain
The NAS is good for your local storage. However If you're looking to keep storing files locally there's always the chance long term the hardware will fail. So would always recommend a sync to cloud based storage.
From your NAS you can sync files or take backups to Wasabi it's very affordable. https://wasabi.com/pricing
If it's just your happy snaps and family photos, brand name Blu-ray disc will last 20-30 years. If it's something critically important you need to just bite the bullet and pay for mdisc (it's not even that expensive, they're $20-30 each) or tape. My experience with mdisc is that the drives can be temperamental, not all writers can write BDXL even if it's listed on the specs. So you may only get 25gb out of each disc.
I would use a cloud backup option, and offline HDD backup too, and ideally a 3rd option.
We have a NAS as home and i tell my wife to backup to cloud too.
It is simple really, Cloud + an Offline copy. well and truly within your budget. Just remember like has been mentioned NAS is NOT a backup solution, way too many points of failure, it is great for a High availability access to the Data but you should have a versioned cloud backup and an offline copy.
Onedrive, backblaze, google or whatever. They all can do the job cheaply combined with a backup.
Personally I use NAS locally (for HA), One Drive for versioned cloud backup, seperate USB drives for offsite backup.
The most reliable media you can find.
I've gone with WD Elements hard drives after a bad run of Seagate 3/4 TB drives a few years ago.
At least 3 copies and at least one of those off site.
Reason: To minimize risk of being hosed by ransomware, only connect 2nd and 3rd copies to your computers when updating them.
Never update until you're extremely confident you've sorted it if you suspect corruption or malware
Reason for off site: Protect against fire/flood/theft etc affecting your home.
Even better if you can get media from 3 different brands.
Reason: Issues that don't show up long term less likely to corrupt all 3 copies.
Refresh your copies every 3 years.
Reason: Moving parts and electronics don't last forever. Spinning disks have moving parts and some are filled with helium that will eventually leak.
I don't trust cloud storage.
Reason 1: It's under someone else's control. Their agenda is not your agenda. They can change terms of service, go bankrupt, have a problem with their data center etc.
Reason 2: If your collection is large consumer upload speeds in Aus are a joke.
you either choose local or cloud, these days nothing is guaranteed going online, you'll just have to let go for the conveniences, unless you lock your data in a vault in the bank, there're just too many things can go wrong locally as well, speaking as guy who experienced a break-in and death the hard drive due to surges or blackout before, so things can happen
You absolutely can have a mix of cloud and local. You don't have to pick just one. I'd only ever use cloud as an additional form of storage, not a primary. First it takes too long to do a full backup to cloud, and then at any time your backup provider can change terms or lose your data. Never ever let anyone have exclusive control of anything you care about if you have any other option.
There are ways to mitigate against local disasters. Multiple sites. How likely are you to have a breakin/fire/flood at home AND at work, or a trusted relative's house.
some cloud service, I went through the whole backup process once before by asking my provider send a hard disk over from the states (where they are located) and sent it back afterwards without being charged anything, the whole process was smooth, but knowing that with a slight chance your files can be intercepted in the middle somewhere (I don't remember if the disk was encrypted and I was given a password), so there's this risk. I wouldn't trust NAS either, a single break-in can compromise any setup you have locally.
2x backups to Bluray, one kept offsite and another onsite, and 1x backup to the cloud such as an Azure blob with zone redundant storage. The cloud backup will cost you around AUD $7 per month for 1TB with zone redundancy.
Best not to use flash drives or SSD's, they will work for a while but down the track they will eventually fail or the data will corrupt over long periods of time.
Backblaze does full PC backups for 99 USD a year, Google Drive is 125 AUD a year for 2TB.
Either is heaps for you
an offline archive
External USB hard drive.
For disaster-proofing, buy two and ensure that they are never both in your house at once. Or use a cloud based backup in addition to local storage.
Hardisk drives have been around for a good amount of time to prove their storage longevity.
It is also a tech that drive down the cost of cloud bulk storage.
I prefer it as it has better memory retention compared to Nand flash, very cheap, widely common to replace, reasonably fast(compared to tapes), and plug and play.
No special software required, no expensive drives to read your storage medium, extremely compatible with most PC/Mac.
Some people may call this a spinning rust, but powered up once a week to do a weekly backup and unplugged most time may extend it's service life way beyond a lifetime.
Am I the only one who read pornographic?
Looking at storage options for some of my wife's pornographic files i.e. an offline archive.
I just use 3x seperate Nvme drives and back up photos to all three once a month or so. All manual but Nvme drives over Thunderbolt ports are Super fast so even terrabites of data take only minutes so easy and quick enough to do. I don't believe in any online backup (I'm old school) so keep all with me (kept in different locations in case something happens - burglary, house on fire etc etc)
It's a shame best free cloud option is 15gb from Google drive.
There are some cloud storage of 1tb you can buy from appsumo. However, risk of those small companies going under.
It's a shame the free tier has been 15gb for such a long time. No one has increased the free tier usage
I know not everyone use up their 15GB but for every 1GB they need 2GB to store your data and a backup copy for restore sometimes even more for revisions.
I have about the same volume in photos, and more in videos.
Currently have sort of a 3-2-1 setup
Maybe overkill but paying to have peace of mind
I don't know much, but what about a Microsoft Family Plan with 6 x 1TB OneDrives + rsync with your NAS?
Does she need to edit the files? If not, just fill an HDD oldest to newest and stash it somewhere safe periodically.
How much do you need to store?
Whats your budget?
What do you have them stored on now?