Please Help Me Sort out All My Photos

patthecat on 01/05/2021 - 09:47
Last edited 01/05/2021 - 10:38 by 1 other user

I have thousands of digital photos taken over the last 20 years, some are in CD, some on external hard drives and some in Google photos. They are all in such a mess!

I tried to tackle this a few years ago (left unfinished) and possibly have some of CD photos downloaded on the hard drives but not all of them and so there would be double ups of some of the photos. Also I notice some of the digital camera photos have the same numbers but are different photos. I don’t want to accidentally delete any and I would also like to keep the photo resolution as per the original not compressed or anything (hope the last bit makes sense as I am not very technical)

I need a way of downloading/ uploading all photos onto ‘something’ which will sort out any doubles and from there I can organise into date and then back up on a hard drive. Thus giving me a copy in the cloud and on a hard drive of only one of each photo.

Any ideas would be greatly appreciated please.

Computing

Comments

  • Hithere on 01/05/2021 - 10:06
    +6

    sort out any doubles

    I've used Awesome Duplicate Photo Finder in the past, a very basic program but it does what it says, looks for doubles and photos that are very similar and gives you the option of deleting them.

    https://www.duplicate-finder.com/photo.html

    • MS Paint on 01/05/2021 - 10:09

      Does that just look at file name duplicates or actual image duplication?

      Also I notice some of the digital camera photos have the same numbers but are different photos.

      • Hithere on 01/05/2021 - 10:12
        +7

        @MS Paint It brings up the duplicate images and advises what percentage of similarity the photos have, giving the option as to which photo you want to delete.

    • savemore on 01/05/2021 - 10:09
      +1

      Don't suppose you know of programs that allow you to do the same with Microsoft Office files?

    • bazingaa on 01/05/2021 - 10:16
      +1

      thanks, looks like a nice tool

    • bazingaa on 01/05/2021 - 12:07
      +2

      one more question, just found 1000s of duplicates, is there a way to batch delete instead of checking one by one ?

      edit: Found a way but not that straightforward and should be very careful.

      1. export to csv
      2. open in Excel -> text to column
      3. remove rows where "similarity != 100" (later found a setting on program too)
      4. add "=MOD(ROW(),2)" to each row, filter out 0s and delete those rows
      5. concatenate "del /f " infront of each file name Ex: =CONCATENATE("del /f ",B3)
      6. export all commands to a batch file and run :D
      • BLAIL on 01/05/2021 - 14:39
        +1

        Please upload video tutorial - would be much appreciated

        • squidz on 01/05/2021 - 16:44
          +2

          (.bat) is basically a list of executable commands for windows (command prompt)
          imagine step 5 is creating a line for every file he/she wanted to delete.
          Ex =CONCATENATE("del /f ",B3) will be shown as del /f pr0n.jpg
          assuming the file name is pr0n & extension if jpg

          copy pasta the list into csv n save as xxx.bat

          double click n watch the files get terminated..

          Pew Pew

          • BLAIL on 01/05/2021 - 18:07
            +1

            @squidz: well it seemed to work fine so +1

            • bazingaa on 01/05/2021 - 22:18

              @BLAIL: hope it is clear now :) but be careful with the files :D

  • bazingaa on 01/05/2021 - 10:12
    +2

    tried to tackle this a few years ago (left unfinished)
    same here :D

    Better to copy everything to a large hard disk, remove duplicates and backup. Better to use MD5 based duplicate remover as it will compare the whole file not the file names. Just Googled and found this but I haven't used it. May be others and share a better software. If you need to batch process like resize, rename etc, IrfanView is the best free tool.

  • kiitos on 01/05/2021 - 10:30
    +1

    Have the same problem, folders scattered across drives due to space, backups on other internal and external drives, USB sticks for transferring.

    Trying to work out a way to categorise the photos so I can find certain people later, it's very difficult to work out a good way using just for names and folders. Any suggestions there welcomed.

    Uploading them all to Google and letting it identify the people is probably simplest, but would be a huge cost unless I do then all at lower res and soon.

  • patthecat on 01/05/2021 - 10:41

    Thank you for the ideas,I will look into them all.
    Seems like a I am not the only one with the same problem and that there are solutions out there.

    • pharkurnell on 01/05/2021 - 14:48

      Let us know what you find - Im also in that boat

  • Master Bates on 01/05/2021 - 10:52
    +7

    This is how I would tackle the problem:

    1. Collect all CDs, USBS, HDDs
    2. Download folders from Google Photos
    3. Copy everything to a primary HDD
    4. Backup this primary HDD with all of your unsorted photos to a secondary HDD and offsite cloud (as a straight copy of your folders/files rather than in Google Photos)
    5. Start sorting the primary HDD into folders by rough dates in the format yyyymmdd (this will ensure chronological order) and a subject (e.g. birthday, Christmas, etc.)
    6. If you’re confident you haven’t f*****d anything up so far, backup the changes to the secondary HDD
    7. Use dedupe software to (carefully) start deduping, whilst making sure to keep the higher quality files (cloud ones might be a lower resolution)
    8. Once satisfied with your efforts, backup to the secondary HDD and back to the Cloud again
    9. Enjoy

    It looks like a lot of work at the outset, but chunking up in stages will help you get through it.

    • bazingaa on 01/05/2021 - 11:32
      +2

      Backup this primary HDD with all of your unsorted photos to a secondary HDD and offsite cloud (as a straight copy of your folders/files rather than in Google Photos)
      Start sorting the primary HDD into folders by rough dates in the format yyyymmdd (this will ensure chronological order) and a subject (e.g. birthday, Christmas, etc.)
      If you’re confident you haven’t f*****d anything up so far, backup the changes to the secondary HDD

      +1 for this, I am exactly doing the same but still haven't sorted everthing out. I got 2 * 2TB HDD from two brands and copied everything to one and cloned it to other before sorting out the first one. After sorting, backup 1 = HDD, backup 2 = BD-R, backup 3 = resized images on GDrive, backup 4 = clone to other HDD after finishing

  • fantombloo on 01/05/2021 - 11:24
    +1

    I too have tonnes of photos but am slowly going through them and deleting heaps. There's only so many identical poses with something slightly different that I need - I am trying to get the minimal amount of pictures to tell a story. Too many photos just make it hard to manage.

    A friend of mine told me he set a hard photo limit on his travels through many parts of the world to three photos a day. Yes, he may have snapped more but he would cull it to a max 3 at the end of the day.

    A good idea is to cull and then collate to photo books - digital or, even better, printed. If it doesn't belong in a photo book then consider whether it's worth keeping at all. A photo book on the coffee table or entertainment unit is much more likely to be picked up and viewed than millions of DCM53849268.jpg archived on a hard drive.

  • thatonethere on 01/05/2021 - 13:13
    +1

    I found good success with the paid version of Google Photos.

    Scan everything in and upload it. Keep a copy on a good nas as well just in case. Google Photos can dedupe as well reasonably well but only exact copies. Not near copies.

  • badg3rz on 02/05/2021 - 08:19
    +1

    Buy storage for Google (I think it's $12.49m for 2TB) and upload it all onto Google photos, make sure you have changed it to "Original Quality"(yes Google sorts out duplicate photos but if there are two photos with the same file number it won't overwrite it as I've tried it) then download them all from Google (as in export your data out).

    One, two, three easy as, I'm doing the same, you just drop folders onto Google Photos, but just do one folder at a time to make sure you don't have connection problems.

  • jetblack on 02/05/2021 - 08:40
    +5

    I attempt this very thing about once a year, and it goes something like this:

    • work hard for a couple hours figuring out where I have duplicates and why
    • get distracted just looking at pictures and reminiscing
    • get frustrated at how difficult the whole process is and tell myself I'll take a break and come back to it in the afternoon
    • forget about it for another year
    • katz on 02/05/2021 - 08:42
      +2

      This

  • jrvb42 on 02/05/2021 - 09:29
    +1

    dupeguru will find and delete duplicate files of all types, and lets you select all duplicates and delete in one action.
    You can specify which is the master/reference directory so you can keep originals in one place.

  • gorgrond on 03/05/2021 - 10:03

    Synology NAS does that.

    Buy a nice one, add the discs (quite cheap now, deals every month in OzBargain). Moments (the official photos app) allows you to upload the pictures and sorts by date, and it has a dedupe feature built-in (will ask you if they are the same). Use RAID1 with large discs for dumb easy disc failure safety.

    Then, configure Synology to backup to a cloud service, and you're done.

    • badg3rz on 15/05/2021 - 12:52

      Also Netgear NAS

Login or Join to leave a comment
Login Join