Find Duplicates (Hash Preferable) Application in Network Drive

BigLeviathan on 17/05/2018 - 00:49
Last edited 17/05/2018 - 00:50

Hi All,

My NAS is getting full with ISOs and I need to find a way to find duplicates, requirements:

  • Can do the duplicate check based on the hash/MD5, not based on the file name
  • Can search network drive map (my plan is to run Windows virtual machine inside the NAS itself and use shared folder function to reduce network latency)
  • Preferably open source/free

I have tried a couple, but none of them works with network drive (I even used Windows mklink function, similar to ln in Linux to fool the software, but it did not work):

  • CCleaner
  • AusLogic Duplicate finder

Thanks heaps!

Computing

Comments

  • Quantumcat on 17/05/2018 - 06:23

    If you like I can try and write you something later. It won't be fast.

  • MynyMouqe on 17/05/2018 - 08:19

    Have a search for DeDupe software - they generally work from hash, and can work on any mapped drive.

    Don't have any personal recommendations unfortunately, I'm using the dedupe feature built into Storage Spaces on Windows Server.

    • BigLeviathan on 17/05/2018 - 08:42

      DupeGuru you meant?

      • MynyMouqe on 17/05/2018 - 09:50

        That's one of the tools - looks like it would suit your circumstances.

        https://dupeguru.voltaicideas.net/ is the current website of the project.

      • JimmyF on 17/05/2018 - 09:57

        No his using data deduplication on the storage layer, so having 1 or 20 copies of the same file regardless of the filename, means it only takes up the space of one file on the storage.

  • ropdoh on 18/05/2018 - 21:04

    I have found the Duplicate Cleaner Free does the job for me. Can scan my NAS which is mapped as a network drive.
    https://www.digitalvolcano.co.uk/dcdownloads.html

    • BigLeviathan on 18/05/2018 - 22:37

      Thank you for this, will definitely give this a try

      • BigLeviathan on 20/05/2018 - 00:07

        Wow, this application is good! even the free version does hash check and quick-hash too…thank you!

  • BigLeviathan on 18/05/2018 - 23:23

    In the spirit of duplicate cleaner, found this in bitsdujour: Wise Duplicate Finder v1.2.5

Login or Join to leave a comment
Login Join