dragonfire:

1. reads all the static libraries in one pass
2. deduplicates the object files inside them based on their size and naming (Rust has its own, unique naming convention for object files -- pretty useful!)
3. copies the duplicate objects into a new static library (usually called gstrsworkspace as its primary use is for the GStreamer ecosystem)
4. removes the duplicates from the rest of the libraries
5. updates the symbol table in each of the libraries with the bundled LLVM tools.

Related contents:

• Linking and shrinking Rust static libraries: a tale of fire @ Centricular.
• Shrinking while linking @ Tweag.

turn almost any device into a file server with resumable uploads/downloads using any web browser.

Portable file server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++ all in one file, no deps.

Related contents:

• Copyparty - Le serveur de fichiers qui tient dans un seul fichier Python @ Korben :fr:.
• Host Your Own File Server with Copyparty and Docker @ Noted.
• Ep 63: Running Linux on an IPad @ Linux Matters.

An extension toolkit for Immich enabling advanced management capabilities through AI-powered similarity detection

Tools for deduping file systems.

Duperemove is a simple tool for finding duplicated extents and submitting them for deduplication. When given a list of files it will hash their contents on a block by block basis and compare those hashes to each other, finding and categorizing extents that match each other. When given the -d option, duperemove will submit those extents for deduplication using the Linux kernel FIDEDUPERANGE ioctl.

• Duperemove @ GitHub.
• Episode 588: Clearing out the Tumbleweeds @ Linux Unplugged.

Best-Effort Extent-Same, a btrfs dedupe agent.

bees is a block-oriented userspace deduplication agent designed for large btrfs filesystems. It is an offline dedupe combined with an incremental data scan capability to minimize time data spends on disk from write to dedupe.

• Episode 588: Clearing out the Tumbleweeds @ Linux Unplugged.

Extremely fast tool to remove duplicates and other lint from your filesystem.

rmlint finds space waste and other broken things on your filesystem and offers to remove it.

• rmlint @ GitHub.
• Rmlint – Pour optimiser votre espace disque sous Linux @ Korben :fr:.

Deduplicating archiver with compression and authenticated encryption. BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption. The main goal of Borg is to provide an efficient and secure way to back up data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to targets not fully trusted.

Deduplicating Archiver. BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption. The main goal of Borg is to provide an efficient and secure way to backup data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to not fully trusted targets.

Borg @ GitHub.