Best way to compress a directory tree containing archives and files selectively extracted from them? [closed]

Question

I have a file structure that looks something like this:

.
└── base/
    ├── archives/
    │   ├── foo.zip
    │   ├── bar.rar
    │   └── baz.7z
    ├── unzipped-1/
    │   └── complex subdir tree/
    │       ├── selected files from archives
    │       └── files not from archives
    ├── unzipped-2/
    │   └── same deal as unzipped-1
    └── etc.

Over time, contents of archives/ will change, and which files I choose to extract from those archives will also change. In unzipped/ directories, about 95% of the files come from an archive. The rest were added by other means.

I want to create compressed backups of the whole directory tree, starting from base/. I want the file size of the compressed backup to be as small as possible. Time and memory requirements are not an issue for me.

Logically, it makes sense to use an archiving tool which is smart enough to detect that some files came from archives that are also being included in the current backup. There's no need to compress both the source archive and the extracted file: instead, this tool just needs to record where a file from the source archive should be extracted.

Clarification: during extraction, it will be necessary to not only extract the intact archives back into archives/, but it will also be necessary to extract selected files from each archive into unzipped/.

How can I do this?

Answer 1

You would need to create a list of the files from the zipped archives, and use that as an exclusion list when zipping the base folder.

Assuming 7Zip as the tool, and assuming that file-names do not repeat in different folders/archives but with different contents:

• Many posts for listing bare file names from an archive can be found, for example 7z list only filenames

• For exclusion, use 7Zip with the -x @listfile parameter. For example, see the post 7zip: How to exclude files (not file types) using an exclude list file?