How do you untar multiple files into multiple subdirectory based on filename

Question

I have a single tar file containing about 19 million files (no folders)

0000107b869682826003b04a40e6394.txt
00029237482s8923789423ud8923892.txt
2c002y8378723887292377a79237649.txt
f598238209237408238742308374038.txt

how do I untar all the files such that they appear in subdirectories named after the first four chars of the files. So for the example above, it would create 0000, 2c02, and f599 directories and each would have the following files.

0000\0000107b869682826003b04a40e6394.txt
0002\00029237482s8923789423ud8923892.txt
2c02\2c002y8378723887292377a79237649.txt
f598\f598238209237408238742308374038.txt

I've already tried creating a script that goes through the files in the tar file, creates a directory and extracts that file from the tar and puts it in the directory. This works for small number of files, but when the tar has millions, extracting takes a really long time.

Answer 1

With GNU tar and its s command with syntax from sed. I switched from s/// to s|||.

tar -xvf file.tar --transform 's|\(....\).*|\1/&|' --show-transformed-names

Answer 2

I created a test tarball, no directories:

pg@TREX:~/test$ tar -tvf test.tar | rev | cut -c -8 | rev
0001.txt
0002.txt
0003.txt
0004.txt
0005.txt
0011.txt
0012.txt
0013.txt
0014.txt
0015.txt
0021.txt
0022.txt
0023.txt
0024.txt
0025.txt

I run this script (tartest.sh):

#!/bin/bash

tar -xf tarfile.tar
i=$(ls *.txt | cut -c -3 | sort | uniq) 
echo "$i" >> directory_list 
mkdir $i 
while read line; do mv $line*.txt $line/; done < directory_list

Result:

pg@TREX:~/test$ tree
.
├── 000
│   ├── 0001.txt
│   ├── 0002.txt
│   ├── 0003.txt
│   ├── 0004.txt
│   └── 0005.txt
├── 001
│   ├── 0011.txt
│   ├── 0012.txt
│   ├── 0013.txt
│   ├── 0014.txt
│   └── 0015.txt
├── 002
│   ├── 0021.txt
│   ├── 0022.txt
│   ├── 0023.txt
│   ├── 0024.txt
│   └── 0025.txt
├── directory_list
├── tartest.sh
└── test.tar

I'm sure this'll take a bit of time with 19mil files, and I'm sure more elegant solutions exist... but seems to do what you asked :-)