38

I am trying to semi-securely erase a bunch of hard drives. The following is working at 20-50Mb/s

dd if=/dev/zero of=/dev/sda

But 

dd if=/dev/random of=/dev/sda

seems not to work. Also when I type 

dd if=/dev/random of=stdout

It only gives me a few bytes regardless of what I pass it for bs= and count=

Am I using /dev/random wrong? What other info should I look for to move this troubleshooting forward? Is there some other way to do this with a script or something like 

makeMyLifeEasy | dd if=stdin of=/dev/sda

Or something like that...

Improve this question
2

6 Answers 6

Reset to default
53

Both /dev/random and /dev/urandom use an "entropy pool". When the pool runs out, /dev/random waits for it to refill, which requires monitoring system behavior (keyboard input, mouse movement, etc.), whereas /dev/urandom will continue to give you pseudo-random data. /dev/random is theoretically higher quality, but /dev/urandom is almost certainly good enough for your purposes. (But even /dev/urandom is likely be slower than some other methods. A faster, but lower quality, generator is probably good enough for erasing hard drives. It's not clear that an attacker would gain any advantage from knowing the sequence that's going to be generated, or that random numbers are better for this purpose than a sequence like 0, 1, 2, 3, 4, ....)

Quoting the random(4) man page:

If you are unsure about whether you should use /dev/random or /dev/urandom, then probably you want to use the latter. As a general rule, /dev/urandom should be used for everything except long-lived GPG/SSL/SSH keys.

UPDATE : The random(4) man page has been updated since I wrote that. It now says:

The /dev/random interface is considered a legacy interface, and /dev/urandom is preferred and sufficient in all use cases, with the exception of applications which require randomness during early boot time; for these applications, getrandom(2) must be used instead, because it will block until the entropy pool is initialized.

See also "Myths about /dev/urandom" by Thomas Hühn.

But /dev/urandom, even though it won't block, is likely to be too slow if you want to generate huge amounts of data. Take some measurements on your system before trying it.

EDIT : The following is a digression on "true" random numbers vs. pseudo-random numbers. If all you're interested in is a practical answer to the question, you can stop reading now.

I've seem claims (including in other answers here) that /dev/random implements a "true" random number generator, as opposed to a pseudo-random number generator (PRNG). For example, the Wikipedia article makes such a claim. I don't believe that's correct. There's some discussion of it here which refers to hardware random number generators, but I see no evidence that /dev/random typically uses such a device, or that typical computers even have such a device. They differ from PRNGs like the C rand() function in that they're not deterministic, since they harvest entropy from sources that are practically unpredictable.

I'd say there are three classes of "random" number generators:

  1. Deterministic PRNGs, like C's rand() function, which use an algorithm to generate repeatable sequences that have (more or less) the statistical properties of a truly random sequence. These can be good enough for games (given a good way of seeding them), and are necessary for applications that require repeatability, but they're not suitable for cryptography.

  2. Generators like /dev/random and /dev/urandom that harvest entropy from some practically unpredictable source like I/O activity (this is why pounding on the keyboard or moving the mouse can cause /dev/random to produce more data). It's not clear (to me) whether these satisfy the definition of a PRNG (I've seen definitions that say a PRNG is deterministic), but neither are they true random number generators.

  3. Hardware random number generators that are physically unpredictable even with complete knowledge of their initial state, and that additionally use mathematical techniques to ensure the right statistical properties.

Improve this answer
3
  • 2
    Even /dev/urandom is kind of slowish if you have to fill in huge amounts of disk space (like whole partitions, before creating encrypted filesystems on them). This should be taken as small addition to the excellent answer and detailed explanation.
    – vtest
    Commented Nov 24, 2011 at 5:55
  • Since you cannot compute/derive/create/... more than one bit of entropy from one random bit, anything which generates/outputs more 'random' bits than it received as input is by definition pseudo-random at best. Hence, /dev/urandom clearly is pseudo-random. /dev/random differs in that it tries to make a conservative estimate of its input's entropy and does not output more entropy than it (thinks it) actually can. This is independent of the presence of a dedicated TRNG device, because true entropy can also be obtained from independent events of any kind, like keyboard or network IO vs. time.
    – JimmyB
    Commented Feb 12, 2016 at 12:35
  • Nowadays, many computers will have hardware random number generators (e.g. modules or on-chip generators) that contribute to the "/dev/random" entropy pool. Try running rngd --list to list the entropy sources used, it may list a TPM or something like "Intel RDRAND Instruction RNG".
    – Kevin
    Commented Jul 22, 2020 at 16:52
14

/dev/random is a source of true entropy, truly random bytes. As such, it needs a source of randomness. You can 'use up' the randomness by reading from it. It will give you all the randomness that it has, then block until it gets more. You're probably just sitting there waiting, and the machine gets very little new randomness, and it just waits.

/dev/random for truly random crypto, high quality randomness. As such, it is an overkill for a disk drive overwrite. Writing from /dev/zero a few times is fine. Or you can write from /dev/urandom, which won't block and gives pseudo-random numbers when it runs out of true randomness.

Improve this answer
2
  • 11
    No, /dev/random doesn't generate "truly random bytes". It generates higher-quality pseudo-random bytes than /dev/urandom does.
    – Keith Thompson
    Commented Nov 23, 2011 at 9:24
  • Both paragraphs start with /dev/random, should it be /dev/urandom?
    – A.L - say no to AI
    Commented Dec 2, 2022 at 18:13
8

In Linux /dev/random is a special file which serves high quality pseudo random numbers. This implementation collects entropy from events originating from the keyboard, mouse, disk and system interrupts.(refer this document) So when there are no such events, the entropy pool is empty, reads from /dev/random will block until additional environmental noise is gathered. This explains your problem. To fill the entropy pool you can press keys on keyboard.

On the other note a truly random number generator uses Hardware random number generator which generates random numbers from physical processes.These processes include microscopic phenomena that generate a low-level, statistically random "noise" signal, such as thermal noise or the photoelectric effect or other physical phenomena. These processes are, in theory, completely unpredictable, and the theory's assertions of unpredictability are subject to experimental test.

A hardware random number generator typically consists of a transducer to convert some aspect of the physical phenomena to an electrical signal, an amplifier and other electronic circuitry to increase the amplitude of the random fluctuations to a macroscopic level, and some type of analog to digital converter to convert the output into a digital number, often a simple binary digit 0 or 1. By repeatedly sampling the randomly varying signal, a series of random numbers is obtained.

The Hardware Random Number Generator gathers environmental noise from device drivers and other sources into an entropy pool. From this entropy pool random numbers are created. When read, the /dev/random device will only return random bytes within the estimated number of bits of noise in the entropy pool. This explains your problem.

Some implementations of Hardware RNG are explained in kernel doc and information on a device.

A counterpart to /dev/random is /dev/urandom ("unlocked"/non-blocking random source) which reuses the internal pool to produce more pseudo-random bits. This means that the call will not block, but the output may contain less entropy than the corresponding read from /dev/random.

So if your intent is not to generate CSPRNG(Cryptographically secure pseudorandom number generator), you should use /dev/urandom.

Improve this answer
7
  • Does /dev/random really use sources like thermal noise? My understanding is that it uses information from (relatively) unpredictable system status, such as I/O activity and process status. I don't think most Linux systems even have hardware that can harvest thermal noise. Can you cite some documentation on this?
    – Keith Thompson
    Commented Nov 22, 2011 at 0:37
  • yes, you are right. The information I had mentioned is applicable for generic Hardware Random Number Generator.
    – Sachin Divekar
    Commented Nov 22, 2011 at 10:44
  • Look at the doc on how it is implemented in linux at link. There it is mentioned that In a PC environment, the LRNG collects entropy from events originating from the keyboard, mouse, disk and system interrupts. In other environments, the LRNG gathers entropy from the available resources. For example, an OpenWRT router does not include a hard disk, mouse and keyboard and therefore these cannot be used as an entropy sources. On the other hand, the router collects entropy from network events.
    – Sachin Divekar
    Commented Nov 22, 2011 at 10:51
  • Perhaps you could update your answer. I don't believe it's accurate to say that /dev/random generates "truly random numbers".
    – Keith Thompson
    Commented Nov 23, 2011 at 9:21
  • /dev/random article on wikipedia says, Linux was the first operating system to implement a true random number generator in this way in first paragraph.
    – Sachin Divekar
    Commented Nov 23, 2011 at 9:50
3

Just use the shred command that comes with coreutils. It uses random data in an efficient way. dd is a low level tool and that's probably a bit too low level for this task. shred will efficiently skip unwritable portions of the device for example.

Improve this answer
2

Without answering your question -- there are complete answers here already -- you could also check out Darik's Boot and Nuke a.k.a. DBAN which is a live-CD drive wiper.

Improve this answer
1

A 2023 answer of this problem.

/dev/urandom or /dev/random both are slow if those are used as an infinite source of random numbers. A simple hack is to make a sufficiently large block of random numbers and then repeat that block in an infinite loop.

dd if=/dev/random of=/dev/shm/rblob bs=32M status=progress count=10
while true; do cat /dev/shm/rblob; done | dd of=/dev/null bs=32M status=progress

On a GCP VM of sku n2d-standard-16 (16 vCPU, 64G ram), the results are as following

dd if=/dev/random of=/dev/shm/rblob bs=32M status=progress count=10
268435456 bytes (268 MB, 256 MiB) copied, 1 s, 236 MB/s
10+0 records in
10+0 records out
335544320 bytes (336 MB, 320 MiB) copied, 1.41458 s, 237 MB/s

while true; do cat /dev/shm/rblob; done | dd of=/dev/null bs=32M status=progress
189909041152 bytes (190 GB, 177 GiB) copied, 237 s, 801 MB/s
0+1177900 records in
0+1177900 records out
190149951488 bytes (190 GB, 177 GiB) copied, 237.336 s, 801 MB/s

Almost 3.4 times faster.

The size of precomputed random numbers can vary. I chose 320 MB as a sufficiently large size of random blob (considering most compression algorithm's dictionary will fail to detect this repetition).

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .