RAM versus I/O

And this my friends, is why I love having extra memory!

[terrypoulin@zeta ~]$ dd if=/dev/zero of=./dd.test bs=1M count=2000
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB, 2.0 GiB) copied, 0.38554 s, 5.4 GB/s
[terrypoulin@zeta ~]$ dd if=/dev/zero of=./dd.test2 bs=1M count=2000 oflag=direct
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB, 2.0 GiB) copied, 4.01215 s, 523 MB/s
[terrypoulin@zeta ~]

This machine has a cheapo 1 TB Inland Professional 2.5″ SATA SSD to serve as its system disk. But she’s got 64 freaking gigs of RAM. Yes, that’s right – sixty-four freaking gigs!

[terrypoulin@zeta ~]$ free -h
               total        used        free      shared  buff/cache   available
Mem:            61Gi       966Mi        57Gi       8.0Mi       4.1Gi        60Gi
Swap:           31Gi          0B        31Gi

The first dd command writes 2 GB of zeros to a file one MB at a time, as fast as the system can go. Thanks to the OS being able to say, “Hey, I’ve got memory to buffer that; carry on wayward son,” it is completed Really Damn Fast. This buffering isn’t good for the case of a slow removable disk (or IMHO, oh shit, batteries), but is very effective when doing a lot of file I/O such as compiling software or working on large projects with many files. By contrast if the system had little available memory, it wouldn’t go so fast.

The second command, effectively says the same thing but uses Direct I/O to ensure the data is spewed the disk quickly and immediately, meaning that we get the speed a decent SATA SSD can achieve when combined with its own little bit of internal buffering. But we don’t experience the crazy speeeeeeeed that is RAM.

Why is this important for Zeta? Well, Zeta is replacing Cream — which means she has an 8 TB storage array to take care of, spends most of her life dealing with networked file transfers, streaming media to client machines, and unlike Cream, will end up running several virtual machines thanks to having enough extra memory. Did I mention, fire sales on DDR4 have already begun to make way for DDR5? 😁