So I recently wasted 70 hours waiting for a couple of badblocks tests on my “new” NAS system at home. I’m using 2 TB SAS drives (quite old, but quite cheap at ~$10 USD per drive) which I don’t trust yet, so I figured that two runs of badblocks on each drive should be enough to tell if they are good or not.

Problem was, instead of the expected ~24 hours per run (out of four tests in each badblocks run, if that makes sense), I had just finished the first test after 70 hours! Looking at iostat, my writes were approx. 7 megabytes per second, WTF?

So I went on another round of googling, and eventually found a relevant thread on the iXsystems forums. So, it seems that the writeback cache isn’t enabled for my drives when using FreeBSD/FreeNAS, but the OP stated that this was the case when using CentOS (maybe a udev rule at play there?).

I tried to enable the writeback cache, and my write speeds (and writes per second) increased by 10-20 times. That’s better, but it doesn’t seem to stick between boots - but that’s easy to fix with some terrible bash scripting!

#!/usr/bin/env bash

for dev in /dev/da*; do
    current_state=$(smartctl -g wcache $dev|grep -i "Writeback Cache is"|grep -iEow "(Enabled|Disabled)$")

    if [ $current_state != "Enabled" ]; then
        new_state=$(smartctl -s wcache,on $dev)
        success=$?

        if [ $success -ne 0 ]; then
            echo "Something went south when enabling writeback cache on ${dev}"
            echo "${new_state}"
        elif [ $success -eq 0 ]; then
            echo "Writeback cache enabled for ${dev}"
        fi

    elif [ $current_state == "Enabled" ]; then
        echo "Writeback cache already enabled! (${dev})";
    fi
done

iostat -Kx after enabling the writeback cache:

                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
ada0           0       0      0.0      0.0     0     0     0     0    0   0
da0            0    1725      0.0 110428.0     0     0     0     0    1  98
da1            0    1748      0.0 111924.1     0     0     0     0    1  98
da2            0    1783      0.0 114123.6     0     0     0     0    1  98
da3            0    1901      0.0 121702.2     0     0     0     0    1  97
da4            0    1756      0.0 112388.7     0     0     0     0    1  98
da5            0    1735      0.0 111097.2     0     0     0     0    1  98
pass0          0       0      0.0      0.0     0     0     0     0    0   0
pass1          0       0      0.0      0.0     0     0     0     0    0   0
pass2          0       0      0.0      0.0     0     0     0     0    0   0
pass3          0       0      0.0      0.0     0     0     0     0    0   0
pass4          0       0      0.0      0.0     0     0     0     0    0   0
pass5          0       0      0.0      0.0     0     0     0     0    0   0
pass6          0       0      0.0      0.0     0     0     0     0    0   0

Before enabling the writeback cache:

                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
ada0           0       0      0.0      0.0     0     0     0     0    0   0
da0            0     111      0.0   7145.1     0     8     0     8    1 100
da1            0     112      0.0   7174.8     0     8     0     8    1 100
da2            0     112      0.0   7191.8     0     8     0     8    1  99
da3            0     112      0.0   7204.5     0     8     0     8    1 100
da4            0     112      0.0   7191.8     0     8     0     8    1  99
da5            0     112      0.0   7174.8     0     8     0     8    1 100
pass0          0       0      0.0      0.0     0     0     0     0    0   0
pass1          0       0      0.0      0.0     0     0     0     0    0   0
pass2          0       0      0.0      0.0     0     0     0     0    0   0
pass3          0       0      0.0      0.0     0     0     0     0    0   0
pass4          0       0      0.0      0.0     0     0     0     0    0   0
pass5          0       0      0.0      0.0     0     0     0     0    0   0
pass6          0       0      0.0      0.0     0     0     0     0    0   0