Life-saving OpenZFS

I learned today: OpenZFS is a lifesaver. Had stuff really gone belly-up, I could have rolled back to an earlier snapshot and troubleshooted on my own time.

‒ Ruben Schade, https://rubenerd.com/erroneous-systemd-ghost-errors-in-freebsd/

I had two drives in my mirrored zpool die within 8 minutes of one another.
Both HGST drives too. A very sad day.
Thankfully I had been regularly zfs sending my contents to another site and lost very little data.
ZFS is rad.

– https://news.ycombinator.com/item?id=37230619

There's a really important subtlety that a lot of people are missing in this. The bug is not in reads. If you read data, its there. The bug is that sometimes, asking the filesystem "is there data here?" it says "no" when it should say "yes". This distinction is important, because the vast majority of programs do not ask this - they just read.
Further, the answer only comes back "no" when it should be "yes" if there has been a write on that part of the file, where there was no data before (so overwriting data will not trip it), at the same moment from another thread, and at a time where the file is being synced out already, which means it had a change in the previous transaction and in this one.

– ZFS bug description, https://news.ycombinator.com/item?id=38553008