[zfs-discuss] Compressratio vs du
richard.elling at richardelling.com
Tue Feb 20 14:46:16 EST 2018
> On Feb 16, 2018, at 3:35 AM, Audun via zfs-discuss <zfs-discuss at list.zfsonlinux.org> wrote:
> This seems intriguing, so I tried exploring this a bit:
> So I created a test file by concatenating /etc/hosts a bunch of times
> to make sure it's easily compressible, and not all zeroes:
> # ls -lh /tank/test/file3
> -rw-r--r-- 1 root root 183K Feb 15 15:24 /tank/test/file3
ls can show the size in addition to the length of a file. Try:
ls -lsh /tank/test/file3
stat can also show more allocation details.
Know that if copies > 1, then the size is expected to be larger than the length.
> # ls -l /tank/test/file3
> -rw-r--r-- 1 root root 186840 Feb 15 15:24 /tank/test/file3
> du shows the file to take 17k:
> # du -h /tank/test/file3
> 17K /tank/test/file3
> # du --block-size=1 /tank/test/file3
> 16896 /tank/test/file3
> Side note, 16896 is 4*4096 + 512
> # du -h --apparent-size /tank/test/file3
> 183K /tank/test/file3
> # du --block-size=1 --apparent-size /tank/test/file3
> 186840 /tank/test/file3
> The file compresses nicely.
> Here is output from zdb:
> # zdb -vvv -O tank test/file3
> Object lvl iblk dblk dsize dnsize lsize %full type
> 156175 2 128K 128K 16K 512 256K 100.00 ZFS plain
> file (K=inherit) (Z=inherit)
> 168 bonus System attributes
> dnode flags: USED_BYTES USERUSED_ACCOUNTED USEROBJUSED_ACCOUNTED
> dnode maxblkid: 1
> uid 1
> gid 1
> atime Thu Feb 15 15:25:34 2018
> mtime Thu Feb 15 15:24:56 2018
> ctime Thu Feb 15 15:24:56 2018
> crtime Thu Feb 15 15:23:13 2018
> gen 18796876
> mode 100644
> size 186840
> parent 222678
> links 1
> pflags 40800000004
> Indirect blocks:
> 0 L1 0:c399652000:1000 20000L/1000P F=2 B=18796896/18796896
> 0 L0 0:c399632000:1000 20000L/1000P F=1 B=18796895/18796895
> 20000 L0 0:c399650000:1000 20000L/1000P F=1 B=18796896/18796896
> segment [0000000000000000, 0000000000040000) size 256K
> I'm no expert, but I read this as the file taking 12.5k (12800 bytes) on disk:
> dnsize (512) + 3x indirect blocks of 0x1000P (4k/4096) each.
> Where does du get 17k from? There seems to be exactly 4096 bytes too
> much from what I see in zdb.
> The pool does not have ashift set explicitly, but disks are 4k and zdb
> -C shows ashift=12 on the single mirrored vdev in this pool.
> I'm no good at reading zdb output, it would be really great if someone
> would point to some documentation.
> Best Regards,
> Audun Gangsto
> On 15 February 2018 at 09:31, Stefan Ring via zfs-discuss
> <zfs-discuss at list.zfsonlinux.org> wrote:
>> On Wed, Feb 14, 2018 at 10:27 PM, Joel Krauska via zfs-discuss
>> <zfs-discuss at list.zfsonlinux.org> wrote:
>>> I've been trying to better understand how to evaluate effective compression
>>> I've come across many examples that compare the output from
>>> du -s . and du -s --apparent-size
>>> as a technique to validate/confirm compression.
>>> # dd if=/dev/zero of=zeros bs=1M count=100
>>> # du -s zeros
>>> 1 zeros
>>> # du -s --apparent-size zeros
>>> 102400 zeros
>>> Using this technique I've been attempting to establish which
>>> directories/datasets compress well and which do not.
>>> However on the whole I'm seeing a pretty big disparity between what du
>>> reports (typically 3x-4x) and what 'zfs get compressratio' reports
>>> (typically 7-8x).
>> If you write only zeroes, that will be converted to a sparse file, and
>> I guess that this does not even count as compression, as there is
>> nothing to compress.
>> zfs-discuss mailing list
>> zfs-discuss at list.zfsonlinux.org
> zfs-discuss mailing list
> zfs-discuss at list.zfsonlinux.org
More information about the zfs-discuss