Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It depends on the application. Often applications do a kind of COW on their own, behind the scene they write out a new file with a randomized naming scheme, when that succeeds, do a rename/replace. So in this case you're getting a new file, as in, a file with a new inode.

If your application is smarter, then it could just COW one block, whatever the minimum block size is for that file system. e.g. for Btrfs it's page size, so x86 that's 4KiB, ppc64le it's 64KiB. I'm not sure what it is for APFS, I'd guess 4KiB. It could actually be inefficient depending on the use case, e.g. a 1MiB file with a single byte change in the middle, if it were a single 1MiB extent, becomes three extents with COW:

    0-500KiB - extent 1
    500-504KiB - hole where stale data is
    504KiB-1MiB - extent 3
    4KiB   - extent 2, with new data, a copy of 4095 bytes from the above hole, with 1 bytes of new data.
Edit: 2nd paragraph suggests the application would COW if it's smarter; that's not correct. If it's smarter, it would request an overwrite of some portion of the file and the file system is what would COW (if it's a COW file system). Granularity depends on a combination of application and file system.

Edit 2: More detailed answer on Btrfs. The whole file system on a zram device is about 260 lines using 'btrfs inspect-internal dump-tree' command. I'll excerpt just the file extents.

1. 'dd if=/dev/zero of=/mnt/test/bunchofzeros.txt bs=M count=1' results in this:

    item 7 key (257 EXTENT_DATA 0) itemoff 3409 itemsize 53
    generation 7 type 1 (regular)
    extent data disk byte 5283840 nr 1048576
    extent data offset 0 nr 1048576 ram 1048576
    extent compression 0 (none)
2. Using vi to change one character and save as the same file, ':wq', I get:

    item 7 key (262 EXTENT_DATA 0) itemoff 3409 itemsize 53
    generation 9 type 1 (regular)
    extent data disk byte 7401472 nr 1052672
    extent data offset 0 nr 1052672 ram 1052672
    extent compression 0 (none)
So you can see it's a new file, new inode, and looks like length is changed too, maybe something to do with encoding.

3. Delete that file and create a new one with the same 'dd' command as in 1.

    item 7 key (266 EXTENT_DATA 0) itemoff 3409 itemsize 53
    generation 11 type 1 (regular)
    extent data disk byte 5283840 nr 1048576
    extent data offset 0 nr 1048576 ram 1048576
    extent compression 0 (none)

    [root@flap ~]# dd conv=notrunc if=/dev/urandom bs=1 seek=524290 count=1 of=/mnt/test/bunchofzeros.txt 
    [root@flap ~]# hexdump -C /mnt/test/bunchofzeros.txt | more
    00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    *
    00080000  00 00 5d 00 00 00 00 00  00 00 00 00 00 00 00 00  |..].............|
    00080010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    *
    00100000

    item 7 key (266 EXTENT_DATA 0) itemoff 3409 itemsize 53
    generation 11 type 1 (regular)
    extent data disk byte 5283840 nr 1048576
    extent data offset 0 nr 524288 ram 1048576
    extent compression 0 (none)
    item 8 key (266 EXTENT_DATA 524288) itemoff 3356 itemsize 53
    generation 14 type 1 (regular)
    extent data disk byte 5242880 nr 4096
    extent data offset 0 nr 4096 ram 4096
    extent compression 0 (none)
    item 9 key (266 EXTENT_DATA 528384) itemoff 3303 itemsize 53
    generation 11 type 1 (regular)
    extent data disk byte 5283840 nr 1048576
    extent data offset 528384 nr 520192 ram 1048576
    extent compression 0 (none)
Clever. item7 and item9 actually reference the same original 1MiB data extent, but use offsets to point to the unchanged portion "creating" a 4KiB hole, and that hole is filed by item 8, a 4KiB extent containing the change. So it really did only COW one block.


Sorry, I can’t tell, are you still talking about APFS?


No the example under Edit 2 is Btrfs. On the details I'd expect APFS to differ. But the gist is that how changes manifest on disk do also depend on the application being used. On any file system, an edit in vi means a whole new file is written out, so it's effectively COW at an application level. Whereas an edit with an application that does byte level overwrites, would get translated into COW on a COW file system like Btrfs, APFS or ZFS. And I'd expect this COW operation to operate at the file system's minimum block level size.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: