Please enable JavaScript.
Coggle requires JavaScript to display documents.
qcow2-rs write IO - Coggle Diagram
qcow2-rs write IO
cluster allocator
check if the last refcount block cache exsits
yes, cache hit
check if there is cluster availabe
yes, allocate one and mark it refcount as 1
no
grow refcount table
refcount table index += 1
allocate new refcount block cache
allocate one cluster from this cache
no, load this refcount blk table into cache
check if there is cluster availabe
cache key is host cluster offset
lock design
mutex is ok?
every new allocated cluster should be discarded first?
yes
one hashmap for recording clusters to be discarded
mark new allocated cluster
for data
discard other parts before completing write request
for refblock table
discard this cluster before flushing any slice in this table
mark new cluster in reserved bits of l1 entry
for l2 table
discard the cluster before flushing any slice in this table
mark new cluster in reserved bits of reftable entry
requirements
only support aligned RW
FS block size aligned
support concurrent calling
COW
backing
similar
compression
decompress data into temp cluster buffer
then write data into this buffer
finally write whole cluster and map it to this entry
concurrent
updating l2 cache
lock is needed
async/.await way
wait until mapping is ready
wait until all cluster IO is setup
questions
when to complete write IO?
data write is completed
meta flushing can be done in bg
how to flush real meta IO really?
async or sync?
futures join!()
data integrity vs. data loss
care integrity more from begining
flush meta in background
only meta load happens in io code path
how to deal with running out of cache
wait until cache is flushed out
or start & wait foreground flushing
background meta flushing is required
running out of cache may not be triggered
bg flushing starts from less used caches
flushing refcount before mapping meta
tolerate data leak
easier to fix
how to model meta dirty?
store cache key into dirty vector?
or store dirty into into cache entry
cache can be queried, but can't be updated when flushing
how to model meta cluster discard
discard is needed for new mapping
use top table reserved bit?
meta data flushing order
take soft update approach?
write cases
meta write
mapping meta
l2 slice flush
l1 table
refcount meta
refcount blk slice
refcount table
cluster needs to be discarded before adding mapping to table
data write
in-place write
discard isn't needed
discard is needed when adding this mapping
io write completes when data write is done
don't wait meta flushing
write IO
partition IO into clusters
this cluster mapping is hit in l2 cache?
L1 entry doesn't exist
allocate one cluster as l2 table
fill the mapping in the l2 table
allocate one data cluster
L1 entry exist
load l2 table by starting IO
l2 mapping exists?
no, allocate data cluster and setup the mapping
yes, make sure cache is updated
cache key is virtual offset