PFS DD: replication and garbage collection
diff --git a/docs/dd_pfs.md b/docs/dd_pfs.md
index 1cf6c7c..6410195 100644
--- a/docs/dd_pfs.md
+++ b/docs/dd_pfs.md
@@ -33,9 +33,15 @@
Size int
// Number of bytes committed to disk
Committed int
+ // TODO(lekva): add file path and update bytes bellow
}
```
Total of 16 + 3 * 32 = 112 bytes are needed to store single chunk metadata. On top if this thread-safe hash map backed ChankInfoStore structure will be built with two Load and Store methods. Store method will update in memory hash map and also append it to transaction logs. Background process will compact transaction logs periodically and persist full hash map contents on disk.
Controller will not persist any data locally. Instead it will receive state of chunk servers periodically using heart beats. This makes it is easier to keep metadata stored in controller and chunk servers consistent.
+Chunk server will provide RPC interface with ListChunks, CreateChunk, GetChunkInfo, ReadChunk, WriteChunk, RemoveChunk methods. ReadChunk and WriteChunk methods will ask for offset which in conjunction with ChunkInfo.Commited variable will be used to correctly access chunk payload.
+
+Controller will provide RPC interface with CreateBlob, GetBlobInfo and RemoveBlob methods. When creating blob, controller will decide how to cut it into smaller chunks and their assignment to chunk servers, then call CreateChunk method on all involved assigned chunk servers, and return BlobInfo back to the client with blob and chunk ids. For each chunk one of the chunk servers will be chosen as a primary which will receive data from the client while all others, secondary replicas, will start a background process of replicating data from primary.
+
+Controller will run background process assesing health of chunk servers. It will trigger new chunk creation and replication as soon as one of the chunk servers goes down. Similarly there will be a background garbage collector running whuch will trigger removal of unneccessary chunk replicas. This can happen once previously offline chunk server, whos contents have already been replciated, comes back online. Same functionallity will be used to lazily delete blobs which have been marked for deletion.
\ No newline at end of file