Cluster Scheduling and Distributed File Systems
Stub. Fair resource allocation for shared clusters (dominant resource fairness, DRF) and distributed file systems, whose data-placement and access guarantees are governed by the consistency models developed alongside replication.
Two “back to basics” problems underneath the cloud-scale systems already covered. Scheduling asks how to fairly divide a shared cluster’s multiple resource types (CPU, memory, …) among competing users — dominant resource fairness generalizes max-min fairness from one resource to many, with a fairness axiomatization similar in spirit to Nash bargaining. Distributed file systems ask how to give many clients a shared, POSIX-like file namespace over data replicated/partitioned across many machines — whose read/write guarantees are exactly an application of the data-centric and client-centric consistency models covered in replicated-data-management-and-consistency.
Outline (TODO — flesh out each)
- Cluster scheduling and fairness — dominant resource fairness (DRF)
- Distributed file systems — architecture, data placement
- Consistency models for a distributed file system (pointer to replicated-data-management-and-consistency)