# Overview The approaches to consider for scaling writes include... #flashcard - Vertical Scaling and Database Choices - [[Back of the Envelope Estimation]] - Sharding and Partitioning - [[Partitioning]] - Handling Burts with Queues and Load Shedding - Batching and Hierarchical Aggregation # Key Considerations You can apply some tuning to databases to improve write scaling, such as... #flashcard - Turn off foreign key constraints, complex triggers, or full-text search - Tune write-ahead logging (batch multiple transactions before flushing to disk) - Reduce index overhead ## How to handle "bursty" traffic? There are 2 broad approaches: - Buffer the writes so we can process them as quickly as possible without failure - Add a queue and use workers to pull writes onto the DB. Will require clients have a way to call back to check the write was eventually made to the database. - Only offers a temporary solution if writes continue to pour in. The queue's length will continue to grow and writes will take longer and longer to process. - Get ride of writes in a way that is acceptable for the business (aka [[load shedding]]). - For Uber these might be location updates that are within seconds of a previous update. For an analytics system, we might drop impressions for a while to ensure we can process the more important clicks. # Implementation Details ## Common Deep-dives ### How do you handle resharding when you need to add more shards? #flashcard - Production systems use gradual migration which targets writes to both locations (e.g. the shard we're migrating from and the shard we're migrating to). This allows us to migrate data gradually while maintaining availability. - The dual-write phase ensures no data is lost during migration. You write to both old and new shards, but read with preference for the new shard. This allows you to migrate data gradually while maintaining availability. # Useful Links # Related Topics ## Reference #### Working Notes #### Sources