Architecture, Design

Keep it simple

  • Choose boring technology: Bringing in a new technology is expensive to a business, and a team. Everyone has to learn how it works, how to support it, and how it fails. Make sure you do this for good reasons

Batch, Job Execution, Async

  • Job scheduling at Dropbox: Requirements:
    • Priority scheduling
    • At least once execution
    • Only 1 instance of a job is running at a time (no concurrent execution of the same job)
    • SLA: Guaranteed start time of a job after submission is 5 seconds (95th)
    • High availability of the platform
    • Tasks should be idempotent since they can be restarted on failure