Toil
Toil (SRE Concept)
Toil is manual, repetitive, automatable work that scales linearly with service growth and provides no lasting value. It's a key concept in SRE that represents work to eliminate.
Characteristics of Toil
Work is toil if it's:
Manual: A human has to do it Repetitive: Done the same way each time Automatable: A machine could do it Tactical: Reactive, not strategic No lasting value: Doesn't improve the system Scales with service: More users = more toil
Examples of Toil
- Manually restarting services - Copying files between systems - Running routine health checks - Provisioning accounts by hand - Responding to the same alert daily - Manual deployment processes
Why Toil is Harmful
- Doesn't scale: Linear effort for linear growth - Prevents improvement: Time spent on toil isn't spent on engineering - Burns out engineers: Repetitive work is demotivating - Creates risk: Manual processes have human error
The 50% Rule
Google's SRE principle: No more than 50% of time should be toil.
The other 50% should be engineering work that: - Automates existing toil - Improves reliability - Builds tools and systems - Creates lasting value
Eliminating Toil
1. Measure it - Track what percentage of time is toil 2. Prioritize - Which toil is most painful/frequent? 3. Automate - Build it once, run it forever 4. Resist new toil - Question manual processes before accepting them