SlideShare a Scribd company logo
Embracing
the
Imposter
vsp@spotify.com
Embracing The Imposter
Over 60M active users
Over 60M active usersOver 30M songs
Over 60M active usersOver 30M songs
58 markets
SAD STORY
TO
START
WITH
WE ARE DOING
MICROSERVICES
FOR YEARS
Microservices 101
Monolith
MicroservicesMonolith
Embracing The Imposter
Multiple datacenters
Thousands of servers
Hundreds of services
Embracing The Imposter
Distributed
System
Distributed
System
AKA
“complicated”
Distributed
System
No
reproducibility
Distributed
System
No
failure cases
for tests
LEGACY
x2 FUN
HARDWARE
x10 FUN at scale
FAILURES
Actually…
Embracing The Imposter
Test Double
IMPOSTER
Imposter
Imposter
Fail miserably
Imposter
Fail miserably
Here you go!
Imposter
Here you go!
Fail in 10 seconds
Imposter
Cat videos
Here you go!
Actually…
Send A, B and C
At rate X
For Y time
Send A, B and C
At rate X
For Y time
Send A, B and C
At rate X
For Y time
Load Injector
Send A, B and C
At rate X
For Y time
Load Injector
Some
More
Steps
Add functions like:
Drop packets
Add
hooks to
service discovery
Add functions like:
Drop packets
Add
pseudo-random
delays
Add functions like:
Drop packets
Add
pseudo-random
payload
Add functions like:
Drop packets
[your function here]
Ways to use
Embracing The Imposter
Embracing The Imposter
Embracing The Imposter
Embracing The Imposter
Embracing The Imposter
Legacy dependency!
Look Ma! No legacy!
Fault injection
Crazy neighbour
Crazy neighbour
Broken hardware
Crazy neighbour
Broken hardware
Cat on a keyboard
Crazy neighbour
Broken hardware
Cat on a keyboard
nuke your DC
future plans
SUT Production
SUT ProductionImposter
Configuration Madness
Imposter, but… for client side
Embracing The Imposter
distributed
systems
are
complicated
extremely
it’s still possible
to improve
testability
takes
effort
to do it
vsp@spotify.com

More Related Content

Embracing The Imposter