research-article

Local-first software: you own your data, in spite of the cloud

Authors:

Martin Kleppmann,

Peter van Hardenberg, and

Mark McGranaghanAuthors Info & Claims

Onward! 2019: Proceedings of the 2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software

October 2019

Pages 154 - 178

https://doi.org/10.1145/3359591.3359737

Published: 23 October 2019 Publication History

Abstract

Cloud apps like Google Docs and Trello are popular because they enable real-time collaboration with colleagues, and they make it easy for us to access our work from all of our devices. However, by centralizing data storage on servers, cloud apps also take away ownership and agency from users. If a service shuts down, the software stops functioning, and data created with that software is lost.

In this article we propose local-first software, a set of principles for software that enables both collaboration and ownership for users. Local-first ideals include the ability to work offline and collaborate across multiple devices, while also improving the security, privacy, long-term preservation, and user control of data.

We survey existing approaches to data storage and sharing, ranging from email attachments to web apps to Firebase-backed mobile apps, and we examine the trade-offs of each. We look at Conflict-free Replicated Data Types (CRDTs): data structures that are multi-user from the ground up while also being fundamentally local and private. CRDTs have the potential to be a foundational technology for realizing local-first software.

We share some of our findings from developing local-first software prototypes at the Ink & Switch research lab over the course of several years. These experiments test the viability of CRDTs in practice, and explore the user interface challenges for this new data model. Lastly, we suggest some next steps for moving towards local-first software: for researchers, for app developers, and a startup opportunity for entrepreneurs.

References

[1]

Dan Abramov. 2018. Redux tutorial: Reducers. https://redux.js.org/ basics/reducers

[2]

J Chris Anderson, Jan Lehnardt, and Noah Slater. 2010. CouchDB: The Definitive Guide . O’Reilly Media. http://guide.couchdb.org/

[3]

Oliver Joseph Ash. 2015. Building an offline page for theguardian.com. https://www.theguardian.com/info/developer-blog/ 2015/nov/04/building-an-offline-page-for-theguardiancom

[4]

Atlassian. 2019. Exporting data from Trello. https://help.trello.com/ article/747-exporting-data-from-trello-1

[5]

Hagit Attiya, Sebastian Burckhardt, Alexey Gotsman, Adam Morrison, Hongseok Yang, and Marek Zawirski. 2016. Specification and Complexity of Collaborative Text Editing. In ACM Symposium on Principles of Distributed Computing (PODC) . 259–268.

Digital Library

[6]

Alexei Baboulevitch. 2018. Data Laced with History: Causal Trees & Operational CRDTs. http://archagon.net/blog/2018/03/24/datalaced-with-history/

[7]

Peter Bailis, Aaron Davidson, Alan Fekete, Ali Ghodsi, Joseph M Hellerstein, and Ion Stoica. 2014. Highly Available Transactions: Virtues and Limitations. In 40th International Conference on Very Large Data Bases (VLDB) . http://arxiv.org/pdf/1302.0309.pdf

[8]

Cihan Biyikoglu. 2018. Under the Hood: Redis CRDTs (Conflict-free Replicated Data Types). http://lp.redislabs.com/rs/915-NFD-128/images/WP-RedisLabs-Redis-Conflict-free-Replicated-DataTypes.pdf

[9]

Peter Bourgon. 2014. Roshi: a CRDT system for timestamped events. https://developers.soundcloud.com/blog/roshi-a-crdtsystem-for-timestamped-events

[10]

Peter Bourgon and Matthias Radestock. 2016. Effortless Eventual Consistency with Weave Mesh. In QCon London. https://www.infoq. com/presentations/weave-mesh

[11]

Reg Braithwaite. 2014. Rendered Prose Diffs. https://github.blog/ 2014-02-14-rendered-prose-diffs/

[12]

Andrew Carter and Michael J. Prichard. 2016. Parse Shutdown: What It Means and What You Can Do. https://willowtreeapps.com/ideas/ parse-shutdown-what-it-means-and-what-you-can-do

[13]

Scott Chacon and Ben Straub. 2014. Pro Git (2nd ed.). https://gitscm.com/book/en/v2

Digital Library

[14]

Steven Chan. 2018. How should I backup my documents? https://support.goodnotes.com/hc/en-us/articles/202168425-Howshould-I-backup-my-documents-

[15]

Roshan Choxi. 2018. PushPin video. https://www.youtube.com/ watch?v=Dox3XAoTCyg

[16]

Council of Europe. 2010. European Convention on Human Rights. https://www.echr.coe.int/Documents/Convention_ENG.pdf

[17]

Council of Europe. 2018. Guide on Article 9 of the European Convention on Human Rights. https://www.echr.coe.int/Documents/ Guide_Art_9_ENG.pdf

[18]

Geoff Cox. 2017. CouchDB, PouchDB and Hoodie as a Stack for Progressive Web Apps. https://medium.com/offline-camp/couchdbpouchdb-and-hoodie-as-a-stack-for-progressive-web-appsa6078a985f18

[19]

Joseph Cox. 2019. Snapchat Employees Abused Data Access to Spy on Users. https://www.vice.com/en_us/article/xwnva7/snapchatemployees-abused-data-access-spy-on-users-snaplion

[20]

Evan Czaplicki and Stephen Chong. 2013. Asynchronous functional reactive programming for GUIs. In 34th Annual SIGPLAN Conference on Programming Language Design and Implementation (PLDI) . 411– 422.

Digital Library

[21]

Matt Day, Giles Turner, and Natalia Drozdiak. 2019. Amazon Workers Are Listening to What You Tell Alexa. https://www.bloomberg.com/news/articles/2019-04-10/is-anyonelistening-to-you-on-alexa-a-global-team-reviews-audio

[22]

Drew DeVault. 2018. The advantages of an email-driven git workflow. https://drewdevault.com/2018/07/02/Email-driven-git.html

[23]

Facebook, Inc. 2017. OpenR documentation: KvStore – Store and Sync. https://github.com/facebook/openr/blob/master/openr/docs/ KvStore.md

[24]

Alex Feyerke. 2013. Say hello to Offline First. http://hood.ie/blog/sayhello-to-offline-first.html

[25]

Brian Fung. 2017. A mysterious message is locking Google Docs users out of their files. https://www.washingtonpost.com/news/theswitch/wp/2017/10/31/a-mysterious-message-is-locking-googledocs-users-out-of-their-files/

[26]

Christina Garman, Matthew Green, Gabriel Kaptchuk, Ian Miers, and Michael Rushanan. 2016. Dancing on the Lip of the Volcano: Chosen Ciphertext Attacks on Apple iMessage. In 25th USENIX Security Symposium . https://www.usenix.org/conference/usenixsecurity16/ technical-sessions/presentation/garman

[27]

Matt Gaunt. 2019. Service Workers: an Introduction. https://developers.google.com/web/fundamentals/primers/serviceworkers/

[28]

Pallab Ghosh. 2015. Google’s Vint Cerf warns of ’digital Dark Age’. https://www.bbc.co.uk/news/science-environment-31450389

[29]

Victor B F Gomes, Martin Kleppmann, Dominic P Mulligan, and Alastair R Beresford. 2017. Verifying strong eventual consistency in distributed systems. Proceedings of the ACM on Programming Languages (PACMPL) 1, OOPSLA (Oct. 2017).

Digital Library

[30]

Google. [n.d.]. See what’s changed in a file. https://support.google. com/docs/answer/190843

[31]

Google. [n.d.]. Suggest edits in Google Docs. https://support.google. com/docs/answer/6033474

[32]

Google LLC. 2019. Google Drive Terms of Service. https://www. google.com/drive/terms-of-service/

[33]

Ink & Switch. 2018. Capstone, a tablet for thinking. https://www. inkandswitch.com/capstone-manuscript.html

[34]

JMichaelTX. 2015. How do I resolve Sync Conflicts? https://discussion.evernote.com/topic/86113-how-do-i-resolvesync-conflicts/

[35]

Meggin Kearney and Jonathan Garbee. 2019. Optimize Performance Under Varying Network Conditions. https://developers.google.com/ web/tools/chrome-devtools/network/network-conditions

[36]

Paul Kinlan. 2019. Adding a Service Worker and Offline into your Web App. https://developers.google.com/web/fundamentals/codelabs/ offline/

[37]

Martin Kleppmann. 2018. CRDTs and the Quest for Distributed Consistency. In QCon London. https://www.infoq.com/presentations/ crdt-distributed-consistency

[38]

Martin Kleppmann and Alastair R Beresford. 2017. A ConflictFree Replicated JSON Datatype. IEEE Transactions on Parallel and Distributed Systems (TPDS) 28, 10 (April 2017), 2733–2746.

Digital Library

[39]

Dan Luu. 2017. Computer latency: 1977–2017. https://danluu.com/ input-lag/

[40]

Igor Mandrigin. 2016. Optimistic UIs in under 1000 words. https: //uxplanet.org/optimistic-1000-34d9eefe4c05

[41]

Mattt. 2018. Network Link Conditioner. https://nshipster.com/ network-link-conditioner/

[42]

Cameron McEfee. 2011. Behold: Image view modes. https://github. blog/2011-03-21-behold-image-view-modes/

[43]

Mark McGranaghan. 2018. Slow Software. https://www. inkandswitch.com/slow-software.html

[44]

Christopher Meiklejohn. 2019. Applied Monotonicity: A Brief History of CRDTs in Riak. http://christophermeiklejohn.com/erlang/lasp/ 2019/03/08/monotonicity.html

[45]

Long Tien Nguyen and Alan Kay. 2015. The Cuneiform Tablets of 2015. In ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) . 297–307.

Digital Library

[46]

Roisin O’Connor. 2019. Myspace loses ‘over 50 million songs’ from website after server migration project goes wrong. https://www.independent.co.uk/arts-entertainment/music/news/ myspace-songs-lost-website-move-migration-mp3-music-serveraccounts-a8827881.html

[47]

Cody Ogden. 2019. Killed by Google. https://killedbygoogle.com

[48]

Anupam Rastogi. 2018. Are Distributed Teams the new Cloud for startups? https://medium.com/@anupamr/distributed-teams-arethe-new-cloud-for-startups-14240a9822d7

[49]

J Rosenberg, R Mahy, P Matthews, and D Wing. 2008. Session Traversal Utilities for NAT (STUN) . Technical Report RFC5389. IETF Network Working Group. https://tools.ietf.org/html/rfc3489

[50]

Alyssa Rosenzweig. 2019. The Federation Fallacy. https://rosenzweig. io/blog/the-federation-fallacy.html

[51]

Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. 2011. A comprehensive study of Convergent and Commutative Replicated Data Types . Technical Report 7506. INRIA. http://hal.inria.fr/ inria-00555588/

[52]

Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. 2011. Conflict-Free Replicated Data Types. In 13th International Symposium on Stabilization, Safety, and Security of Distributed Systems (SSS) . 386–400.

[53]

Dharma Shukla. 2018. Azure Cosmos DB: Pushing the frontier of globally distributed databases. https://azure.microsoft.com/enus/blog/azure-cosmos-db-pushing-the-frontier-of-globallydistributed-databases/

[54]

Mike Skalnik. 2013. 3D File Diffs. https://github.blog/2013-09-17-3d-file-diffs/

[55]

SQLite. 2018. LoC Recommended Storage Format. https://www. sqlite.org/locrsf.html

[56]

Stack Overflow. [n.d.]. When is localStorage cleared? https: //stackoverflow.com/questions/8537112/when-is-localstoragecleared

[57]

Chengzheng Sun and Clarence Ellis. 1998. Operational transformation in real-time group editors: Issues, algorithms, and achievements. In ACM Conference on Computer Supported Cooperative Work (CSCW) . 59–68.

Digital Library

[58]

UN Special Rapporteur on the promotion and protection of the right to freedom of opinion and expression. 2015. Submission to the Joint Committee of the draft Investigatory Powers Bill. http://data.parliament.uk/writtenevidence/committeeevidence. svc/evidencedocument/draft-investigatory-powers-billcommittee/draft-investigatory-powers-bill/written/26353.pdf

[59]

Peter van Hardenberg. 2018. Pixelpusher: Real-time peer-to-peer collaboration with React. https://medium.com/@pvh/pixelpusherreal-time-peer-to-peer-collaboration-with-react-7c7bc8ecbf74

[60]

Jonathan Vanian. 2017. Microsoft Office 365 Just Hit a Big Milestone. http://fortune.com/2017/07/20/microsoft-office-365-earnings/

[61]

Chris Watterston. 2016. My ‘There Is No Cloud’ Sticker. http: //www.chriswatterston.com/blog/my-there-is-no-cloud-sticker

[62]

WhatsApp Inc. 2017. WhatsApp Encryption Overview. https://www. whatsapp.com/security/WhatsApp-Security-Whitepaper.pdf

[63]

Adam Wiggins. 2018. Decentralized, offline-first, realtime collaboration with Automerge. https://www.youtube.com/watch?v= L9fdyDlhByM

[64]

Adam Wiggins. 2018. The iPad as a fast, precise tool for creativity. https://medium.com/@hirodusk/the-ipad-as-a-fast-precisetool-for-creativity-21384ea18659

Cited By

Da LKleppmann M(2024)Extending JSON CRDTs with Move OperationsProceedings of the 11th Workshop on Principles and Practice of Consistency for Distributed Data10.1145/3642976.3653030(8-14)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3642976.3653030
Stewen LKleppmann M(2024)Undo and Redo Support for Replicated RegistersProceedings of the 11th Workshop on Principles and Practice of Consistency for Distributed Data10.1145/3642976.3653029(1-7)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3642976.3653029
Haas JMogk RYanakieva EBieniusa AMezini M(2024)LoRe: A Programming Model for Verifiably Safe Local-first SoftwareACM Transactions on Programming Languages and Systems10.1145/363376946:1(1-26)Online publication date: 15-Jan-2024
https://dl.acm.org/doi/10.1145/3633769
Show More Cited By

Index Terms

Local-first software: you own your data, in spite of the cloud

Recommendations

Binary compatible graphics support in Android for running iOS apps
Middleware '17: Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference

Mobile apps make extensive use of GPUs on smartphones and tablets to access Web content. To support pervasive Web content, we introduce three key OS techniques for binary graphics compatibility necessary to build a real-world system to run iOS and ...
Read More
Cider: native execution of iOS apps on android
ASPLOS '14

We present Cider, an operating system compatibility architecture that can run applications built for different mobile ecosystems, iOS or Android, together on the same smartphone or tablet. Cider enhances the domestic operating system, Android, of a ...
Read More
Mobile Software Ecosystem (MSECO): A Systematic Mapping Study
COMPSAC '15: Proceedings of the 2015 IEEE 39th Annual Computer Software and Applications Conference - Volume 02

The development of mobile applications around a central software platform has been impacting the software industry. Software solutions are collaboratively built within a dynamic market, often requiring adaptation of software development processes. This ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Onward! 2019: Proceedings of the 2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software

October 2019

197 pages

ISBN:9781450369954

DOI:10.1145/3359591

General Chairs:
Hidehiko Masuhara
Tokyo Institute of Technology, Japan
,
Tomas Petricek
University of Kent, UK

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Boeing

Conference

SPLASH '19

Sponsor:

SIGPLAN

SPLASH '19: 2019 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity

October 23 - 24, 2019

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 40 of 105 submissions, 38%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
643
Total Downloads

Downloads (Last 12 months)170
Downloads (Last 6 weeks)15

Other Metrics

View Author Metrics

Citations

Cited By

Da LKleppmann M(2024)Extending JSON CRDTs with Move OperationsProceedings of the 11th Workshop on Principles and Practice of Consistency for Distributed Data10.1145/3642976.3653030(8-14)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3642976.3653030
Stewen LKleppmann M(2024)Undo and Redo Support for Replicated RegistersProceedings of the 11th Workshop on Principles and Practice of Consistency for Distributed Data10.1145/3642976.3653029(1-7)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3642976.3653029
Haas JMogk RYanakieva EBieniusa AMezini M(2024)LoRe: A Programming Model for Verifiably Safe Local-first SoftwareACM Transactions on Programming Languages and Systems10.1145/363376946:1(1-26)Online publication date: 15-Jan-2024
https://dl.acm.org/doi/10.1145/3633769
Kato JHara KHirasawa N(2024)Griffith: A Storyboarding Tool Designed with Japanese Animation ProfessionalsProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642121(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642121
Stokkink QPouwelse J(2024)A Local-First Approach for Green Smart ContractsDistributed Ledger Technologies: Research and Practice10.1145/36071963:2(1-21)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3607196
de Vos MIshmaev GPouwelse J(2024) DeScanFuture Generation Computer Systems10.1016/j.future.2023.11.008152:C(257-272)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1016/j.future.2023.11.008
Ignat CElvinger VBa H(2024)Synql: A CRDT-Based Approach for Replicated Relational Databases with Integrity ConstraintsDistributed Applications and Interoperable Systems10.1007/978-3-031-62638-8_2(18-35)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-62638-8_2
Lavoie E(2023)Designing Peer-to-Peer Systems as Closed Knowledge CommonsProceedings of the 4th International Workshop on Distributed Infrastructure for the Common Good10.1145/3631310.3633491(1-6)Online publication date: 11-Dec-2023
https://dl.acm.org/doi/10.1145/3631310.3633491
Nasirifard PMayer RJacobsen H(2023)OrderlessChainProceedings of the 24th International Middleware Conference10.1145/3590140.3629111(137-150)Online publication date: 27-Nov-2023
https://dl.acm.org/doi/10.1145/3590140.3629111
Litt GSchiefer NSchickling JJackson D(2023)Riffle: Reactive Relational State for Local-First ApplicationsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606801(1-16)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606801
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents