Setting up an encrypted git-repository

Question

Situation

Hi, I would like to save data with git, encrypted (on some platform like bitbucket or github). Therefore the question:

Question

I am looking for different hassle-free ways on:
How to set up an encrypted repository on bitbucket(/github)? Now, I am new to git, thus an instruction with all necessary steps or step by step would be much appreciated!

"Research"

git-crypt
I found git-crypt, but on the site it is mentioned that it's intended for single file encryption. If one wanted to encrypt the whole repository they forward to git-remote-gcrypt.

git-remote-gcrypt
In their README.rst they put it as simple as

Quickstart

git remote add cryptremote gcrypt::rsync://example.com:repo
git push cryptremote master
> gcrypt: Setting up new repository
> gcrypt: Remote ID is :id:7VigUnLVYVtZx8oir34R
> [ more lines .. ]
> To gcrypt::[...]
> * [new branch]      master -> master

or under

Examples

# notice that the target git repo must already exist and its
# `next` branch will be overwritten!
git remote add gitcrypt gcrypt::[email protected]:repo#next
git push gitcrypt master

Tries

I prefer full repository encryption, hence I tried git-remote-gcrypt with variations of the Quickstart and Example. So far I tried pushing an existing repository by following their instructions. Which yields this: (note: I have purposely changed the username to user)

-> with ssh as in the provided example

[...]/git_test$ git remote add origin gcrypt::[email protected]:user/test.git
[...]/git_test$ git push -u origin --allgcrypt: Development version -- Repository format MAY CHANGE
gcrypt: Repository not found: [email protected]:user/test.git
gcrypt: Setting up new repository
gcrypt: Remote ID is :id: ...
Zähle Objekte: 10, Fertig.
Komprimiere Objekte: 100% (6/6), Fertig.
Total 10 (delta 0), reused 0 (delta 0)
gcrypt: Encrypting to: --throw-keyids --default-recipient-self
gcrypt: Requesting manifest signature
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
error: Fehler beim Versenden einiger Referenzen nach 'gcrypt::[email protected]:user/test.git'

or with https (which worked)

[...]/git_test$ git remote add gitcrypt gcrypt::https://[email protected]/user/test.git
[...]/git_test$ git push -u gitcrypt --allgcrypt: Development version -- Repository format MAY CHANGE
Password for 'https://[email protected]': 
gcrypt: Repository not found: https://[email protected]/user/test.git
gcrypt: Setting up new repository
Password for 'https://[email protected]': 
gcrypt: Remote ID is :id: ...
Zähle Objekte: 10, Fertig.
Komprimiere Objekte: 100% (6/6), Fertig.
Total 10 (delta 0), reused 0 (delta 0)
gcrypt: Encrypting to: --throw-keyids --default-recipient-self
gcrypt: Requesting manifest signature
Password for 'https://[email protected]': 
To gcrypt::https://[email protected]/user/test.git
 * [new branch]      master -> master
Branch master konfiguriert zum Folgen von Remote-Branch master von gitcrypt.

Nevertheless, I don't understand how to add users or even just pull my backup onto another machine (since my gpg key was generated locally) !? Feel free to just answer on usage of git-remote-gcrypt.

Answer 1

A free and partially open-source tool is Keybase :

Git supports remote helpers. And we've made an open source one.

Keybase's remote helper performs all the crypto while letting git do its thing. This might sound impressive, but Keybase has not reimplemented git from scratch. We provide a remote helper, powered by the excellent go-git project, which we've begun contributing to.

We bring to the table: (1) crypto, (2) team + multi-device key management, (3) a safer concept of identity.

It is end-to-end encrypted. It's hosted, like, say, GitHub, but only you (and teammates) can decrypt any of it. To Keybase, all is but a garbled mess. To you, it's a regular checkout with no extra steps.

Even your repository names and branch names are encrypted, and thus unreadable by Keybase staff or infiltrators.

Teamwork is supported via Keybase Teams :

A Keybase team is a named group of people, with flexible membership. Let's say you work on a project called Treehouse. You could register treehouse on Keybase. This team name is universal; there can be only one Keybase team with a given name.

Teams get chats and channels. The chat looks a bit like Slack or Discord:

But Keybase teamwork is end-to-end encrypted, which means you don't have to worry about server hacks.

Answer 2

Step by step instructions, using git-remote-gcrypt and Bitbucket:

Prerequisites

• GnuPG (brew install gpg on macOS)
• git-remote-gcrypt (brew install git-remote-gcrypt on macOS)

Overview

This guide will walk you through the process of creating and using a git repo called “gcrypt-sample” with Bitbucket, taking advantage of git-remote-crypt with a dedicated PGP keypair to provide end-to-end encryption.

Every push will be a force push!

Steps

Setting Up

1. Set up your local and remote repositories:
   • Set up local repository with git init gcrypt-sample
   • On bitbucket.org, create a repository gcrpyt-sample
2. Set up your ssh key if you haven’t already: ssh-keygen
3. Add a gcrypt remote: git remote add cryptremote gcrypt::[email protected]:user/gcrypt-sample.git
4. Generate your GPG keypair. Recommend this guide; for this sample, simply running gpg --gen-key will work.
5. Configure gcrypt to accept the gpg key you just created:
   1. git config remote.cryptremote.gcrypt-participants <GPG key fingerprint>
   2. git config remote.cryptremote.gcrypt-signingkey <GPG key fingerprint>
6. Edit your git repo:
   1. echo 'Hello, world!' > hello.txt
   2. git add hello.txt
   3. git commit
7. git push -u cryptremote master
   • If you have multiple PGP keypairs, you’ll have to cycle through until you get the one you want—this is because you pull before pushing, every time; see the note below.

A note about pulling from the repo

Everytime you pull down (including when you push), you will have to cycle through your PGP keys until you get to the correct one. The reason for this is that the default for git-remote-gcrypt is to use a privacy guard which redacts the identity of the signing key. Consequently, when pulling from the repository, you have to check all of your available keys to see which one is the signing key. The signing key used can be published (and therefore anonymity revoked) by setting the remote.cryptremote.gcrypt-publish-participants property (to anything).

If you set an explicit and singular PGP key for the repo, doing that should be fine. However, doing so does have implications for shared repos (different commits can be tied undeniably to different users).

What’s Encrypted?

To my naked eye, pretty much everything.

What IS encrypted

• Filenames (including paths)
• File contents
• Branches (I don’t know if the first branch pushed is logged under master, but all subsequent ones were)
• Commit history: There is a single commit (“Initial commit”) and a single date (although dates of your pushes may be tracked by your remote separately, of course—BitBucket’s “Last updated” shows the accurate time. Of course, Bitbucket’s probably not paying attention to the commits themselves—it’s probably assuming you’re doing that.)

What is NOT encrypted

• Repository name (though that’s obvious enough); just use a codename
• The number or size of files (they are not distributed in any way); not sure what the implications are for different branches

Answer 3

I recently had an issue syncing two different machines via an encrypted repository on GitHub with the help of git-remote-gcrypt. So I hope the following will answer the questions on how to add users or even just pull my backup.

Other Sources

These are the sources I used:

• https://manpages.debian.org/testing/git-remote-gcrypt/git-remote-gcrypt.1.en.html seem to be the official manpages.
• https://caolan.uk/notes/2017-04-27_encrypted_git_repositories.cm
• https://git-annex.branchable.com/tips/fully_encrypted_git_repositories_with_gcrypt/
• https://www.alwaysrightinstitute.com/gcrypt/
• these might also be helpful for comparison against other tools: http://ptspts.blogspot.com/2017/10/comparison-of-encrypted-git-remote.html

Prerequisities

I assume that you have setup separate gpg keys on your two computers (key1 on machine 1, key2 on machine 2). Additionally you should create a preferabily private repository on Bitbucket, GitLab or GitHub. I assume a GitHub repository here.

Installation

You should be able to install git-remote-gcrypt easily via

sudo apt install git-remote-gcrypt

or the equivalent statement in your distro. Otherwise the above sources have more information on the installation.

Global configs on both machines

Compatibility with gpg-agent

In order to use the gpg-agent to decrypt the repository run the following command on both machines

git config --global --add gcrypt.gpg-args "--use-agent"

Optional: publish participants

see this note: https://manpages.debian.org/testing/git-remote-gcrypt/git-remote-gcrypt.1.en.html#gcrypt.publish To use the option run on both machines

git config --global --add gcrypt.publish-participants true

Exchange gpg keys

see also here: https://www.gnupg.org/gph/en/manual/x56.html

1. On machine 1 run: gpg --output key1.pgp --armor --export key1
2. and on machine 2: gpg --output key2.pgp --armor --export key2
3. Copy the file key1.gpg to machine 2 and vice versa with key2.gpg.
4. Then run on machine 1: gpg --import key2.gpg
5. and on machine 2: gpg --import key1.gpg
6. After importing the keys you have to validate them as explained in the link above.

Of course you could also use only one key. Then you need to copy the secret key from one machine to the other.

Setup

Initial push on machine 1

1. cd into the folder that you want to sync or backup
2. git init
3. Run git remote add origin gcrypt::[email protected]:<USER>/<MY_REPO>.git
4. Add the keys that are used to encrypt the repository: git config remote.origin.gcrypt-participants "key1 key2"
5. In case that key1 is not your default signing key on machine 1, add it as signing key for this repository with git config remote.origin.gcrypt-signingkey "key1".
6. Add erverything git add .
7. And commit: git commit -m "Initial commit"
8. Optional: git branch -M main
9. And push the repository: git push -u origin main

Clone on machine 2

1. git clone gcrypt::[email protected]:<USER>/<MY_REPO>.git
   1. In case you get prompted to type in the password for your gpg key, cancel until you see that you`re typing the password for the correct key.
   2. Ignore the warning: You appear to have cloned an empty repository.
2. Then cd into the newly created folder.
3. git pull origin main
4. And switch to correct branch: git checkout main
5. Add the participants list on this machine: git config remote.origin.gcrypt-participants "key1 key2".
6. In case that key2 is not your default signing key on machine 2, add it as signing key for the repository on machine 2 with git config remote.origin.gcrypt-signingkey "key2".

Test syncing between both machines

1. Add and commit a change on machine 2.
2. git push
3. Run git pull on the first machine. See the newly added content.
4. Do the same three steps on machine 1 and pull on machine 2.

Troubleshooting

gcrypt: Failed to decrypt manifest!

Make sure that the key that is used to sign the manifest file is in the keys listed in remote.origin.gcrypt-participants. Use the config variables user.signingkey or remote.origin.gcrypt-signingkey to specify the key that is used for signing.

If you want to use a separate key (key3) to sign your git commits from the keys in the remote.origin.gcrypt-participants list (key1 and key2) you need to use the following config:

git config user.signingkey "key3"
git config remote.origin.gcrypt-participants "key1 key2"
git config remote.origin.gcrypt-signingkey "key*"  # Use the correct key depending on your machine!

Therefore the git config variables have the following functions in connection with git and gcrypt

• To sign your commits, git uses user.signkey
• To sign the manifest file, gcrypt uses remote.origin.gcrypt-signingkey. If that is not set, gcrypt bounces back to user.signingkey. Whichever key is used to sign the manifest, this key needs to be part of the remote.origin.gcrypt-participants list.
• gcrypt encrypts your repository, so that it can be decrypted with the keys listed in remote.origin.gcrypt-participants.

Connection reset by peer after git repack

As mentioned in the manpages git-remote-gcrypt sometimes repacks the repository (https://manpages.debian.org/testing/git-remote-gcrypt/git-remote-gcrypt.1.en.html#KNOWN_ISSUES). In my case this lead to a Connection reset by peer error over ssh. Currently, I have to ways to solve that problem:

1. Delete your GitHub repo and recreate it newly. Then re-initiate the connection from your local repo to the newly created remote repo.
2. As this is a very bad solution I tried to also add a https remote:

Use https remote

I created a (classical) personal access token for GitHub and added a separate git remote for https on both machines:

git remote add https gcrypt::https://<USER>@github.com/<USER>/<MY_REPO>.git
# very important to not destroy your repo!!!!
git config remote.https.gcrypt-participants "key1 key2"
git push https # you get several prompts for your https password/token
# pull with the following command
git pull https

I forced a repack with this https remote and the Connection reset by peer error did not occur. However, I do not know what will happen when the repack is initiated by gcrypt itself.

restart gpg-agent

In many cases it does make sense to restart gpg-agent with gpgconf --kill gpg-agent or restart your terminal. This will solve the not found issues as mentioned here: https://manpages.debian.org/testing/git-remote-gcrypt/git-remote-gcrypt.1.en.html#KNOWN_ISSUES

Answer 4

For people following the answer by Niklas Netter of using git-remote-gcrypt installed with

brew install git-remote-gcrypt

Its normal for the final git push to say "repository not found" this is mentioned in the github readme under "known issues" However it still failed for me at the last bit with the error of

gpg: signing failed: Inappropriate ioctl for device
gpg: [stdin]: sign+encrypt failed: Inappropriate ioctl for device"

What commands did I execute:

git remote add gitcrypt gcrypt::[email protected]:YourUserName/your-git-repo.git#main
git push gitcrypt main

Result of those commands:

gcrypt: Repository not found: [email protected]:YourUserName/your-git-repo.git
gcrypt: Setting up new repository
gcrypt: Remote ID is :id:123/someidehere
gcrypt: Due to a longstanding bug, this push implicitly has --force.
gcrypt: Consider explicitly passing --force, and setting
gcrypt: gcrypt's require-explicit-force-push git config key.
Enumerating objects: 6, done.
Counting objects: 100% (6/6), done.
Compressing objects: 100% (3/3), done.
Total 6 (delta 0), reused 0 (delta 0), pack-reused 0
gcrypt: Encrypting to: --throw-keyids --default-recipient-self
gcrypt: Requesting manifest signature
gpg: signing failed: Inappropriate ioctl for device
gpg: [stdin]: sign+encrypt failed: Inappropriate ioctl for device
error: failed to push some refs to 'github.com:YourUserName/your-git-repo.git#main'

However i was able to fix this by following the advise of this other stackoverflow ticket by simply adding this to your bash/zsh profile

Final fix:

GPG_TTY=$(tty)
export GPG_TTY

Answer 5

This can be accomplished in a decoupled way by using an actively developed user-space encrypted filesystem tool such as gocryptfs or cryptomator. The file paths are also encrypted.

gocryptfs usage

Personally I prefer gocryptfs over cryptomator because the former is a single binary and can also more easily be automated. That's even though cryptomator looks to be a more mature project.

gec is a bash utility which attempts to make it more convenient to use an encrypted gocryptfs filesystem in a git repo. At a minimum, note the -sharedstorage option which it uses with gocryptfs.

Cryptomator usage

1. Create an encrypted vault in say ~/cryptomator/encrypted/myvault1. Optionally delete the useless file IMPORTANT.rtf from this directory.
2. Run git init in the above directory containing the encrypted vault. Add one or more git remote URLs.
3. Decrypt the vault into say ~/cryptomator/decrypted/myvault1. Add or change one or more files in the unlocked vault.
4. Commit and push all changes in the git repo.