SPM unknown error reference not found when changing branch

Question

Some of my project's dependencies are hosted on a private repository. It works most of the time, but sometimes when I change the current branch with git, I get this error :

❌ git@my_private_repo.git: An unknown error occurred. reference 'refs/remotes/origin/main' not found (-1)

And from that point, it's impossible to compile and my only option is to reset the SPM cache which takes a lot of time.

Any idea on what causes this and how to solve it?

Answer 1

Like others have answered, the issue can be worked around by deleting Swift package caches both in the derived data directory of your project, as well as the global Swift Package Manager cache in ~/Library/Caches/org.swift.swiftpm. Even better, by deleting only the affected remotes directory, but it can be time consuming to find the package and the folder.

Here is a small script I quickly put together that will delete all directories named remotes both in the global Swift Package Manager cache and in the derived data repositories directory of your project.

#!/bin/bash

if [[ $# -eq 0 ]] ; then
    echo 'Please call the script with the name of your project as it appears in the derived data directory. Case-insensitive.'
    echo 'For example: ./fix-spm-cache.sh myproject'
    exit 0
fi

# Delete all directories named "remotes" from the global Swift Package Manager cache.
cd ~/Library/Caches/org.swift.swiftpm/repositories

for i in $(find . -name "remotes" -type d); do
    echo "Deleting $i"
    rm -rf $i
done

# Find derived data directories for all projects matching the script argument, and
# delete all directories named "remotes" from source package repositories cache for those projects. 

cd ~/Library/Developer/Xcode/DerivedData/

for project in $(find . -iname "$1*" -type d -maxdepth 1); do
    for i in $(find "$project/SourcePackages/repositories" -name "remotes" -type d); do
        echo "Deleting $i"
        rm -rf $i
    done
done

If you save the code in a file named fix-spm-cache.sh, then you can do chmod +x fix-spm-cache.sh to make the file executable. Afterwards, just run the script with the project name, like ./fix-spm-cache.sh myproject, when you encounter the error in Xcode.

You can keep Xcode open while running the script. Once the script finishes executing, try resolving or updating your packages again. It should work, and it should be relatively fast since we are not deleting the whole cache.

This should resolve both the error mentioned in this topic, as well as the infamous SwiftPM.SPMRepositoryError error 5 that happens in Xcode 13, which is probably the same error, just with a different message.

Answer 2

I think I may have found the issue here! After a ton of digging it seems Xcode and Swift PM has a bug with repos using git@ rather than https://

Using ssh we are getting a hanging ref to remote/origin/main in our caches and derived data. When using https this is not the case. This makes sense as the only dep in our project using ssh are our internal deps.

I tested this by adding a brand new 3rd party dependency [email protected]:jpsim/Yams.git into our project and saw the cache in org.swift.swiftpm update incorrectly.

UPDATE: 10 days later and it seems this was the issue all along. However even after changing the references it took a full DerivedData/SPM cache/Package.resolved wipe before Xcode no longer had any references to the git@ repos. I've filed a feedback with Apple for this bug.

Answer 3

After having this problem in XCode and AppCode, which does not have a "Reset Package Cache" button and takes forever re-indexing the whole iOS Framework when you delete the DerivedData folder, i tried to minimize the folders i needed to delete.

I ended up with the following: Once you get the error message [email protected]: An unknown error occurred. reference 'refs/remotes/origin/main' not found (-1), you

1. go to your DerivedData folder (~/Library/Developer/Xcode/DerivedData/) (for AppCode, it's located in ~/Library/Caches/JetBrains/AppCode2022.2/DerivedData)
2. go to the following directory, matching your project name, of course: <YourProject>-<RandomCharacters>/SourcePackages/repositories/<FailingLibraryProject>-<RandomCharacters>/refs/
3. delete the remotes folder
4. In XCode, click File -> Packages -> Resolve Package Versions
5. In AppCode, click Tools -> Swift Package Manager -> Resolve Dependencies.
6. Close and Re-open XCode

That's all it takes - at least for me. I guess you need to repeat the steps for every library project that's failing for you. And the best thing: I never need to do this again, no matter how many times i switch branches or dependency versions :)

Answer 4

Had same issue, even created new repo with same content to see that it is working. In the end I found a solution that helped me.

1. Remove package from project
2. Close Xcode
3. Clean SPM - swift package purge-cache
4. remove DerivedData - rm -rf ~/Library/Developer/Xcode/DerivedData
5. Open project and add package again.

P.S. purge-cache w/o removing DerivedData not worked, but it could be that only removing DerivedData was required to solve this issue. Can not re-check as I can not reproduce this issue anymore.

UPD: Step #3 not required.

Answer 5

Something that seemed to work for me today, that only takes a few seconds and that doesn't delete the derived data folder :

• Temporarily remove the dependency that causes the issue in your Swift package
• Close Xcode
• Checkout the Package.resolved to cancel its changes
• Reopen Xcode

After a few seconds, everything seemed to be normal again.

Answer 6

I had the same error but with repo added via https. I just removed cache at ~/Library/Caches/org.swift.swiftpm and seems like it works. Reset packages, clear derived data didn't help.

Answer 7

Multiple functioning workarounds can be found above. I used this approach:

• Quitting Xcode
• Deleting ~/Library/Caches/org.swift.swiftpm
• Deleting ~/Library/Developer/Xcode/DerivedData/YourProjectName
• Launching Xcode

Unfortunately, re-downloading the packages could take a very long time - as well as compiling them all again, when only a single repo fails.

I am not sure why reason Xcode's initial fetch succeeds and its subsequent fetches fail - neither does Xcode: "unknown error". It would be good to figure out, what the cause is. Some setup on my machine seems to be wrong, as my colleagues don't have this problem. Unfortunately, I could not figure it out.

So I ended up building this workaround, that just does what, that Xcode does not do: fetching the repos, that usually fail. It does so by going into the DerivedData folder and fetching either all or the specified repos. Xcode will then be able to resolve the packages.

I put this function into my .profile (sorry, I am still using bash).

function updateXcodeSPM() {
    local current_dir="$PWD"
    local target_dir="$HOME/Library/Developer/Xcode/DerivedData"
    local specific_libraries=("$@")

    find "$target_dir" -type d -name "SourcePackages" | while read spm_dir; do
        if [[ ! "$spm_dir" =~ \.noindex ]]; then
            find "$spm_dir/repositories" -type d -mindepth 1 -maxdepth 1 | while read repo_dir; do
                if [ ${#specific_libraries[@]} -eq 0 ]; then
                    # No specific libraries provided, update all
                    echo "Fetching in $repo_dir"
                    (cd "$repo_dir" && git fetch) &
                    sleep 0.1
                else
                    # Update only specific libraries
                    for lib in "${specific_libraries[@]}"; do
                        if [[ "$repo_dir" == *"repositories/$lib-"* ]]; then
                            echo "Fetching in $repo_dir"
                            (cd "$repo_dir" && git fetch) &
                            sleep 0.1
                        fi
                    done
                fi
            done
        fi
    done

    cd "$current_dir"
}

You can call this function without parameters or with the names of the packages failing to download:

> updateXcodeSPM FailingPackage1 FailingPackage2

or

> updateXcodeSPM

I recommend listing the failing package(s) because it is quicker.

Hope, it helps you.

Answer 8

I made a python script to fix the issue. The idea is to get the latest commit hash and write it to Package.resolved, then resolvePackageDependencies for you.

You can get the script below, or download from here

# Sometimes Xcode cannot resolve SPM(File -> Packages -> Resolve Package versions) if the dependency url is ssh
# This script is a workaround to resolve package versions.
# Usage:
#     python spmResolve.py
# or
#     python3 spmResolve.py
import os.path
import subprocess
import glob
import json


def main():
    package_file = "xcshareddata/swiftpm/Package.resolved"
    xcodeproj = glob.glob('*.xcodeproj')
    xcworkspace = glob.glob('*.xcworkspace')
    spmproj = glob.glob('Package.resolved')
    package_resolved = ""
    if xcodeproj:
        package_resolved = xcodeproj[0] + f"/project.xcworkspace/{package_file}"
    elif xcworkspace:
        package_resolved = xcworkspace[0] + f"/{package_file}"
    elif spmproj:
        package_resolved = spmproj[0]
    else:
        print(f"😱 Cannot find *.xcodeproj, *.xcworkspace or Package.resolved file")
        exit(-1)

    update_package_resolved(package_resolved)


def update_package_resolved(package_resolved):
    if not os.path.exists(package_resolved):
        print(f"😱 Package.resolved file doesn't exit: {package_resolved}")
        exit(-1)

    print(f"Found: {package_resolved}")
    f = open(package_resolved)
    content = json.load(f)
    f.close()
    for pin in content["pins"]:
        url = pin["location"]
        if "branch" in pin["state"]:
            branch = pin["state"]["branch"]
            commit_hash = get_git_revision_hash(url, branch)
            print(f"{url}, {branch}, {commit_hash}")
            pin["state"]["revision"] = commit_hash
        elif "version" in pin["state"]:
            version = pin["state"]["version"]
            commit_hash = get_git_revision_by_tag(url, version)
            print(f"{url}, {version}, {commit_hash}")
            pin["state"]["revision"] = commit_hash

    with open(package_resolved, "w") as output:
        json.dump(content,  output, indent=4)

    # resolve SPM
    subprocess.run(['xcodebuild', '-resolvePackageDependencies'])
    print('🎉 Well done')


def get_git_revision_hash(url, branch) -> str:
    command = f'git ls-remote {url} refs/heads/{branch} | cut -f 1'
    return get_git_command_output(command)


def get_git_revision_by_tag(url, version) -> str:
    command = f'git ls-remote {url} -t {version} | cut -f 1'
    return get_git_command_output(command)


def get_git_command_output(command) -> str:
    return subprocess.check_output(command, stderr=subprocess.STDOUT, shell=True).decode('ascii').rstrip()


if __name__ == '__main__':
    main()

Put the script in your xcode project, along with your *.xcodeproj or *.xcworkspace. Then run

python spmResolve.py

or

python3 spmResolve.py

If you don't have python:

brew install python

Answer 9

Solve problem by running these commands

defaults delete com.apple.dt.xcode DVTSourceControlAccountDefaultsKey

defaults delete com.apple.dt.xcode IDESourceControlKnownSSHHostsDefaultsKey

sudo killall ssh-agent

More info: https://forums.developer.apple.com/forums/thread/662690