Overview
In Git superprojects, a submodule is stored as a gitlink (a SHA-1 pointer). During merges, Git must reconcile two pointers to the submodule. You’ll see messages like:
- error: failed to merge submodule
- CONFLICT (submodule): Merge conflict in <path>
This typically means one of:
- The submodule commit from one side isn’t present locally (not fetched).
- The parent updates point to different submodule commits and Git can’t auto-resolve.
- .gitmodules or submodule metadata is inconsistent.
- The submodule has uncommitted changes (“dirty”), blocking the merge.
Quickstart (Runbook)
- Make sure submodules are initialized and commits are available
# from the superproject root
git submodule sync --recursive
git submodule update --init --recursive --jobs 8
# If still missing commits, fetch inside each submodule
git submodule foreach --recursive "git fetch --all --tags --prune"
- If the merge reports a submodule conflict, choose a side or produce a new submodule commit
- Choose “ours” (keep current branch’s pointer):
git checkout --ours path/to/submodule
git add path/to/submodule
git commit -m "Resolve submodule by ours"
- Choose “theirs” (keep merging branch’s pointer):
git checkout --theirs path/to/submodule
git add path/to/submodule
git commit -m "Resolve submodule by theirs"
- Or merge inside the submodule, then update the parent pointer:
# enter the submodule, merge its branches to create a new commit
cd path/to/submodule
# ensure both SHAs/branches exist locally
git fetch --all
# create/checkout a branch and merge the two commits
# (resolve conflicts here if needed)
# Example:
# git checkout main && git merge <other-commit-or-branch> && git push
cd -
# point superproject to that new submodule commit
( cd path/to/submodule && NEW_SHA=$(git rev-parse HEAD); cd -; )
git add path/to/submodule
git commit -m "Resolve submodule by updating to $NEW_SHA"
- If .gitmodules conflicts, resolve and sync
# edit .gitmodules to the desired URLs/paths
git add .gitmodules
git commit -m "Resolve .gitmodules"
git submodule sync --recursive
Minimal working example
This reproduces a submodule merge conflict and shows two ways to fix it.
set -e
rm -rf /tmp/super /tmp/sub
# Create a submodule repository with divergent commits
mkdir -p /tmp/sub && cd /tmp/sub
git init -q
printf "one\n" > file.txt
git add file.txt && git commit -qm "sub: A"
A=$(git rev-parse HEAD)
git checkout -qb branchB
printf "two\n" >> file.txt
git commit -am "sub: B" -q
B=$(git rev-parse HEAD)
git checkout -qb branchC main
printf "three\n" >> file.txt
git commit -am "sub: C" -q
C=$(git rev-parse HEAD)
cd - >/dev/null
# Create a superproject and add submodule at commit A
git init -q /tmp/super
cd /tmp/super
git submodule add -q /tmp/sub vendor/sub
( cd vendor/sub && git checkout -q $A )
git add . && git commit -qm "super: add submodule at A"
# Branch 1 points submodule to B
git checkout -qb feature
( cd vendor/sub && git checkout -q $B )
git add vendor/sub && git commit -qm "feature: update submodule to B"
# Branch 2 points submodule to C
git checkout -qb staging main
( cd vendor/sub && git checkout -q $C )
git add vendor/sub && git commit -qm "staging: update submodule to C"
# Merge feature into main, then try staging -> conflict expected
git checkout -q main
git merge -q feature
git merge staging || true
echo "Status after conflict:"
git status --short
Fix option A: pick one side (fastest in CI)
# Keep staging's submodule pointer
git checkout --theirs vendor/sub
git add vendor/sub
git commit -m "Resolve submodule conflict: take staging"
Fix option B: merge submodule history and update parent pointer
# Merge B and C inside the submodule to produce a new commit
cd vendor/sub
# create a branch to merge onto (e.g., main)
git checkout -q main
# ensure both commits are present, then merge
git merge -q $B || true # resolve if needed, then:
git commit --no-edit || true
NEW=$(git rev-parse HEAD)
cd ../..
# Update superproject to NEW
( cd vendor/sub && git checkout -q $NEW )
git add vendor/sub
git commit -m "Resolve submodule conflict by merging inside submodule"
Fixing common cases (step-by-step)
- Missing submodule commit (cannot find object)
- Symptom: “fatal: reference is not a tree” or fetch errors.
- Fix:
- git submodule update --init --recursive
- git submodule foreach --recursive "git fetch --all --tags --prune"
- Verify remotes and access tokens for private submodules.
- Plain conflict (two different gitlinks)
- Decide policy per repository:
- Keep ours: git checkout --ours path && git add path && git commit
- Keep theirs: git checkout --theirs path && git add path && git commit
- Integrate changes: merge inside the submodule, push, then set the parent pointer to the new commit and commit in the superproject.
- .gitmodules conflict or drift
- Resolve .gitmodules by hand, then:
- git add .gitmodules && git commit
- git submodule sync --recursive
- git submodule update --init --recursive
- Dirty submodule blocks merge
- Symptom: “local changes in submodule … would be overwritten.”
- Fix:
- Commit or stash inside the submodule: git -C path commit -am … or git -C path stash -u
- Retry the merge.
- Corrupt or mislinked submodule metadata
- Symptoms: .git/modules/<name> out of sync, or submodule is a plain dir.
- Fix (safe reset):
git submodule deinit -f path/to/submodule
rm -rf .git/modules/$(basename path/to/submodule)
git rm -f path/to/submodule
git commit -m "Remove broken submodule"
# re-add
git submodule add <url> path/to/submodule
git commit -m "Re-add submodule"
Diagnostics you can trust
- Show submodule status and diffs:
git status
git diff --submodule=log
- See which commit each side wants during merge:
git ls-files -s path/to/submodule # shows stage 1/2/3 gitlinks
- Inspect inside submodule:
git -C path/to/submodule log --oneline --decorate --graph --all -n 10
Performance notes (DevOps/CI)
- Parallel and shallow updates:
git submodule update --init --recursive --jobs 8 --depth 1
- Avoid unnecessary recursion when not needed:
git -c submodule.recurse=false pull
- Cache submodule repos between CI jobs (e.g., in a shared volume) to skip refetching.
- Only update submodules you need:
git submodule update --init path/one path/two
Pitfalls
- Detached HEAD inside submodule: checkout a branch before committing merges.
- Force-pushed submodule histories can invalidate pointers; fetch all and coordinate teams before rewriting history.
- Private submodules require CI credentials; missing tokens look like “not found” or missing objects.
- Nested submodules require --recursive for update/fetch.
Tiny FAQ
Q: Why does picking ours/theirs “just work”? A: The superproject stores only a pointer (SHA). Choosing a side replaces that pointer and resolves the conflict.
Q: When should I merge inside the submodule? A: When both sides contain meaningful changes you want to keep. Create a new submodule commit that integrates them, then update the parent pointer.
Q: I still get “reference is not a tree”. A: The submodule commit isn’t available. Ensure the submodule remote contains that commit and that your CI/user can fetch it (auth, network, branch).
Q: Can I prevent these conflicts? A: Reduce parallel updates to submodule pointers, track a submodule branch in a single integration branch, and require PRs to fast-forward the submodule or align on a merge policy.
