Resolve curl 18 in Git: transfer closed with data remaining

What the error means

curl error 18 (CURLE_PARTIAL_FILE) indicates the server closed the connection before all expected data was received. In Git workflows (clone/fetch over HTTP/HTTPS), this usually happens mid-packfile download due to a proxy/load balancer timeout, HTTP/2 quirks, or a misbehaving server.

Typical message: curl 18 transfer closed with outstanding read data remaining.

Quickstart (client-side fixes)

Try these in order. Stop when the command succeeds.

• Force HTTP/1.1 (avoid some HTTP/2/proxy issues):
  • git -c http.version=HTTP/1.1 clone https://example.com/repo.git
• Retry with a shallow clone (less data to transfer):
  • git -c http.version=HTTP/1.1 clone --depth 1 https://example.com/repo.git
• Bypass corporate proxy just for the Git host (if allowed):
  • NO_PROXY=git.example.com git clone https://git.example.com/repo.git
• Reduce parallel HTTP requests (less stress on intermediaries):
  • git -c http.maxRequests=1 clone https://example.com/repo.git
• Increase tolerance for slow links (wait longer before aborting):
  • git -c http.lowSpeedLimit=1 -c http.lowSpeedTime=600 clone https://example.com/repo.git

If these consistently fail, see Debugging and Server-side fixes.

Minimal working example: robust clone wrapper

A small script that retries, forces HTTP/1.1, limits parallelism, and falls back to a shallow clone.

#!/usr/bin/env bash
set -euo pipefail

REPO_URL=${1:?"usage: $0 <repo-url> [dest]"}
DEST=${2:-}

attempt_clone() {
  local depth_flag=${1:-}
  git \
    -c http.version=HTTP/1.1 \
    -c http.maxRequests=1 \
    -c http.lowSpeedLimit=1 \
    -c http.lowSpeedTime=600 \
    clone ${depth_flag} --progress "$REPO_URL" ${DEST:+"$DEST"}
}

# Try full clone with retries, then shallow as fallback
for i in {1..3}; do
  echo "Attempt $i: full clone"
  if attempt_clone; then exit 0; fi
  sleep 2
done

echo "Falling back to shallow clone"
for i in {1..3}; do
  echo "Attempt $i: shallow clone"
  if attempt_clone "--depth 1"; then exit 0; fi
  sleep 2
done

echo "Clone failed after retries" >&2
exit 1

Usage:

./safe_git_clone.sh https://example.com/repo.git myrepo

Debugging checklist

1. Confirm it’s curl 18

• Run: GIT_CURL_VERBOSE=1 GIT_TRACE=1 git fetch -p
• Look for curl 18 and where it happens (packfile download).

1. Check proxy and TLS middleboxes

• If using a corporate proxy/SSL inspection, try outside the proxy or set NO_PROXY for the Git host.
• Verify proxy environment variables: env | grep -i _proxy

1. Disable HTTP/2 for the call

• git -c http.version=HTTP/1.1 fetch
• If this helps, make it permanent: git config --global http.version HTTP/1.1

1. Limit concurrency

• git config --global http.maxRequests 1
• Re-try clone/fetch.

1. Test with smaller transfer

• git clone --depth 1 ... or git fetch --depth 1
• If this works, the issue is likely a timeout/size threshold in a proxy or server.

1. Increase low-speed tolerance

• git -c http.lowSpeedLimit=1 -c http.lowSpeedTime=600 fetch
• Useful on high-latency or lossy links.

1. Reproduce with curl (optional)

• If the repo hosts large files over the same domain, try: curl -v --http1.1 -O https://host/path/largefile
• Any truncation or mid-transfer close suggests a proxy/server issue.

1. Gather details for ops

• Note time of failure, repo URL, client IP, and whether VPN/proxy is involved.
• Share GIT_CURL_VERBOSE logs with the ops team.

Common causes and fixes

• Proxy/load balancer idle timeout

  • Symptom: Failures near the same elapsed time, large repos only.
  • Fix: Increase upstream/proxy read/send/idle timeouts; keep-alive tuned.

• HTTP/2 stream reset by intermediary

  • Symptom: Works with HTTP/1.1, fails with HTTP/2.
  • Fix: Force HTTP/1.1 on the client; investigate/proxy HTTP/2 config.

• Server closing early (mis-sized Content-Length or truncated chunk)

  • Symptom: Immediate curl 18 mid-transfer.
  • Fix: Check server logs, disable buffering, ensure correct upstream framing.

• SSL inspection or flaky VPN

  • Symptom: Only on corporate network/VPN.
  • Fix: Exempt Git domains from inspection or adjust MTU/fragmentation settings.

• Over-parallelization

  • Symptom: Concurrent fetches reliably fail; serial clones succeed.
  • Fix: git config http.maxRequests 1 (or low value).

Server-side fixes (DevOps)

If you operate the Git HTTP endpoint or its proxy:

• Increase timeouts for large packfiles
  • Nginx: proxy_read_timeout, proxy_send_timeout, keepalive_timeout.
  • Apache: ProxyTimeout, TimeOut, KeepAliveTimeout.
• Consider disabling HTTP/2 temporarily on the Git path if intermediaries are suspect.
• Avoid buffering large upstream responses
  • Nginx: proxy_request_buffering off (for pushes), tune proxy_buffering for large pulls.
• Ensure upstream Git service streams reliably
  • Keep pack-objects/upload-pack responsive; avoid CPU throttling that stalls streams.
• Right-size connection pools and keepalive
  • Prevent premature upstream closure under load.
• Check for middlebox interference
  • CDN/WAF rules that cut long-lived connections; adjust or bypass for Git paths.

Make changes persistent (client)

# Prefer HTTP/1.1 globally
git config --global http.version HTTP/1.1

# Limit parallel HTTP requests
git config --global http.maxRequests 1

# Be patient on slow links
git config --global http.lowSpeedLimit 1
git config --global http.lowSpeedTime 600

To scope to a single repo, omit --global and run inside the repo.

Pitfalls to avoid

• Relying on http.postBuffer for clone/fetch
  • This setting affects how Git sends data (mainly pushes); it won’t fix mid-download truncation.
• Disabling SSL verification
  • Avoid http.sslVerify=false; it weakens security and does not address truncation.
• Infinite retries
  • Retrying without changing conditions can overload servers and won’t fix deterministic timeouts.
• Ignoring proxies
  • System-wide HTTP(S)_PROXY can affect Git even if you configured per-repo settings.

Performance notes

• Forcing HTTP/1.1 may reduce throughput versus HTTP/2 on healthy paths, but it often stabilizes flaky proxy chains.
• Limiting http.maxRequests lowers concurrency and peak speed but can prevent resets on fragile proxies.
• Shallow clones dramatically cut transfer size; use them for CI when full history isn’t needed.
• Tune lowSpeed settings carefully; too high a lowSpeedLimit or too short a lowSpeedTime increases false positives.

Tiny FAQ

• Q: Is curl 18 a Git bug?

  • A: Not usually. It’s most often a transport/proxy/server issue causing truncation.

• Q: Can I ignore the error and proceed?

  • A: No. The packfile is incomplete; Git data integrity would be compromised.

• Q: Why does it happen only on VPN or in CI?

  • A: Different path, proxies, MTU, or resource limits. CI often hits size/time thresholds.

• Q: Does switching to SSH help?

  • A: Often yes. SSH avoids HTTP proxies and HTTP/2. Try: git@host:org/repo.git.

• Q: How do I prove it’s the proxy?

  • A: Succeeding with NO_PROXY or outside the network, or with HTTP/1.1 and fewer requests, strongly implicates intermediaries.