v0.36.0

Note

This release was brought to you by the Shipyard team.

Overview

This release brings major improvements to the Sweep provider system: detailed runtime statistics, persistence of reprovide state across restarts, and better connection handling. These features ship to Kubo users in v0.39.

Highlights

Sweep provider statistics API

The SweepingProvider now exposes detailed runtime statistics through a new Stats() method, enabling monitoring tools to track provider health and throughput.

Available metrics include:

• Queue sizes and worker utilization
• Reprovide schedule and progress
• Network statistics (connected peers, regions)
• Operation rates and success counts

This powers the ipfs provide stat command in Kubo v0.39+. See #1144.

Reprovide cycle persistence and resume

The Sweep provider now persists its state to the datastore and automatically resumes after restarts:

• Persistent progress: The provider saves its position in the reprovide cycle. On restart, it continues from where it stopped instead of starting over.
• Catch-up reproviding: If the node was offline, all CIDs overdue for reprovide are immediately queued.
• Persistent provide queue: Pending provide operations survive restarts.

This fixes the long-standing issue where restarts would reset the reprovide cycle. See #1167, #1170, #1176,
#1193.

Connectivity callbacks

Users can now register callbacks to be notified when the provider's connectivity status changes. This enables applications to track and react to connectivity state transitions. See #1194.

Connection protection during provides

New option to protect libp2p connections and keep addresses in the peerstore during provide operations. This prevents the connection manager from pruning connections that are actively being used for provides. See #1172.

Metric rename: provider_provides_total

The Sweep provider metric has been renamed from total_provide_count_total to provider_provides_total to follow OpenTelemetry naming conventions and match other kad-dht metrics.

Migration: Update any Prometheus queries or dashboards using the old metric name. See #1195.

What's Changed

• chore(deps): bump github.com/quic-go/quic-go from 0.54.0 to 0.54.1 by @dependabot[bot] in #1168
• Update go-dsqueue to version that does not reuse batch by @gammazero in #1169
• provider: stats by @guillaumemichel in #1144
• feat(provider): persist provide queue by @guillaumemichel in #1167
• provider: custom logger name by @guillaumemichel in #1173
• feat(provider): resume reprovides by @guillaumemichel in #1170
• provider: protect connections by @guillaumemichel in #1172
• feat(provider): exit early region exploration if no new peers discovered by @guillaumemichel in #1174
• tests: fix flaky TestStateTransitions by @guillaumemichel in #1177
• tests: fix flaky TestFindPeerWithQueryFilter by @guillaumemichel in #1178
• tests: fix flaky TestProvidesExpire by @guillaumemichel in #1179
• fix(provider): resume cycle by @guillaumemichel in #1176
• provider: trigger connectivity check when missing libp2p addresses by @guillaumemichel in #1180
• refactor(provider/stats): use int64 to avoid overflows by @guillaumemichel in #1182
• refactor(provider): use adaptive deadline for CycleStats cleanup by @guillaumemichel in #1183
• feat(provider): skip bootstrap reprovide by @guillaumemichel in #1186
• fix(provider): RegionsFromPeers may return multiple regions by @guillaumemichel in #1185
• refactor(provider): switch to bit256.NewKeyFromArray by @guillaumemichel in #1188
• Remove go-libp2p-maintainers from codeowners by @MarcoPolo in #1192
• refactor(keystore): track size by @guillaumemichel in #1181
• refactor(provider): optimize memory when allocating keys to peers by @guillaumemichel in #1187
• feat(provider): trie iterators by @guillaumemichel in #1189
• feat(provider): connectivity callbacks by @guillaumemichel in #1194
• fix(provider): resume cycle from persisted keystore by @guillaumemichel in #1193
• fix(provider): rename metric to follow OpenTelemetry conventions by @lidel in #1195
• fix(provider): remove from trie by pruning prefix by @guillaumemichel in #1198
• fix(provider): conflict resolution by @guillaumemichel in #1199
• fix(ResettableKeystore): race when closing during reset by @guillaumemichel in #1201
• fix(provider): protect SweepingProvider.wg by @guillaumemichel in #1200
• chore(deps): bump golang.org/x/crypto from 0.42.0 to 0.45.0 by @dependabot[bot] in #1203

Full Changelog: v0.35.1...v0.36.0

v0.35.1

Note

This release was brought to you by the Shipyard team.

Provider Memory Optimizations

Memory optimizations for the provider.

What's Changed

• fix(provider): memory usage by @guillaumemichel in #1163
• feat(provider): use Trie.AddMany by @guillaumemichel in #1164

Full Changelog: v0.35.0...v0.35.1

v0.35.0

Note

This release was brought to you by the Shipyard team.

🚀 Introducing the New Provider Package

We're excited to introduce the new provider package - a complete rethinking of content providing works in the DHT, using the Reprovide Sweep technique. Say goodbye to manual reprovide management! The provider package now handles both initial provides and periodic reprovides automatically.

🎯 Why This Matters

This package replaces the boxo provider with a significantly more efficient implementation that addresses key performance bottlenecks:

Intelligent Batching 🧠
Instead of providing records randomly, we now batch provides by keyspace distance. This means fewer DHT lookups, faster provides, and consistent reprovide scheduling.

Smooth Resource Usage 📊
Reprovides are evenly distributed over time rather than creating periodic traffic spikes. Your node's resource consumption is now predictable and steady.

⚠️ Early Access

This is the initial release of the provider package. While fully functional, we're continuing to add features and refine the API in upcoming releases. Expect interface changes as we iterate based on community feedback.

🔜 What's Next

Stay tuned for additional provider features in future releases. Check the provider package documentation for usage examples and current capabilities.

What's Changed

• refactor: apply suggestions in records by @guillaumemichel in #1113
• ci: uci/update-go by @web3-bot in #1134
• chore: remove hardcoded go versions from go-test ci by @guillaumemichel in #1138
• Reprovide Sweep by @guillaumemichel in #1095
• fix(keystore): use new batch after commit by @guillaumemichel in #1154
• provider: default options by @guillaumemichel in #1153
• fix(provider): don't empty mapdatastore keystore on close by @guillaumemichel in #1155
• refactor(provider): closestPeerToPrefix coverage trie by @guillaumemichel in #1156
• refactor: remove provider status command by @guillaumemichel in #1157
• refactor: adjust FIND_NODE response exceptions by @guillaumemichel in #1158
• bump deps by @guillaumemichel in #1161

Full Changelog: v0.34.0...v0.35.0

v0.34.0

What's Changed

• chore(deps): bump github.com/pion/interceptor from 0.1.37 to 0.1.39 by @dependabot[bot] in #1089
• chore: bump boxo by @guillaumemichel in #1091
• refactor(providers): migrate providers/ to records/ by @guillaumemichel in #1094
• fix: move non-error log to warning level by @lidel in #1119
• feat(crawler): configurable protocol messenger by @guillaumemichel in #1128
• chore: update dependencies by @guillaumemichel in #1129
• chore: release v0.34.0 by @guillaumemichel in #1130

Full Changelog: v0.33.1...v0.34.0

v0.33.1

Highlights

• Fix concurrency in fullrt

What's Changed

• fix: fullrt kMapLk unlock by @guillaumemichel in #1085
• fix(fullrt): use correct mutex for reading keyToPeerMap by @guillaumemichel in #1086
• fix(fullrt): mutex cleanup by @guillaumemichel in #1087

Full Changelog: v0.33.0...v0.33.1

This release was brought to you by the Shipyard team.

v0.33.0

What's Changed

• fix/updates to use context passed in New function for context cancellation by @blockchainluffy in #1081

New Contributors

• @blockchainluffy made their first contribution in #1081

Full Changelog: v0.32.0...v0.33.0

v0.32.0

What's Changed

• chore: bump go-log to v2 by @guillaumemichel in #1074
• chore(deps): bump golang.org/x/net from 0.37.0 to 0.38.0 by @dependabot in #1075
• fix: use correct message type attribute in metrics by @diogo464 in #1076
• fix: netsize warning by @guillaumemichel in #1077
• chore: update deps by @guillaumemichel in #1078

New Contributors

• @diogo464 made their first contribution in #1076

Full Changelog: v0.31.0...v0.32.0

This release was brought to you by the Shipyard team.

v0.31.0

What's Changed

• tests: flaky TestSearchValue (dual) by @guillaumemichel in #1060
• fix(crawler): remove peerstore no-op by @guillaumemichel in #1063
• cleanup: fullrt by @guillaumemichel in #1062
• fix: remove deprecated opt package by @guillaumemichel in #1064
• fix: error on no valid provs by @guillaumemichel in #1065
• refactor: use errors.New when no formatting required by @guillaumemichel in #1067
• chore(deps): bump golang.org/x/net from 0.35.0 to 0.36.0 by @dependabot in #1068
• tests: fix flaky TestProvidesExpire by @guillaumemichel in #1069
• ci: uci/copy-templates by @web3-bot in #1071
• query: ip diversity filter by @guillaumemichel in #1070

New Contributors

• @dependabot made their first contribution in #1068

Full Changelog: v0.30.2...v0.31.0

This release was brought to you by the Shipyard team.

v0.30.2

What's Changed

• do not use multiple multi-error packages, pick one by @gammazero in #1058

Full Changelog: v0.30.1...v0.30.2

v0.30.1

What's Changed

• update dependencies by @gammazero in #1056

Full Changelog: v0.30.0...v0.30.1