sourcegraph/conc

> Structured concurrency primitives for Go — scoped goroutine ownership, automatic panic propagation, and typed worker pools over the standard library.

GitHub repo · Blog post · License: MIT

Overview

conc is a small Go library from Sourcegraph that wraps the raw go statement, sync.WaitGroup, and hand-rolled worker pools in higher-level types that are harder to misuse¹. Its three stated goals are to make goroutine leaks harder, to handle panics gracefully, and to make concurrent code shorter to read. The core idea is structured concurrency: every goroutine has an owner, and that owner is responsible for waiting on it before going out of scope.

The library is generics-based (it leans heavily on Go 1.18+ type parameters for iter.Map, pool.ResultPool[T], and friends) and organizes its API into a few packages: conc (the WaitGroup), pool (concurrency-limited task runners), iter (concurrent slice map/for-each), stream (ordered concurrent processing), and panics (a reusable panic catcher). It has roughly 10.4k GitHub stars, which makes it one of the more visible Go concurrency helpers, but it remains explicitly pre-1.0 (see Production Notes)².

The defining tension is that conc sits in an awkward layer: it is more opinionated than golang.org/x/sync/errgroup but is not a full actor/task framework. You adopt it for ergonomics and panic safety, not for capabilities you could not otherwise get — everything it does can be written by hand with the standard library, which is exactly the boilerplate it exists to remove.

Getting Started

go get github.com/sourcegraph/conc

import (
	"github.com/sourcegraph/conc/iter"
	"github.com/sourcegraph/conc/pool"
)

// Concurrently map a slice — f receives a *pointer* to each element.
squares := iter.Map(nums, func(n *int) int { return *n * *n })

// A concurrency-limited pool of fallible tasks; Wait aggregates errors.
p := pool.New().WithMaxGoroutines(4).WithErrors()
for _, url := range urls {
	url := url
	p.Go(func() error { return fetch(url) })
}
err := p.Wait()

Architecture / How It Works

The foundational type is conc.WaitGroup. It wraps sync.WaitGroup and a panics.Catcher. Goroutines are spawned with (*WaitGroup).Go(func()), and (*WaitGroup).Wait() blocks until all of them return. Crucially, Wait() re-panics if any child goroutine panicked, decorating the panic value with the child's stacktrace so the origin is not lost¹. A raw go func() that panics crashes the whole process; a conc.WaitGroup turns that into a controlled re-panic at the collection point.

panics.Catcher is the reusable engine underneath: Try(func()) recovers any panic into a RecoveredPanic (value + captured stack) that callers can later Repanic() or convert to an error. Every higher-level construct is built on it.

The pool package layers configuration onto that base via a builder chain. pool.New() returns a plain pool; .WithMaxGoroutines(n) bounds concurrency; .WithErrors() upgrades it to an ErrorPool whose tasks return error; .WithContext(ctx) produces a ContextPool that cancels the context on the first task error (like errgroup); .WithFirstError() keeps only the first error instead of an aggregate; and pool.NewWithResults[T]() yields a ResultPool[T] that collects return values. The permutations (ResultContextPool, ResultErrorPool, etc.) are generated by combining these methods. Goroutines are spawned lazily up to the limit rather than all at once.

iter.Map / iter.ForEach do not spawn one goroutine per element. They partition the slice across a bounded set of workers (default: GOMAXPROCS) and hand each worker a range, which keeps overhead flat for large slices. The map callback signature is func(*T) R — it receives a pointer into the original slice, deliberately, to avoid copying large elements; mutating through that pointer mutates the input.

stream.Stream handles the harder case of processing an ordered stream concurrently while running per-item callbacks in submission order. Each s.Go returns a stream.Callback closure that the library invokes serially in order, so parallel work fans out but side effects stay ordered.

Production Notes

Still pre-1.0, and effectively dormant on releases. The last tagged release is v0.3.0 from February 2023³; the README's own stated target of a 1.0 in March 2023 was never met¹. The main branch still receives occasional commits, but if you pin a dependency you are pinning a three-plus-year-old tagged API. In practice this cuts both ways: the API has not broken because nothing has shipped, but you should not expect active feature work or a stability guarantee. Vendoring or a pinned module version is prudent.

iter.Map's func(*T) signature is the most common footgun. New users routinely write func(x int) and get a compile error, or dereference wrong. The pointer is into the live input slice — writing through it is a data race against any concurrent reader of that slice and mutates your input in place.

Panics only surface if you call Wait(). The safety guarantee is that Wait() re-panics; if a WaitGroup or pool is dropped without Wait() (or its defer wg.Wait() is forgotten), a child panic is silently swallowed and the leak the library exists to prevent happens anyway. Always defer the wait.

Panic decoration has a cost. Every task runs inside a recover plus a debug.Stack() capture on panic. For extremely hot, tiny tasks the wrapper overhead is measurable versus a bare go; conc is aimed at correctness and readability, not at squeezing maximum throughput out of trivial work.

Loop-variable capture. Examples predating Go 1.22 use the elem := elem shadow. On Go 1.22+ the per-iteration loop variable makes that redundant, but older code and the README still show the shadow — harmless, just dated.

Not a drop-in for errgroup. A ContextPool is close, but errgroup is in golang.org/x/sync, is de facto standard, and has no external dependency. Teams with a low tolerance for non-stdlib deps often keep errgroup for error-cancellation and reach for conc only where its panic capture or pools add real value.

When to Use / When Not

Use when:

• You spawn goroutines that can panic in long-running services and want the

panic propagated with a stacktrace instead of crashing the process.

• You want bounded worker pools or concurrent slice map/for-each without

re-writing the channel/WaitGroup boilerplate each time.

• Readability of concurrent code matters to your team and you accept a small

non-stdlib dependency.

Avoid when:

• You need only error-aware fan-out with cancellation — errgroup is stdlib-adjacent and sufficient.
• You require a maintained, 1.0-stable dependency with active releases.
• Your tasks are extremely hot and tiny, where the recover/stack-capture overhead matters.
• You want goroutine reuse / long-lived worker recycling — conc pools spawn fresh goroutines, not a recycled pool.

Alternatives

• golang/sync — the errgroup package; use instead when you just need Go/Wait with first-error and context cancellation and want zero non-stdlib surface.
• panjf2000/ants — a recycling goroutine pool; use when you want to cap and reuse goroutines for sustained high-throughput workloads.
• alitto/pond — worker pool with metrics and dynamic sizing; use when you want observability and resizable pools out of the box.
• destel/rill — composable streaming/pipeline concurrency; use when your problem is a multi-stage pipeline rather than fan-out over a slice.
• samber/lo — the lop sub-package offers simple parallel Map/ForEach; use for one-off parallel helpers without adopting a concurrency model.

History

References

1. ^ conc README and design goals, sourcegraph/conc. https://github.com/sourcegraph/conc
2. ^ Repository metadata (stars, license, activity) via GitHub API, fetched 2026-07. https://github.com/sourcegraph/conc
3. ^ Release tags, sourcegraph/conc — latest v0.3.0, 2023-02-26. https://github.com/sourcegraph/conc/releases

Tags

go, golang, concurrency, goroutines, structured-concurrency, worker-pool, generics, panic-handling, sourcegraph, library