### Go version
go version go1.21.5 darwin/amd64
### Output of `go env` in …your module/workspace:
```shell
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/Users/cavok/Library/Caches/go-build'
GOENV='/Users/cavok/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/cavok/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/cavok/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/Cellar/go/1.21.6/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/Cellar/go/1.21.6/libexec/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.21.6'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/9y/hlpdgn0s10s5c3_k60jc0d5c0000gn/T/go-build4292255947=/tmp/go-build -gno-record-gcc-switches -fno-common'
```
### What did you do?
I encountered this issue on macOS while importing two Python extensions written in Go using [Pygolo](https://gitlab.com/pygolo/py). I could reduce the problem to the following repro, completely removing Python from the picture.
This is a minimal shared library exporting a dummy function:
```go
package main
import "C"
func main() {
}
//export fun
func fun() {
}
```
This test C program loads two libraries built from the above source code and invokes the exported `fun` function of each.
```C
#include <assert.h>
#include <stdio.h>
#include <dlfcn.h>
typedef void (*fun)(void);
int main(int argc, char* argv[])
{
void *lib1 = dlopen("./lib1.so", RTLD_NOW);
if (!lib1) {
printf("%s\n", dlerror());
}
assert(lib1);
void *lib2 = dlopen("./lib2.so", RTLD_NOW);
if (!lib2) {
printf("%s\n", dlerror());
}
assert(lib2);
fun fun1 = dlsym(lib1, "fun");
assert(fun1);
fun1();
fun fun2 = dlsym(lib2, "fun");
assert(fun2);
fun2();
dlclose(lib1);
dlclose(lib2);
return 0;
}
```
This Makefile builds the two libraries and the test program, invokes the test multiple times. In an handful of attempts the runtime explodes.
```makefile
GO ?= go
LIBS := lib1.so lib2.so
ITERATIONS ?= 1000
all: $(LIBS) test
for n in `seq $(ITERATIONS)`; do ./test || exit 1; printf .; done; echo ok
test: export LDFLAGS := -ldl
%.so: lib.go FORCE
$(GO) build -buildmode=c-shared -o $@ $<
clean:
rm -rf test $(LIBS) $(LIBS:.so=.h)
FORCE:
.PHONY: FORCE
```
### What did you see happen?
After a few executions of the test program, the Go runtime panics. For example:
```
fatal error: bad sweepgen in refill
goroutine 17 [running, locked to thread]:
runtime.throw({0x1505830c6?, 0x0?})
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/panic.go:1077 +0x5c fp=0xc00006ec10 sp=0xc00006ebe0 pc=0x15055341c
runtime.(*mcache).refill(0x1096e6a68, 0x0?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mcache.go:157 +0x20d fp=0xc00006ec50 sp=0xc00006ec10 pc=0x150537b6d
runtime.(*mcache).nextFree(0x1096e6a68, 0x10)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/malloc.go:929 +0x85 fp=0xc00006ec98 sp=0xc00006ec50 pc=0x150531645
runtime.mallocgc(0x58, 0x1505a6ec0, 0x1)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/malloc.go:1116 +0x448 fp=0xc00006ed00 sp=0xc00006ec98 pc=0x150531c08
runtime.newobject(0x0?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/malloc.go:1328 +0x25 fp=0xc00006ed28 sp=0xc00006ed00 pc=0x150532145
runtime.acquireSudog()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:437 +0x229 fp=0xc00006ed90 sp=0xc00006ed28 pc=0x150556489
runtime.chanrecv(0x1c0000a0000, 0x0, 0x1)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/chan.go:563 +0x225 fp=0xc00006ee08 sp=0xc00006ed90 pc=0x15052bc25
runtime.chanrecv1(0x10984e950?, 0x0?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/chan.go:442 +0x12 fp=0xc00006ee30 sp=0xc00006ee08 pc=0x15052b9f2
runtime.cgocallbackg1(0x150580620, 0xc00006efe0?, 0x0)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/cgocall.go:306 +0x214 fp=0xc00006ef00 sp=0xc00006ee30 pc=0x15052a7f4
runtime.cgocallbackg(0x0?, 0x0?, 0x0?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/cgocall.go:245 +0x109 fp=0xc00006ef90 sp=0xc00006ef00 pc=0x15052a549
runtime.cgocallbackg(0x150580620, 0x7ff7b685ff50, 0x0)
<autogenerated>:1 +0x29 fp=0xc00006efb8 sp=0xc00006ef90 pc=0x15057e4e9
runtime.cgocallback(0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/asm_amd64.s:1035 +0xcc fp=0xc00006efe0 sp=0xc00006efb8 pc=0x15057bfcc
runtime: g 17: unexpected return pc for runtime.cgocallback called from 0x1098211e1
stack: frame={sp:0xc00006efb8, fp:0xc00006efe0} stack=[0xc00006e000,0xc00006f000)
0x000000c00006eeb8: 0x000000015057a46e <runtime.exitsyscall+0x000000000000012e> 0x000000c000006680
0x000000c00006eec8: 0x0000000200000003 0x000000c000006680
0x000000c00006eed8: 0x0000000000000000 0x0000000000000000
0x000000c00006eee8: 0x00000001505a99e8 0x000000c00006ef80
0x000000c00006eef8: 0x000000015052a549 <runtime.cgocallbackg+0x0000000000000109> 0x0000000150580620 <_cgoexp_47f08e3a3bbd_fun+0x0000000000000000>
0x000000c00006ef08: 0x000000c00006efe0 0x0000000000000000
0x000000c00006ef18: 0x00000001098211e1 0x0000000000000000
0x000000c00006ef28: 0x0000000000000000 0x0000000000000000
0x000000c00006ef38: 0x0000000000000000 0x0000000000000000
0x000000c00006ef48: 0x0000000000000000 0x0000000000000000
0x000000c00006ef58: 0x000000c00006efe0 0x000000c000006680
0x000000c00006ef68: 0x000000c000060000 0x0000000150580620 <_cgoexp_47f08e3a3bbd_fun+0x0000000000000000>
0x000000c00006ef78: 0x00007ff7b685ff50 0x000000c00006efa8
0x000000c00006ef88: 0x000000015057e4e9 <runtime.cgocallbackg+0x0000000000000029> 0x0000000000000000
0x000000c00006ef98: 0x0000000000000000 0x0000000000000000
0x000000c00006efa8: 0x00007ff7b685fef0 0x000000015057bfcc <runtime.cgocallback+0x00000000000000cc>
0x000000c00006efb8: <0x0000000150580620 <_cgoexp_47f08e3a3bbd_fun+0x0000000000000000> 0x00007ff7b685ff50
0x000000c00006efc8: 0x0000000000000000 0x0000000000000000
0x000000c00006efd8: 0x00000001098211e1 >0x0000000000000000
0x000000c00006efe8: 0x0000000000000000 0x0000000000000000
0x000000c00006eff8: 0x0000000000000000
goroutine 1 [runnable, locked to thread]:
runtime.gcTrigger.test({0x0?, 0x0?, 0x0?})
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mgc.go:569 +0xdc fp=0x1c00006ac70 sp=0x1c00006ac68 pc=0x15053947c
runtime.mallocgc(0x38, 0x15059f6e0, 0x1)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/malloc.go:1245 +0x75d fp=0x1c00006acd8 sp=0x1c00006ac70 pc=0x150531f1d
runtime.newobject(0x1c00005a590?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/malloc.go:1328 +0x25 fp=0x1c00006ad00 sp=0x1c00006acd8 pc=0x150532145
syscall.nametomib({0x150582afe, 0x14})
/usr/local/Cellar/go/1.21.6/libexec/src/syscall/syscall_darwin.go:50 +0x28 fp=0x1c00006ad60 sp=0x1c00006ad00 pc=0x15057fbe8
syscall.SysctlUint32({0x150582afe?, 0x1c00005a5f0?})
/usr/local/Cellar/go/1.21.6/libexec/src/syscall/syscall_bsd.go:465 +0x1c fp=0x1c00006adb8 sp=0x1c00006ad60 pc=0x15057fb1c
syscall.adjustFileLimit(0x1c00006adf0)
/usr/local/Cellar/go/1.21.6/libexec/src/syscall/rlimit_darwin.go:13 +0x25 fp=0x1c00006add8 sp=0x1c00006adb8 pc=0x15057fa05
syscall.init.0()
/usr/local/Cellar/go/1.21.6/libexec/src/syscall/rlimit.go:37 +0x73 fp=0x1c00006ae10 sp=0x1c00006add8 pc=0x15057f9b3
runtime.doInit1(0x1505f8ee0)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:6740 +0xd8 fp=0x1c00006af40 sp=0x1c00006ae10 pc=0x150562a38
runtime.doInit(...)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:6707
runtime.main()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:249 +0x374 fp=0x1c00006afe0 sp=0x1c00006af40 pc=0x150555e54
runtime.goexit()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/asm_amd64.s:1650 +0x1 fp=0x1c00006afe8 sp=0x1c00006afe0 pc=0x15057c1e1
goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:398 +0xce fp=0x1c00005afa8 sp=0x1c00005af88 pc=0x1505561ee
runtime.goparkunlock(...)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:404
runtime.forcegchelper()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:322 +0xb3 fp=0x1c00005afe0 sp=0x1c00005afa8 pc=0x150556073
runtime.goexit()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/asm_amd64.s:1650 +0x1 fp=0x1c00005afe8 sp=0x1c00005afe0 pc=0x15057c1e1
created by runtime.init.6 in goroutine 1
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:310 +0x1a
goroutine 3 [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:398 +0xce fp=0x1c00005b778 sp=0x1c00005b758 pc=0x1505561ee
runtime.goparkunlock(...)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:404
runtime.bgsweep(0x0?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mgcsweep.go:280 +0x94 fp=0x1c00005b7c8 sp=0x1c00005b778 pc=0x150543f34
runtime.gcenable.func1()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mgc.go:200 +0x25 fp=0x1c00005b7e0 sp=0x1c00005b7c8 pc=0x1505392c5
runtime.goexit()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/asm_amd64.s:1650 +0x1 fp=0x1c00005b7e8 sp=0x1c00005b7e0 pc=0x15057c1e1
created by runtime.gcenable in goroutine 1
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mgc.go:200 +0x66
goroutine 4 [GC scavenge wait]:
runtime.gopark(0x1c00007c000?, 0x1505988a8?, 0x1?, 0x0?, 0x1c000007520?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:398 +0xce fp=0x1c00005bf70 sp=0x1c00005bf50 pc=0x1505561ee
runtime.goparkunlock(...)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/proc.go:404
runtime.(*scavengerState).park(0x1505fa360)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mgcscavenge.go:425 +0x49 fp=0x1c00005bfa0 sp=0x1c00005bf70 pc=0x1505417e9
runtime.bgscavenge(0x0?)
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mgcscavenge.go:653 +0x3c fp=0x1c00005bfc8 sp=0x1c00005bfa0 pc=0x150541d7c
runtime.gcenable.func2()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mgc.go:201 +0x25 fp=0x1c00005bfe0 sp=0x1c00005bfc8 pc=0x150539265
runtime.goexit()
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/asm_amd64.s:1650 +0x1 fp=0x1c00005bfe8 sp=0x1c00005bfe0 pc=0x15057c1e1
created by runtime.gcenable in goroutine 1
/usr/local/Cellar/go/1.21.6/libexec/src/runtime/mgc.go:201 +0xa5
```
### What did you expect to see?
I expect to see no panics. Given the repro, I expect a simple `.` for each execution of the test program.
### Various Go versions
Used [goenv](https://github.com/go-nv/goenv) to try different versions on the same system (macOS 14.2.1) starting from 1.10.8.
1.10.8: fails to build, no messages
1.11.13, 1.12.17, 1.13.15, 1.14.15, 1.15.15, 1.16.15: build of the shared library fails with `combining dwarf failed: Unknown load command 0x80000034 (2147483700)`
1.16.15, 1.17.13, 1.18.10, 1.19.13, and 1.20.12: tested 1000 iterations, no panics
1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6: fail after a few iterations with `fatal error: bad sweepgen in refill` as above.
1.22-8db131082d: same as 1.21.x.
On Debian unstable, 1.21.5 works without issues. Did not try all the versions above but the test seems to pass on various distributions like Ubuntu, Debian, Alpine, RHEL, SLE. See https://gitlab.com/pygolo/py/-/blob/main/docs/TEST-MATRIX.md.
I will complete the map of success/failures at [go-multi-c-shared](https://gitlab.com/pygolo/go-multi-c-shared).