Hola folks,
I’m looking for folks who are running macOS and have, or are about to, update to macOS 26.7 (still Tahoe) to run a test. Here’s what happened:
- I have several Crystal apps that use HTTP to talk to local servers
- The local servers are accessible via
localhostand127.0.0.1 - Until two days ago, all my apps worked fine talking to
localhostas host. - After the macOS point update, the same apps, not re-compiled, shipped binaries, all failed with a timeout error.
- Switching to
127.0.0.1they worked; but no longer withlocalhost. - But
curlworked fine with both.
Intriguing, I thought! I pulled out my dweeb hat and empty pipe and leaned my unused violin nearby and sat down to some sleuthing. ;-)
Addrinfo.resolvereturns two entries forlocalhost: an v6 address::1and v4 address127.0.0.1TCPSocket.newgets these two and then callsSocket#connectwhich goes through them and is supposed to go past a failing::1and land on127.0.0.1, which it did in fact do before my macOS update.- However now it “thinks” it succeeded and returns a socket that then fails immediately when the
HTTP::Clienttries to write to it.
I’m the first person to line up and say “it must be me, can’t be Apple, seems too basic.” (I call this the “Oracle problem”, back in the day when fellow devs ran into SQL problems they blamed it on Oracle, but it was never Oracle.)
With help from Claude Opus I learned about how curl does something called “happy eyeballs” or RFC 8305. That alone was worth running this issue. But this still didn’t feel right. After some more research I was able to write a test program in Crystal that tries to extract all the different error codes and confirm / deny the issue.
Also, since this was affecting shipping software I needed to ship a fix, so I worked out a monkey-patched shard that I was able to use in my projects and ship updates for others using my apps. I wasn’t sure if it affected everyone but I couldn’t take a chance.
I’ll paste the test program below, I can’t attach .cr file. It turns out it’s not just me. A colleague hadn’t updated yet, so he ran the test before and after and boom. So I am now looking for more data. Anyone who is willing to run the program, instructions are in the comment of the program, would be greatly appreciated. 
If you have the same problem you will get the following verdict:
== Verdict
FAIL closed port raises
FAIL localhost falls back
FAIL 2nd connect reports refusal
AFFECTED: Crystal does not report refused connections on this system.
If you don’t have the problem you will get this:
== Verdict
ok closed port raises
ok localhost falls back
ok 2nd connect reports refusal
NOT AFFECTED
Here’s the test script to save and run.
# Checks whether Crystal's Socket#connect misses refused connections, as seen
# on macOS 26.7. Standalone: does not use socket_connect_fix and needs no
# running server.
#
# Usage:
# crystal run connect_refusal_check.cr
# crystal run -Devloop=libevent connect_refusal_check.cr
#
# Check 1 decides the verdict. Check 2 shows the user-visible symptom and needs
# localhost to resolve another address before 127.0.0.1. Check 3 shows what the
# kernel returns, independent of Crystal. Exits 1 if affected, 0 if not, 2 if
# check 1 could not run.
require "socket"
{% if flag?(:darwin) || flag?(:bsd) %}
SO_ERROR = 0x1007
{% elsif flag?(:linux) %}
SO_ERROR = 4
{% else %}
{% raise "SO_ERROR constant unknown for this platform" %}
{% end %}
# Returns trimmed stdout of *cmd*, or "n/a" if it cannot run.
def command_output(cmd : String, args : Array(String)) : String
io = IO::Memory.new
status = Process.run(cmd, args, output: io, error: Process::Redirect::Close)
status.success? ? io.to_s.strip.gsub('\n', ' ') : "n/a"
rescue
"n/a"
end
# Returns the kernel's pending error on *fd* and clears it.
def pending_error(fd) : Int32
value = 0
len = LibC::SocklenT.new(sizeof(Int32))
LibC.getsockopt(fd, LibC::SOL_SOCKET, SO_ERROR, pointerof(value), pointerof(len))
value
end
# Returns a port on *host* that nothing is listening on.
def closed_port(host : String) : Int32
server = TCPServer.new(host, 0)
port = server.local_address.port
server.close
port
end
def errno_text(n : Int32) : String
n == 0 ? "0" : "#{n} #{Errno.new(n)}"
end
results = {} of String => Bool?
puts "== Environment"
puts " macOS #{command_output("sw_vers", ["-productVersion"])} (#{command_output("sw_vers", ["-buildVersion"])})"
puts " kernel #{command_output("uname", ["-sr"])}"
puts " arch #{command_output("uname", ["-m"])}"
puts " crystal #{Crystal::VERSION}"
puts " evloop #{Crystal::EventLoop.backend_class}"
puts "\n== Check 1: Socket#connect to a closed 127.0.0.1 port"
begin
port = closed_port("127.0.0.1")
sock = TCPSocket.new(Socket::Family::INET)
begin
sock.connect(Socket::IPAddress.new("127.0.0.1", port))
err = pending_error(sock.fd)
puts " connect returned success; SO_ERROR=#{errno_text(err)}"
results["closed port raises"] = false
rescue ex : Socket::ConnectError
puts " connect raised #{ex.class}: #{ex.message}"
results["closed port raises"] = true
ensure
sock.close
end
rescue ex
puts " could not run: #{ex.class}: #{ex.message}"
results["closed port raises"] = nil
end
puts "\n== Check 2: TCPSocket.new(\"localhost\") with an IPv4-only server"
begin
order = Socket::Addrinfo.tcp("localhost", 80).map(&.ip_address.address)
puts " localhost resolves to #{order.join(", ")}"
if order.first? == "127.0.0.1" || !order.includes?("127.0.0.1")
puts " skipped: needs another address before 127.0.0.1"
results["localhost falls back"] = nil
else
server = TCPServer.new("127.0.0.1", 0)
begin
client = TCPSocket.new("localhost", server.local_address.port, connect_timeout: 3)
begin
peer = client.remote_address.address
puts " connected to #{peer}"
results["localhost falls back"] = true
rescue ex
puts " returned a socket that fails on use: #{ex.class}: #{ex.message}"
results["localhost falls back"] = false
ensure
client.close
end
rescue ex
puts " raised #{ex.class}: #{ex.message}"
results["localhost falls back"] = false
ensure
server.close
end
end
rescue ex
puts " could not run: #{ex.class}: #{ex.message}"
results["localhost falls back"] = nil
end
puts "\n== Check 3: raw libc non-blocking connect trace (no Crystal event loop)"
begin
port = closed_port("127.0.0.1")
addr = Socket::IPAddress.new("127.0.0.1", port)
fd = LibC.socket(LibC::AF_INET, LibC::SOCK_STREAM, 0)
raise "socket() failed: #{Errno.value}" if fd == -1
LibC.fcntl(fd, LibC::F_SETFL, LibC.fcntl(fd, LibC::F_GETFL, 0) | LibC::O_NONBLOCK)
ret1 = LibC.connect(fd, addr.to_unsafe, addr.size)
e1 = ret1 == -1 ? Errno.value.value : 0
sleep 100.milliseconds
ret2 = LibC.connect(fd, addr.to_unsafe, addr.size)
e2 = ret2 == -1 ? Errno.value.value : 0
so = pending_error(fd)
LibC.close(fd)
puts " 1st connect() = #{ret1}, errno #{errno_text(e1)}"
puts " 2nd connect() = #{ret2}, errno #{errno_text(e2)}"
puts " SO_ERROR after = #{errno_text(so)}"
results["2nd connect reports refusal"] = ret1 == -1 && e1 == Errno::ECONNREFUSED.value || e2 == Errno::ECONNREFUSED.value
rescue ex
puts " could not run: #{ex.class}: #{ex.message}"
results["2nd connect reports refusal"] = nil
end
puts "\n== Verdict"
results.each do |name, ok|
mark = ok.nil? ? "SKIP" : (ok ? "ok " : "FAIL")
puts " #{mark} #{name}"
end
case results["closed port raises"]
when false
puts " AFFECTED: Crystal does not report refused connections on this system."
exit 1
when true
puts " NOT AFFECTED"
exit 0
else
puts " INCONCLUSIVE: check 1 could not run."
exit 2
end
Cheers.
PS. The GitHub repo with the workaround/fix shard is here: GitHub - nogginly/socket_connect_fix.cr: Monkey-patched workaround for when Crystal's Socket#connect reports a refused connection as successful. · GitHub