Proposal: SSL Certificate Probing

Crystal Contrib
1

Hey there, Crystal community!

This is my first time posting something here, and before everything else, I would like to thank all the core team members and members of the community for all the work put into building this amazing language!

When it comes to Crystal, I think people tend to gravitate towards web development, but one area that I think the language doesn’t get enough credit for is tooling: it can produce small, performant, and, with very little effort, statically compiled binaries, which are super portable and easy to distribute. This is, of course, until you include something that depends on OpenSSL. Since each system has its own configuration for it, binaries compiled somewhere else usually won’t work out of the box for programs that depend on it.

This issue is something that was discussed before, and with the pointers given in another discussion (thanks, @straight-shoota!) I was able to assemble a little hack to search for certificates in the current system, which allowed me to build binaries that can perform HTTP requests on multiple systems. This was something very useful at the company where I work, and now that we have been using it successfully for almost a year, I feel confident enough to share it here.

We successfully deployed binaries with this code in a variety of environments: Alpines, Ubuntus, OpenSUSEs, Amazon Linuxes, and some others that I can’t remember, so this should be pretty exhaustive:

require "http/client"
require "openssl"

lib LibSSL
  fun X509_get_default_cert_dir : Char*
  fun X509_get_default_cert_file : Char*
end

private CERTS_DIRS = %w[
  /var/ssl
  /usr/share/ssl
  /usr/local/ssl
  /usr/local/openssl
  /usr/local/etc/openssl
  /usr/local/share
  /usr/lib/ssl
  /usr/ssl
  /etc/openssl
  /etc/pki/ca-trust/extracted/pem
  /etc/pki/tls
  /etc/ssl
  /etc/certs
  /opt/etc/ssl
  /data/data/com.termux/files/usr/etc/tls
  /boot/system/data/ssl
]

private CERT_FILENAMES = %w[
  cert.pem
  certs.pem
  ca-bundle.pem
  cacert.pem
  ca-certificates.crt
  certs/ca-certificates.crt
  certs/ca-root-nss.crt
  certs/ca-bundle.crt
  CARootCertificates.pem
  tls-ca-bundle.pem
]

CERTS_LOCATION = begin
  cert_file = ENV["SSL_CERT_FILE"]?
  certs_dir = ENV["SSL_CERT_DIR"]?

  CERTS_DIRS.each do |dir|
    cert_file ||= CERT_FILENAMES
      .map { |file_name| Path[dir, file_name] }
      .find { |path| File.exists?(path) }

    if certs_dir.nil?
      possible_certs_dir = Path[dir, "certs"]
      certs_dir = possible_certs_dir if File.exists?(possible_certs_dir)
    end

    break if cert_file && certs_dir
  end

  {
    ca_bundle_pem: cert_file.to_s || String.new(LibSSL.X509_get_default_cert_dir),
    certs_dir:     certs_dir.to_s || String.new(LibSSL.X509_get_default_cert_file),
  }
end

class HTTP::Client
  # Creates a new HTTP client with the given *host*, *port* and *tls*
  # configurations. If no port is given, the default one will
  # be used depending on the *tls* arguments: 80 for if *tls* is `false`,
  # 443 if *tls* is truthy. If *tls* is `true` a new `OpenSSL::SSL::Context::Client` will
  # be used, else the given one. In any case the active context can be accessed through `tls`.
  def initialize(@host : String, port = nil, tls : TLSContext = nil)
    check_host_only(@host)

    {% if flag?(:without_openssl) %}
      if tls
        raise "HTTP::Client TLS is disabled because `-D without_openssl` was passed at compile time"
      end
      @tls = nil
    {% else %}
      @tls = case tls
             when true
               context = OpenSSL::SSL::Context::Client.new
               context.ca_certificates = CERTS_LOCATION[:ca_bundle_pem]
               context.ca_certificates_path = CERTS_LOCATION[:certs_dir]
               context
             when OpenSSL::SSL::Context::Client
               tls
             when false, nil
               nil
             end
    {% end %}

    @port = (port || (@tls ? 443 : 80)).to_i
  end
end

Now, this clearly involves reopening methods from the stdlib, which isn’t the best approach here. With that said, I would like to ask: would it make sense to include this functionality, maybe behind a compiler flag, in the stdlib itself? And if it doesn’t, is there a better way to write it so it can seamlessly be shared through a shard?

2 Likes
2

Looks like a great tool!
I think for easier integration, the OpenSSL::SSL library could use a global configuration for default ca_certificates and ca_certificates_path values. That would remove the need to re-open HTTP::Server#initialize.
Another improvement would certainly be to extract context creation into a helper method which you would then be able to override in a subclass.

I’m a bit worried about general application though. Unconditionally probing that many different locations could have security implications. If someone manages to install unauthorized certificates in either of these places, it could lead to a compromise.
Maybe I’m a bit overly cautious. It could still be difficult to exploit that. But certainly, many paths are less defensible than a single one.

3

This could be really useful.

It reminds me that, just as a point of interest, the Nim compiler has something very similar built into its standard library:

4

Thanks, that’s very helpful.

Separating the lookup paths per operating system makes a lot of sense and reduces the risk of unauthorized certificate injection.

5

Yeah, I can see why an attack surface increase might turn some people off. The idea of OpenSSL::SSL having a global config would be great because it would still be an opt-in while allowing people to seamlessly integrate it into existing applications. I think this would be huge because it would bring static compiling in crystal to almost a Golang level of simplicity, which is a huge advantage over the vast majority of other compiled languages!

This looks like a great implementation, indeed! I’m always on linux, so I never separated the lookup by OS, but it was something that crossed my mind; should the probing be made available publicly, this is definitely something to aim for.