Making curl-to-crystal tool

Hello!

I made a tool to convert curl commands to crystal http-request code. This project is based on curl-to-ruby, which is based on curl-to-go. I updated curl-to-ruby to Crystal version.

Here is the link of web page.
https://kanezoh.github.io/curl-to-crystal/

And here is the repository. If you like it, plz star my project.

I hope you like and enjoy it!
Also, any bug reports and contributions are welcome!

Thank you!

16 Likes

Nice! If it’s just one request maybe it can be done without instantiating an HTTP::Client, just using HTTP::Client.get. Then you don’t need a URI either.

2 Likes

useless, most of time we need a wrap.

Thank you for your advice!
I made a simple reqest more simple.

input

curl example.com

output

require "http/client"
require "uri"

uri = URI.parse("http://example.com")
response = HTTP::Client.get(uri)

# response.status_code.not_nil!
# response.body.not_nil!

Thank you!

hmmm…

curl-to-go and curl-to-ruby are loved by many programmers, so I think the crystal-version also makes sense.

1 Like

just a simple request, why not just use curl command

Because if you use the standard library it plays nice with the concurrency model. While waiting for a request to be made or finished other fibers can run your code. If you shell out to curl then the entire app blocks.

1 Like

What? That’s not correct. Process.wait is concurrency-aware. In fact, the POSIX implementation is just receive on a channel that transmits the exit signal of the child process.

The main reasons for a native implementation: you obviously get more control, tighter integration with streaming and concurrency APIs. Arguments don’t need to be serialized through the CLI. And it’s less overhead for spawning a new process.

1 Like

Oh, sorry, you are right. I though this was about using libcurl or something like that.

Several reasons depending on what you need/want

  • Have everything in Crystal and in the same process for better error handling, etc…
  • Avoid curl dependency and be able to deploy your application as a single executable (imagine Windows).
  • …

For sure I believe that @pynixwang would post “Useless, why not use Go?” in the blog post announcing Crystal language first time, hehehe

Let people write/create things, what’s useless for one can be very useful for others, telling people that they are doing useless stuff just because it’s not useful for you is the only thing that doesn’t help.

6 Likes

no no no, most of time we need more integration such as add authentication or parameters

curl to cryatal is not enough for this job

because you data is generate from curl, add some stuff is not easy and pretty.

require "http/client"
require "uri"

headers = HTTP::Headers.new
uri = URI.parse("http://example.com/api/login")
client = HTTP::Client.new(uri.host.not_nil!)
headers["Content-Type"] = "application/json"
body = "{\"username\":\"xyz\",\"password\":\"xyz\"}"

response = client.post(uri.request_target, headers: headers, body: body)

sorry, I have a try. it can be integrated more easy. but really just very very simple case.

1 Like

It may just help to get a good starting point.

2 Likes

This is a nice way to learn more about the language, especially if you work with APIs frequently. Thanks!

3 Likes

Hi @pynixwang , others said so, but please bear in mind that it’s better to be nice. For instance, you could say “Nice! But I wonder, how does it compare to simply wrapping the curl command? I fail to see why would I need this project. Thanks!”. After all, you’re talking about somebody’s effort here. No need to be rude. Thanks!

6 Likes

I think I need to mention the significance of this project.

Certainly, if you make a simple http request, using curl command or Crystal code makes no difference.
But if you want to make a web crawler, it is better to use programming language to control the move of the crawler.
Actually, I use curl-to-ruby when I make web crawlers in Ruby as my work.
Using “Copy as Curl” of Chrome’s Developer tool and curl-to-ruby is good to reenact the browsers’s http request as Ruby code.
I think curl-to-crystal also makes sense when making web crawlers in Crystal!

Thanks!

4 Likes

I have this CURL request that I want to know how I could perform it i ncrystal: curl --cacert /tmp/ca.crt -u elastic:changeme https://localhost:9200, I actually I need this because I don’t see how I will pass ca cert alongside the request itself. Unfortunately the curl-to-crystal gave me codes which don’t have a way to send ca certificate, would you give a hand on this? The following are the code given by it:

require "http/client"
require "uri"

headers = HTTP::Headers.new
uri = URI.parse("https://localhost:9200")
client = HTTP::Client.new(uri.host.not_nil!, port: 9200, tls: true)
client.basic_auth("elastic", "changeme")

response = client.get(uri.request_target, headers: headers)

The CA cert isn’t part of the request, but instead of the HTTP connection underlying it since you need to perform the TLS handshake after the TCP connection is established and before any requests are sent along it. So instead of passing tls: true to HTTP::Client.new, you need to pass an OpenSSL::SSL::Context that knows about that cert.

tls = OpenSSL::SSL::Context::Client.new
tls.ca_certificates = "/tmp/ca.crt"
client = HTTP::Client.new(uri.host.not_nil!, port: 9200, tls: tls)

You can see how I implement it in my Elasticsearch client.

5 Likes

Why doesn’t the standard library add http/https/socks proxy implementation?

This worked, and you saved me, thank you so much!