# Download a file

**URL:** <https://forum.crystal-lang.org/t/download-a-file/3610>\
**Category:** Help & Support\
**Created:** [August 13, 2021, 10:16pm UTC](https://forum.crystal-lang.org/t/download-a-file/3610 "2021-08-13T22:16:35Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fulgurance](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/fulgurance/32/1463_2.png) [@Fulgurance](https://forum.crystal-lang.org/u/Fulgurance)\
**Post date:** [August 13, 2021, 10:16pm UTC](https://forum.crystal-lang.org/t/download-a-file/3610/1 "2021-08-13T22:16:35Z")

</div>

Hello, actually I’m writing a package manager for Linux with Crystal.  
I have just one problem, I don’t understand how to download a file with crystal.

For example if I want to download this archive:[https://download.savannah.gnu.org/releases/acl/acl-2.3.1.tar.xz](https://download.savannah.gnu.org/releases/acl/acl-2.3.1.tar.xz)

How I can do ?

---

<div class="post-metadata">

**Author:** ![Blacksmoke16](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/blacksmoke16/32/1241_2.png) [@Blacksmoke16](https://forum.crystal-lang.org/u/Blacksmoke16)\
**Post date:** [August 13, 2021, 10:31pm UTC](https://forum.crystal-lang.org/t/download-a-file/3610/2 "2021-08-13T22:31:29Z")

</div>

I’m assuming you just want to save the file to the file system? If so it would be something like:

```auto
require "http"

HTTP::Client.get "https://download.savannah.gnu.org/releases/acl/acl-2.3.1.tar.xz" do |response|
  File.open("out.tar.xz", "w") do |file|
    IO.copy response.body_io, file
  end
end

```

However, `HTTP::Client` doesn’t support redirects at the moment, see [HTTP::Client: follow redirects · Issue #2721 · crystal-lang/crystal · GitHub](https://github.com/crystal-lang/crystal/issues/2721), so you will probably have to add in some extra logic to handle that case. I.e. check for `response.status.redirection?` and if `true`, look for the `location` header and make another request before copying the contents to the file.

---

<div class="post-metadata">

**Author:** ![Fulgurance](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/fulgurance/32/1463_2.png) [@Fulgurance](https://forum.crystal-lang.org/u/Fulgurance)\
**Post date:** [March 31, 2023, 7:39pm UTC](https://forum.crystal-lang.org/t/download-a-file/3610/3 "2023-03-31T19:39:22Z")

</div>

Hi again, sorry I know it’s a very old post.

So I followed your example, and I tried with your explanation to have the expected result.

```auto
require "http"

HTTP::Client.get("https://www.kernel.org/pub/linux/kernel/v5.x/linux-5.13.12.tar.xz") do |response|

    if response.status.redirection?
        HTTP::Client.get(response.headers["location"]) do |response|

            File.open("linux-5.13.12.tar.xz", "w") do |file|
                IO.copy response.body_io, file
            end

        end
    else
        File.open("linux-5.13.12.tar.xz", "w") do |file|
            IO.copy response.body_io, file
        end
    end

end

```

I am not very good actually with networking 🤐

I am definitely sure this can be improve a lot, but I think I didn’t understand everything.

If I use this code, is it okay or it’s a bit slow ?  
Like that can I handle multiple redirections ?

How can I mesure received and sended data amount ? Because I would like to provide a nice cli output

---

<div class="post-metadata">

**Author:** ![weirdbricks](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/weirdbricks/32/1741_2.png) [@weirdbricks](https://forum.crystal-lang.org/u/weirdbricks)\
**Post date:** [April 1, 2023, 1:25am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/4 "2023-04-01T01:25:15Z")

</div>

I haven’t tried this shard but it looks like it can handle redirects and there’s an example to download a file:

> **[GitHub - mamantoha/crest: HTTP and REST client for Crystal](https://github.com/mamantoha/crest#streaming-responses)**
>
> HTTP and REST client for Crystal. Contribute to mamantoha/crest development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![npn](https://avatars.discourse-cdn.com/v4/letter/n/e68b1a/32.png) [@npn](https://forum.crystal-lang.org/u/npn)\
**Post date:** [April 1, 2023, 10:09am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/5 "2023-04-01T10:09:31Z")

</div>

rewrite your code to

```crystal
require "http/client"

def download_with_redirect(download_link : String, save_path : String = File.basename(download_link), redirect_countdown : Int32 = 1)
  HTTP::Client.get(download_link) do |response|
    if response.status.success?
      File.open(save_path, "w") { |file_io| IO.copy response.body_io, file_io }
    elsif !response.status.redirection?
      raise "error response" # define your error handling here
    elsif redirect_countdown > 0 # follow redirect
      new_link = response.headers["localtion"]
      download_with_redirect(new_link, save_path, redirect_countdown - 1)
    else
      raise "too many redirection"
    end
  end
end

download_with_redirect("https://www.kernel.org/pub/linux/kernel/v5.x/linux-5.13.12.tar.xz", "linux-5.13.12.tar.xz", 5) # 5 is number of redirection attempts

```

and it should handle redirects correctly.

---

<div class="post-metadata">

**Author:** ![Fulgurance](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/fulgurance/32/1463_2.png) [@Fulgurance](https://forum.crystal-lang.org/u/Fulgurance)\
**Post date:** [April 1, 2023, 10:17am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/6 "2023-04-01T10:17:30Z")

</div>

Oh thanks a lot. How can I calculate the received amount data ? How I now how much data I received ?

---

<div class="post-metadata">

**Author:** ![MistressRemilia](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mistressremilia/32/2585_2.png) [@MistressRemilia](https://forum.crystal-lang.org/u/MistressRemilia)\
**Post date:** [April 1, 2023, 10:34am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/7 "2023-04-01T10:34:40Z")

</div>

A simple way is to just use an intermediate buffer, where you read X bytes from the server, then write X bytes to the destination.

Here’s a whole example client that lets you download something over HTTP(S) to a file and handles redirects.

```crystal
require "http/client"

abort "No URL" unless ARGV.size > 0
abort "No output file" unless ARGV.size == 2

# Don't clobber files.
abort "File exists" if File.exists?(ARGV[1])

url = ARGV[0]
downloaded = false

# Loop until we've downloaded something (or raise an exception and die spectacularly).
until downloaded
  # Do a GET to the url
  HTTP::Client.get(url) do |resp|
    # Check for a redirect
    if resp.status.redirection?
      # Get a new URL
      if location = resp.headers["location"]
        url = location
        puts "Redirected to #{url}"
      else
        abort "Got status #{resp.status_code}, but no location was sent"
      end

      break # Get out of the .get
    end

    # Check for status 200
    if resp.status_code == 200
      # We'll use a buffer so that we can print a percentage.
      buf = Bytes.new(65536)
      totalRead = 0
      len = 0

      # If the headers have a Content-Length, then we can provide a
      # percentage.
      if resp.headers["Content-Length"]?
        len = resp.headers["Content-Length"].to_i32
      end

      # Open the output file
      File.open(ARGV[1], "wb") do |file|
        # Start reading the data from the HTTP client response.
        while (pos = resp.body_io.read(buf)) > 0
          file.write(buf[0...pos])

          # Print the progress
          if len > 0
            totalRead += pos
            puts "Downloaded: #{totalRead} / #{len}"
          else
            totalRead += pos
            puts "Downloaded: #{totalRead} / ???"
          end
        end
      end

      downloaded = true
    else
      # Don't handle any others.
      abort "Received HTTP status code #{resp.status_code}"
    end
  end
end

```

---

<div class="post-metadata">

**Author:** ![Fulgurance](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/fulgurance/32/1463_2.png) [@Fulgurance](https://forum.crystal-lang.org/u/Fulgurance)\
**Post date:** [April 1, 2023, 10:41am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/8 "2023-04-01T10:41:00Z")

</div>

Wow nice, thanks a lot.

---

<div class="post-metadata">

**Author:** ![Fulgurance](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/fulgurance/32/1463_2.png) [@Fulgurance](https://forum.crystal-lang.org/u/Fulgurance)\
**Post date:** [April 1, 2023, 11:32am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/10 "2023-04-01T11:32:39Z")

</div>

So I guess in this code the **pos** variable is the amount of downloaded data ?

I would like to print the download data per second

---

<div class="post-metadata">

**Author:** ![npn](https://avatars.discourse-cdn.com/v4/letter/n/e68b1a/32.png) [@npn](https://forum.crystal-lang.org/u/npn)\
**Post date:** [April 1, 2023, 12:04pm UTC](https://forum.crystal-lang.org/t/download-a-file/3610/11 "2023-04-01T12:04:29Z")

</div>

you can use `Time.monotonic` to measure time duration between each write loop  
the rest is just math.

it will be more accurate if you use channel to print the status every 1 seconds, but the code will be more complex.

---

<div class="post-metadata">

**Author:** ![Fulgurance](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/fulgurance/32/1463_2.png) [@Fulgurance](https://forum.crystal-lang.org/u/Fulgurance)\
**Post date:** [April 3, 2023, 8:32am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/12 "2023-04-03T08:32:20Z")

</div>

Thanks but to be honest I prefer to use the standard API

---

<div class="post-metadata">

**Author:** ![MistressRemilia](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/mistressremilia/32/2585_2.png) [@MistressRemilia](https://forum.crystal-lang.org/u/MistressRemilia)\
**Post date:** [April 3, 2023, 9:12am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/13 "2023-04-03T09:12:40Z")

</div>

I added this to one of my programs just last night. Taking the code from before…

```crystal
require "http/client"

abort "No URL" unless ARGV.size > 0
abort "No output file" unless ARGV.size == 2
abort "File exists" if File.exists?(ARGV[1]) # Don't clobber files.

url = ARGV[0]
downloaded = false

# Loop until we've downloaded something (or raise an exception and die spectacularly).
until downloaded
  # Do a GET to the url
  HTTP::Client.get(url) do |resp|
    # Check for a redirect
    if resp.status.redirection?
      # Get a new URL
      url = resp.headers["location"]? || abort "Got status #{resp.status_code}, but no location was sent"
      puts "Redirected to #{url}"
      break # Get out of the .get
    end

    lastSpeedUpdate = Time.monotonic

    # Used to hold the average speed
    avg = 0

    # The number of bytes we've read between updates.
    bytesLastPeriod = 0

    # Check for status 200
    if resp.status_code == 200
      # We'll use a buffer so that we can print some progress.
      buf = Bytes.new(65536)
      totalRead = 0
      len = 0

      # If the headers have a Content-Length, then we can provide a
      # percentage.
      len = resp.headers["Content-Length"].to_i32 if resp.headers["Content-Length"]?

      # Open the output file
      File.open(ARGV[1], "wb") do |file|
        # Start reading the data from the HTTP client response.
        while (pos = resp.body_io.read(buf)) > 0
          # Calculate average speed, one per second.
          lapsed = Time.monotonic - lastSpeedUpdate
          if lapsed.total_seconds >= 1
            div = lapsed.total_nanoseconds / 1_000_000_000
            avg = (bytesLastPeriod / div).to_i32!
            bytesLastPeriod = 0
            lastSpeedUpdate = Time.monotonic
          end

          # Copy from the buffer to the output file, up to pos bytes.
          file.write(buf[0...pos])
          bytesLastPeriod += pos

          # Print the progress
          if len > 0
            totalRead += pos
            puts "Downloaded: #{totalRead} / #{len} #{avg.humanize_bytes}/s"
          else
            totalRead += pos
            puts "Downloaded: #{totalRead} / ??? #{avg.humanize_bytes}/s"
          end
        end
      end

      # Done downloading
      downloaded = true
    else
      # Don't handle any others.
      abort "Received HTTP status code #{resp.status_code}"
    end
  end
end

```

This updates the speed once per second.

---

<div class="post-metadata">

**Author:** ![Fulgurance](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/fulgurance/32/1463_2.png) [@Fulgurance](https://forum.crystal-lang.org/u/Fulgurance)\
**Post date:** [April 3, 2023, 9:17am UTC](https://forum.crystal-lang.org/t/download-a-file/3610/14 "2023-04-03T09:17:32Z")

</div>

It’s exactly what I was looking for ! Thanks a lot my friend !

I will change the appearance, but it’s the base I need 🥰