Upload image failed use HTTP::Client but test with postman work

zw963 · June 19, 2025, 4:54am

Following is my code (I need form_data to upload a file, but for now, test with a source: "https://image_url" is enough.

post "/api/upload" do
    # file_path = params.from_multipart.last["source"].path

    io = IO::Memory.new

    form_data = HTTP::FormData::Builder.new(io)
    form_data.field("key", "my_key")
    form_data.field("source", "https://avatars.githubusercontent.com/u/549126?s=96&v=4")

    response = HTTP::Client.post(
      url: "https://freeimage.host/api/1/upload",
      headers: HTTP::Headers{"Content-Type" => "multipart/form-data"},
      form: io
    )

    pp! response
    if response.success?
      image_url = JSON.parse(response.body).dig("image", "display_url")
      json({image_url: image_url}, HTTP::Status::OK)
    else
      json(HTTP::Status::BAD_REQUEST)
    end
  end

When I upload, I always get error like this:

{\"status_code\":400,\"error\":{\"message\":\"Invalid API v1 key.\",\"code\":100},\"status_txt\":\"Bad Request\"}"

But if i test use a tool like postman, it works (check following screenshot)

Following is the screenshot of API page

I have another question.

Either a image URL or a base64 encoded image string. You can also use FILES[“source”] in your request.

What is the means of FILES["source"] in your request, how to do that use HTTP::FormData::Builder?

thanks

lasso · June 19, 2025, 1:44pm

Docs (HTTP::FormData::Builder - Crystal 1.16.3) says that the finish method must be called to finalize the form data.

Using HTTP::FormData.build() with a block instead of using HTTP::FormData::Builder.new() seems to call finish automatically at the end of the block, so maybe that is a better way.

Blacksmoke16 · June 19, 2025, 2:30pm

Also may need to do like form: io.rewind to ensure it starts reading from the beginning of the IO.

zw963 · June 19, 2025, 5:10pm

Wired, after apply both changes, still not work.

post "/api/upload" do
    # file_path = params.from_multipart.last["source"].path

    io = IO::Memory.new

    HTTP::FormData.build(io) do |form_data|
      form_data.field("key", "my_key")
      form_data.field("source", "https://avatars.githubusercontent.com/u/549126?s=96&v=4")
    end

    response = HTTP::Client.post(
      url: "https://freeimage.host/api/1/upload",
      headers: HTTP::Headers{"Content-Type" => "multipart/form-data"},
      form: io.rewind
    )

    pp! response
    if response.success?
      image_url = JSON.parse(response.body).dig("image", "display_url")
      json({image_url: image_url}, HTTP::Status::OK)
    else
      json(HTTP::Status::BAD_REQUEST)
    end
  end

Following is the response output:

web          | response # => #<HTTP::Client::Response:0x7f56eab4fc30
web          |  @body=
web          |   "{\"status_code\":400,\"error\":{\"message\":\"Invalid API v1 key.\",\"code\":100},\"status_txt\":\"Bad Request\"}",
web          |  @body_io=nil,
web          |
web          |  @cookies=nil,
web          |  @headers=
web          |   HTTP::Headers{"CF-RAY" => "95249ca8a8a70399-HKG",
web          |    "Cache-Control" => "no-cache, must-revalidate",
web          |    "Cf-Cache-Status" => "DYNAMIC",
web          |    "Connection" => "keep-alive",
web          |    "Content-Type" => "application/json; charset=UTF-8",
web          |    "Date" => "Thu, 19 Jun 2025 17:05:34 GMT",
web          |    "Expires" => "Thu, 19 Nov 1981 08:52:00 GMT",
web          |    "Last-Modified" => "Thu, 19 Jun 2025 17:05:34GMT",
web          |    "Nel" =>
web          |     "{\"report_to\":\"cf-nel\",\"success_fraction\":0.0,\"max_age\":604800}",
web          |
web          |    "Pragma" => "no-cache",
web          |    "Report-To" =>
web          |     "{\"group\":\"cf-nel\",\"max_age\":604800,\"endpoints\":[{\"url\":\"https://a.nel.cloudflare.com/report/v4?s=gOaUwEYuOLapkef4w4cfodQzHx9nu%2BDxzWvpV5C1BQhcvyn4WiQhxvzIYjbWvq8a8uVxcU%2BOfYXlmAMH5V4Zc2eRlHs%2BpZ%2FGc82fU0v1Wn%2F189T5am5Fr5iw\"}]}",
web          |
web          |    "Server" => "cloudflare",
web          |    "Set-Cookie" => "PHPSESSID=flotqnvatdlnj5gka8hnp03q7f; Path=/",
web          |    "Transfer-Encoding" => "chunked",
web          |    "X-Frame-Options" => "DENY",
web          |    "alt-svc" => "h3=\":443\"; ma=86400"},
web          |
web          |  @status=HTTP::Status::BAD_REQUEST,
web          |  @status_message="Bad Request",
web          |
web          |  @version="HTTP/1.1">

I try change form: io.rewind to body: io.rewind, no luck.

Any idea? thanks

zw963 · June 19, 2025, 5:20pm

I write a reproducable example.

require "http"
require "json"

io = IO::Memory.new

HTTP::FormData.build(io) do |form_data|
  form_data.field("key", "my_key")
  form_data.field("source", "https://avatars.githubusercontent.com/u/549126?s=96&v=4")
end

response = HTTP::Client.post(
  url: "https://freeimage.host/api/1/upload",
  headers: HTTP::Headers{"Content-Type" => "multipart/form-data"},
  form: io.rewind
)

body = JSON.parse(response.body)

pp! body

{"status_code" => 400,
   "error" => {"message" => "Invalid API v1 key.", "code" => 100},
   "status_txt" => "Bad Request"}

But, if test with postman, it work.

Blacksmoke16 · June 19, 2025, 5:39pm

Okay, I spent some time looking into this and there are a few things going on:

Are mixing application/x-www-form-urlencoded and multipart/form-data content types
In the latter case, not including the boundary in the header

When you pass something to the form property, it internally overrides your Content-Type header to application/x-www-form-urlencoded, yet the body of the request is multipart/form-data. You can reproduce this in Postman too by unchecking the default Content-Type header and adding a custom one with application/x-www-form-urlencoded.

Ultimately the fix is to settle on one or the other:

# application/x-www-form-urlencoded
response = HTTP::Client.post(
  url: "https://freeimage.host/api/1/upload",
  form: {
    "key"    => "6d207e02198a847aa98d0a2a901485a5",
    "source" => "https://avatars.githubusercontent.com/u/549126?s=96&v=4",
  }
)

However, because you said you’re going to be uploading files, will likely want to use multipart/form-data. Which you’d want to handle by just passing it as the body. E.g. body: io.rewind. But there is still one thing missing. The content-type headers needs the boundary for multipart requests. Which you can get from the HTTP::FormData::Builder instance:

io = IO::Memory.new

form_data = HTTP::FormData::Builder.new(io)
form_data.field("key", "6d207e02198a847aa98d0a2a901485a5")
form_data.field("source", "https://avatars.githubusercontent.com/u/549126?s=96&v=4")
form_data.finish

response = HTTP::Client.post(
  url: "https://freeimage.host/api/1/upload",
  headers: HTTP::Headers{"content-type" => form_data.content_type},
  body: io.rewind
)

zw963 · June 19, 2025, 6:07pm

Thanks very much for awesome explain.

Following is my final uploading image file directly version.

require "http"
require "json"

io = IO::Memory.new

file = File.open("rainbow.png")
form_data = HTTP::FormData::Builder.new(io)
form_data.field("key", "my_key")
form_data.file("source", file, HTTP::FormData::FileMetadata.new(filename: "rainbow.png"))
form_data.finish
file.close

response = HTTP::Client.post(
  url: "https://freeimage.host/api/1/upload",
  headers: HTTP::Headers{"content-type" => form_data.content_type},
  body: io.rewind
)

body = JSON.parse(response.body)

pp! body

It works! https://iili.io/FoQ92Xs.png

aiac · June 19, 2025, 6:47pm

Is there any way to avoid coping file data into memory?

zw963 · June 19, 2025, 6:52pm

My understanding , it’s just copying the IO object from file to form_data?

correct me if I am wrong.

aiac · June 19, 2025, 6:56pm

It tell me it would copy …

github.com/crystal-lang/crystal

src/http/formdata/builder.cr

63650d76a


      
          
            # We don't add a crlf before the first boundary if this is the first body part.
            @io << "\r\n" unless @state.start?
            @io << "--" << @boundary
            headers.each do |name, values|
              values.each do |value|
                @io << "\r\n" << name << ": " << value
              end
            end
            @io << "\r\n\r\n"
            IO.copy(io, @io)
          
            @state = State::FIELD
          end
          
          # Finalizes the multipart message, this method must be called before the
          # generated multipart message written to the IO is considered valid.
          def finish : Nil
            case @state
            in .start?
              fail "Cannot finish form-data: no body parts"

zw963 · June 19, 2025, 7:10pm

I guess I misunderstood something. I thought IO.copy not take too much memory, it just copies from one IO to another IO.

aiac · June 19, 2025, 7:17pm

Yeah, but the copy target is IO::Memory.

Maybe BodyType of post method should add a type BodyIOConsumer. then we could obtain client body socket io as copy target rather than IO::Memory.

Eidt

However, this may require the caller to provide an accurate content-length header. Perhaps this is why the standard library does not implement this feature.

Topic		Replies	Views
HTTP post a file Help & Support	2	360	February 1, 2021
Adding helpers to HTTP::Context to handle uploads and params Help & Support	4	419	November 12, 2020
POST request parsing error: could not find boundary in Content-Type Help & Support	5	310	March 30, 2023
IO+ vs IO from a request.body Help & Support	2	438	January 5, 2019
[solved] Cannot handle exception MIME::Multipart::Error Help & Support	7	311	July 27, 2023

Upload image failed use HTTP::Client but test with postman work

Eidt

Related topics