Possible to serializer/deserialize and object like Ruby marshaling?

#1

Trying to figure out if this is possible. It doesn’t look like it is, but maybe someone knows of a way

0 Likes

#2

Have you checked JSON and YAML modules? Isn’t it what you are looking for?
https://crystal-lang.org/api/0.27.2/JSON.html
https://crystal-lang.org/api/0.27.2/YAML.html

1 Like

#3

Hey @avitkauskas! Those would work but require you to manually set to how serialize/deserialize. I was hoping there was a way to do it automatically to YAML/JSON/some binary format.

Maybe that just doesn’t exist though

0 Likes

#4

I proposed this a while ago: https://github.com/crystal-lang/crystal/issues/6309

(there’s no generic binary format like Marshal, but it could be done with this )

But the community and core team rejected it.

I still think any object should be automatically serializable to any format, like in Java and C# (and you can control this serialization with attributes).

1 Like

#5

I’ll take a look at Cannon. It seems Procs and a few other things are missing, but Procs are super hard to do anyway. It might work out!

0 Likes

#6

How do you serialize procs? :thinking:

1 Like

#7

Is also https://github.com/Blacksmoke16/CrSerializer. Is based on the *::Serializable stuff but with some extra features.

0 Likes

#8

There’s something I’d like to see in Crystal (actually I started it, but with all the other things it’s going quite slow…) it’s a serialization mechanism similar to https://serde.rs where you tell how your object is to be serialized but not with which serialization mechanism, and you can use anything like json, yaml, msgpack,… easily…

And like you say @asterite you can configure some things using attributes, like field ignore, additional fields, conversions, custom serialization (still mechanism agnostic)

2 Likes

#9

Yes, that’s something I’d like to see as well. It’s so much hassle when you have type that should be serializable to different formats.

0 Likes

#10

So in this generic serialization format… how do you say something should be an XML attribute or an XML element?

0 Likes

#11

In this case there would be some specific flags for XML (de-)serializer I guess…

https://github.com/RReverser/serde-xml-rs doesn’t seem to have that, maybe the (de-)serializer is smart for some things? (no time to check right now)

0 Likes

#12

As far as I understand it, serde-xml-rs uses both attributes and child elements to deserialize an object. Serialization seems to not create any attributes by default. See their test suite.

0 Likes

#13

I have experimented a bit, and got this:

require "json"

module JSON
  def self.serialize(data, &block)
    pull = PullParser.new(data)
    pull.read_begin_object
    while pull.kind != :end_object
      key = pull.read_object_key
      yield key, pull
    end
  end

  def self.deserialize(**members)
    String.build do |str|
      JSON.deserialize str, **members
    end
  end

  def self.deserialize(io : IO, **members)
    JSON.build(io) do |json|
      members.build json
    end
  end
end

class String
  def self.from(pull : JSON::PullParser)
    pull.read_string
  end

  def build(builder : JSON::Builder)
    to_json builder
  end
end

struct Int32
  def self.from(pull : JSON::PullParser)
    v = pull.int_value.to_i32
    pull.read_next
    v
  end

  def build(builder : JSON::Builder)
    to_json builder
  end
end

struct NamedTuple
  def build(builder : JSON::Builder)
    to_json builder
  end
end

class Object
  def from(type, data)
    {% for ivar in @type.instance_vars %}
    _{{ivar.id}} = nil
    {% end %}

    {% begin %}
    # Standard JSON/YAML etc iterator
    type.serialize(data) do |key, pull|
    case key
    {% for ivar in @type.instance_vars %}
    when {{ivar.stringify}} then _{{ivar.id}} = {{ivar.type.id}}.from(pull)
    {% end %}
    else raise "unknown key: #{key}"
    end
    end

    {{@type.id}}.new(
      {% for ivar in @type.instance_vars %}\
        {{ivar.id}}: _{{ivar.id}}.as({{ivar.type}}),
      {% end %}\
    )
    {% end %}
  end
  
  macro method_missing(build)
  def build(type)
    type.deserialize(
      {% for ivar in @type.instance_vars %}\
        {{ivar}}: @{{ivar}},
      {% end %}
    )
  end
  end
end

record Point, x : Int32, y : String
data = %({"x": 1, "y": "abc"})

point = Point.from(JSON, data)
puts point #=> Point(@x=1, @y="abc")

puts point.build(JSON) #=> {"x":1,"y":"abc"}
1 Like

#14

The implementation is imperfect, it only exists to show that’s possible. We can then implement custom generic annotations, inspired by Serializable.
If we implement a new way to map JSON/YAML, we have to think how to phase out .mapping and Serializable. It won’t be reasonable to have 3 ways to do the same thing in the stdlib.

0 Likes