Should Json::Serializable provide a method to emit a Json::Any?

Crystal Contrib
1

JSON::Serializable and JSON::Any represent different strategies to Json, and I’m not sure what the long term strategy for crystal core is. Serializable represents a way for getting to and from strings, and Any is a kind of non-string “soft serialized” json – it’s not really usable as is except for transmission between libraries.

Crystal pg uses JSON::Any to read and write Postgres jsonb columns, but there is little-to-no ergonomic support for that anywhere, really.

Casting something manually into a JSON::Any is a verbose exercise. In this contrived example I have a database table which has a column that is postgres jsonb typed, which could represent a person. Given that I build that Person object elsewhere and want to insert it into the database, I have to provide a way to turn it into a JSON::Any:

class Person
  include JSON::Serializable

  @[JSON::Field]
  property name : String

  @[JSON::Field]
  property location : String

  @[JSON::Field]
  property favorite_color : String

  def to_json_any : JSON::Any
    JSON::Any.new({
      "name" => JSON::Any.new(name),
      "location" => JSON::Any.new(location),
      "favorite_color" => JSON::Any.new(favorite_color)
    })
  end
end

Is there misalignment of the way newer Crystal std-lib might recommend pg or an ORM do this? Does it make sense to provide a generated to_json_any method in the Serializable module? What about a from_json_any class method?

1 Like
2

The best will be that crystal-pg also supports JSON::Serializable. It is a type, just like JSON::Any.

3

I think it’s important to understand that these two concepts are not interchangeable. JSON::Serializable is a way to (de)serialize a known structure to/from json. While JSON::Any allows processing dynamic JSON. They each have their own uses. I.e. it’s not likely that one will be removed in favor of another.

This makes sense given PG doesn’t know what data you’ll be putting into it. I think there is room for some sort of abstraction to make working with structured data easier. E.x. having some functionality/allowing functionality to be added to allow for like “hey this type is_a? JSON::Serialiazble and this column is a json column, i can just use to/from_json instead given the structure is static/known.”

For example Granite allows doing stuff like this: column my_type : MyType?, column_type: "JSON", converter: Granite::Converters::Json(MyType, JSON::Any). Which tells it that it should use the Json converter to go to/from the database. Which just wraps the to/from_json methods. granite/src/granite/converters.cr at master · amberframework/granite · GitHub

EDIT: Of course It would be more ideal if this concept was baked into the DB abstractions. I.e. so that the data is (de)serialized directly versus needing to do the slightly hacky value.to_json since the DB driver deserializes it into a JSON::Any first if the column was set to JSON::Any.

I would be strongly against adding those methods automatically. It would be pretty trivial to add a def self.new(json : JSON::Any) : self overload if you really needed it. However, given you’re working with known types JSON::Any has little use.

3 Likes
4

Direct integration for JSON::Serializable in db would be nice.

But even without that, you can do better. The trick is to read/write the database value as string and convert with from_json/to_json. There’s no need for JSON::Any as an intermediary.

1 Like
5

@Blacksmoke16 You better stated the difference between Serializable and Any, thank you.

I think the subtlety I’m seeing is that, from the database driver perspective, it is meaningless json so Any is a reasonable type. From my application’s perspective it has meaning so I’m going to have something more concrete representing it.

Adding JSON::Serializable as a type to the pg driver makes the most sense to me as well. Thanks everyone.

6

Sure, you should definitely post a feature request for the database driver.

It’s not going to be easy though. Decoders are statically mapped from postgres’ oids. Not sure how a disambiguation for different Crystal data types could fit into that.

7

Does Json columns are read as JSON::PullParser instead of JSON::Any by matthewmcgarvey · Pull Request #232 · will/crystal-pg · GitHub by chance improve performance of JSON API’s?

8

I implemented something like that here https://github.com/hugopl/tijolo/blob/main/src/from_json_any.cr

The weird thing is that I remember doing some benchmarks and checking that parsing the JSON as JSON::Any and using this constructor was faster than calling from_json.

The use case that made me implement this was to avoid need to parse the JSON twice when I need to fetch some info from the JSON itself to discover what object is serialized there.

1 Like
9

A to_json_any method that works for any non-cyclic aggregate compatible with JSON::Any::Type can be defined as follows:

require "json"

module JSON
  def self.any(x : Array) : Any
    Any.new(x.map { |v| any(v) })
  end
  
  def self.any(x : Hash(String, _)) : Any
    Any.new(x.to_h { |k, v| {k, any(v)} })
  end
  
  def self.any(x : Int) : Any
    Any.new(x.to_i64)
  end
  
  def self.any(x : Float) : Any
    Any.new(x.to_f64)
  end
  
  def self.any(x : Bool | String | Nil) : Any
    Any.new(x)
  end
end

This has nothing to do with JSON::Serializable, but it should be easy to write a version that infers the appropriate raw types from the JSON field specifications.

4 Likes