Have an alternative to JSON.parse that works on dynamic structures, but statically types the values

21

@Blacksmoke16 Yeah, problem is why write all that when you can just do .to_i and JSON.parse then?

  • Now, everything is accessed by dot notation, not brackets. Which I guess isn’t a problem for most, but for me, everything in my codebase is accessed with brackets. I also like bracket access because the keys can be accessed by client input, can’t do that with dot notation (AFAIK). Just feels weird.

  • If there is another amount key in that JSON data, I would have to update JSON.mapping. When I already specified that the key amount should be a Int32. That feels redundant (my map_keys macro idea would fix that)

22

You’re shooting yourself in the foot IMO, but you do you.

EDIT: Also there couldn’t be another amount key in the same object as they would conflict not matter how you did this.

EDIT2: You can also just define a [](key : String) method on Object that will return the value of the ivar with the name of key if you really wanted to keep the bracket notation Carcin

EDIT3: That would cause return type to be Union of all types, probably is a way around that.

23

https://play.crystal-lang.org/#/r/6k7g/edit

See what I mean, and how verbose it will get it? And redundant
How many amount keys does a developer need?!
The developer explicitly specified the amount key should be an Int32

edit: You can imagine, with such a complex data structure, there will be plenty of structs with JSON.mapping all over. My idea (map_keys macro), would allow a developer to just do JSON.parse and specify the keys to their types. And boom, done.

24

How would you handle if one amount key is Int32 but another is Float64? I feel like your could probably do something like what you want for your specific use case, but it wouldn’t work all too well given other scenarios.

EDIT: Inheritance is your friend. https://play.crystal-lang.org/#/r/6k7l Now you only have one amount again.

2 Likes
25

girng is trying to find an easier way to work with dynamic data, but the resistance he get on his way disappoints me about the Crystal community (except vladfaust who proposed a solution).

26

except vladfaust who proposed a solution

what’s that solution?

1 Like
27

@Sorin I don’t really like being accused of what you are implying. I’m just trying to understand how @girng thinks this would work, because personally I don’t see how it could, althought maybe someone will be able to prove me wrong. There isn’t a way I can think of that would allow you to cast dynamic keys into given types without knowing that ahead of time. Even then it would break down if you say all keys named "value" are Int32, but then what happens if there is another key that is a of a different type?

If you’re at the point where you are defining key to type mappings, the structure has to be somewhat known already, so what is the benefit of just explicitly defining the mappings using JSON.mapping or include JSON::Serializable?

1 Like
28

I’m thinking from design point of view.
Let’s say I want to build a big app that is based on micro services.

A very important moment here is to have a standard uncoupled format to exchange data between different parts of app. In this case the structure of a data instance should not be known, because it will became a limitation then app will change during time and evolve. But the protocol to transform data from an instance to an “native object” of chosen language should be well defined. Json is the ideal candidate here because it was designed for such tasks and is “wide adopted”.

Let’s say I need my back-end program to talk to a js script on front end in an easy, predictable and decoupled way. For example if I do a json_decode(string, true) in Php I get an ready to use associative array and I can (recursively) foreach() it getting keys and values. The same easy process is for Js and Python.

For me, right now the lack of this functionality in Crystal is a deal breaker.

29

Those are all dynamic interpreted languages without a strong enforced type system.

Crystal is none of those things.
Take this JSON:

{
  "firstName": "John",
  "lastName": "Smith",
  "isAlive": true,
  "age": 27,
  "address": {
    "streetAddress": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postalCode": "10021-3100"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ],
  "children": [],
  "spouse": null
}

The issue is not that crystal doesn’t know that firstName is a String.
Obviously it is and your programm will know it.
BUT if you use data[“firstName”] somewhere in your program and for example want to remove whitespace from the end.

data["firstName"].chomp

what happens if you feed that programm this JSON?

{
  "firstName": 4711,
  "lastName": "Smith",
  "isAlive": true,
  "age": 27,
  "spouse": null
}

firstName is no longer a string. .chomp nolonger works and you get a runtime error due to a type mismatch.
That is exactly what strongly typed languages will help avoid, and that is also the reason why you need to deal with JSON::Any if you parse unknown JSON and check and do this:

obj = JSON.parse(%({"access": [{"name": "mapping", "speed": "fast"}, {"name": "any", "speed": "slow"}]}))
obj["access"][1]["name"].as_s  # => "any"
obj["access"][1]["speed"].as_s # => "slow"

I hope you now better understand the point of Crystal and other strongly typed languages.

3 Likes
30

Now this is a helpfull comment. Thank you.

To be honest I’m in process of migration to a language (for back-end development) I’ll be happy with in next years. It’s a problem of time investment. In paralel to Crystal I’m learning about other languages as Julia, Nim, Elixir and so on. The constant rise of Python shows that sintax matter. Another point of consideration is speed. (Tweeter Ruby case) So I’m looking at typed and/or compiled languages.
“Json problem” is just one of use cases that might give me an idea about how well a language may help me in my migration.

31

You can so the same thing in Crystal using JSON.parse. You may just need some explicit type enforcements here and there which would be implicit in a dynamic language, but might lead to silent errors in return. So in Crystal you need to write a little bit more but it gives you more safety.

3 Likes
32

Hehe. @Sorin, please don’t view this topic as a deal breaker! Most of my issues are mixed with me still learning the language (and programming concepts in general). The community has gone out of their way to help, and some.