Java-style Object field

Hello. I’m trying to implement a simple AST interpreter in Crystal. I need a Token class. My current attempt at writing it looks like this:

class Token
  getter type : TokenType
  getter literal
  getter lexeme : String
  getter line : Int32

  def initialize(@type, @literal, @lexeme, @line)
  end
end

The problem is that I don’t know what type I should use for the literal field. In theory, it should be possible to keep values of different types in it. In Java, there’s the Object class that can be used as an universal wrapper for instances of any classes. But in Crystal, Object works in a different way (if I understood it correctly).

So, what should I use instead?

Generics would be good for this.

https://crystal-lang.org/reference/syntax_and_semantics/generics.html

If all the possible tokens are known, you could also use an enum. See https://crystal-lang.org/api/master/JSON/Token/Kind.html

1 Like

It’s common for tokens to contain properties of each type they can represent. For example:

class Token
  enum Type
    Bool
    String
    Int
    Float
  end

  property type : Type
  property string_value : String = ""
  property int_value : Int::Primitive = 0
  property float_value : Float::Primitive = 0.0
  property bool_value : Bool

  def value : Value
    case type
    when .float?
      float_value
    when .int?
      int_value
    when .bool?
      bool_value
    when .string?
      string_value
    end
  end
end

alias Value =
  Bool |
  Int::Primitive |
  Float::Primitive |
  String
2 Likes

As the others here outlined, you basically need to list out all the possible types. Given literals, these shouldn’t be too many. The most simple solution could be an alias like alias Literal = Int|Float|String|Bool

1 Like

Why don’t you just use the class hierarchy directly?

abstract class Token
  getter line : Int32
  getter lexeme : String
  def initialize(@lexeme, @line); end
end

class Token::Number < Token
  getter literal : Int32
  def initialize(@literal, *args)
    super *args
  end
end

class Token::String < Token
  getter literal : String
  def initialize(@literal, *args)
    super *args
  end
end

In general: It would be great if Crystal’s Object type (or at least Reference) could be used for instance variables. Right now it doesn’t, but it might some day. You could hack your way around it by including a module into Object and using the module instead. But that stinks and breaks easily.

For your specific example, I second @jhass that you can probably just use a union of a few types that could be used to represent literal values. There should just be a hand full.

Using subclasses as suggested by @stronny looks way more complex than necessary. An easier alternative would be a generic type Token(T), but even that is probably too much and not necessary.