Issue with having to compile all shards when compile the application

Community
1

Hello,

I happened to read this article:

“Incremental compilation for Crystal - Part 3”

This is an interesting problem. Scala (some alternative language for the JVM) also heavily relies on type inference. Types in signatures of public methods in Scala may also not have types declared, but there is no problem in Scala that Scala libraries that are imported also need to be compiled. But Scala is different as being a JVM language that uses a VM that interpretes intermediate byte code (and creates machine code on the fly). Secondly, Java aka the JVM is build on type erasure, that is the type information is thrown away once the intermediate byte code was genereated by the Java or Scala compiler that is interpreted by the VM.

Maybe one solution would be also to have some intermediate code and binding it to the hardware is only done on the first invocation when starting up everything. This is also what the Microsoft languages do (Visual Basic, C#, F#). But I’m a “normal” application developer with little knowledge about compiler construction. Just my 5 cents. Would be so nice this problem in Crystal could be solved.

2

Thanks for bringing this up, but I don’t see any connection that generating intermediate bytecode would help anything for incremental compilation.
The main problem is in the semantics, specifically the features that make Crystal dynamic.
I suppose Scala just has some slight differences in the type system and other things that allows to reuse previously code more easily. Although I’m not sure about the details in Scala. Maybe it’s not even easy, but the difference is that it’s implemented, so it works. I’m sure we’ll find a way to make incremental compilation work in Crystal as well, but we’re just not there yet and the path could be long and difficult.

2 Likes
3

Yes, I believe you are right. Maybe you could ask the Scala people at the technical university in Lausanne (the EPFL) about your poblem and they would provide some ideas how to tackle it.

Martin Odersky, the language creator of Scala, is a computer science professor at EPFL (made his thesis under Niklaus Wirth, the creator of Pascal and Modula-2). He is a “true scientist” and might be willing to share some input free of charge.

He once offered the Go people some help who declined. Can’t really remember what it was about. Maybe the GC.

4

AFAICT Scala can’t reopen classes. It has open classes, but they must define the interface to implement (fully typed), so it’s closer to abstract (again, fully typed). Each class ain’t scattered, so once built the dependency graph can’t change outside of what has been modified.

The main dynamism of Crystal is that we can reopen types anywhere, so require order is significant — think of require order vs include or extend vs super or previous_def. Then macros generate code that can also reopen and redefine anything, anywhere, and even read and modify constants, making require order even more important.

Trying to build a call graph or dependency graph sounds very hard :anxious_face_with_sweat: That doesn’t mean we can’t, we could start with something simple, and abort when we reach an “not yet solved” and restart from scratch… but just noticing the cases sounds complex.

2 Likes
5

What Scala does (AFAIK) is to infer the output type of functions based on the returned elements. But the inputs have to be typed.

This is valid:

def hello(a_string: String) =
  if (scala.util.Random.nextBoolean())
    a_string + " world"
  else
    true

But if we remove the type to a_string it’s a syntax error.