I had not seen LangArena before, so I popped in and … it’s running the crystal code single-threaded, which may explain why Matmul1T has a great score but Matmul4T 8T and 16T do rather horribly so there is some low hanging fruit for optimization there!
Before:
Matmul1T: OK in 5.598s
Matmul4T: OK in 5.809s
Matmul8T: OK in 5.776s
Matmul16T: OK in 5.763s
After:
Matmul1T: OK in 5.637s
Matmul4T: OK in 1.678s
Matmul8T: OK in 1.523s
Matmul16T: OK in 1.737s
I had not seen LangArena before, so I popped in and … it’s running the crystal code single-threaded, which may explain why Matmul1T has a great score but Matmul4T 8T and 16T do rather horribly so there is some low hanging fruit for optimization there!
Yes, that’s what I thought at first.
However, this is probably a benchmark for the initial state of the language. And if that’s the case, it wouldn’t be a fair comparison to enable experimental features only for Crystal. Enabling experimental features in other languages as well would probably lead to further speed improvements.
My assumption was apparently wrong, as the pull request to enable multithreading was merged.
However, since every language probably has experimental optimization techniques, I personally find it difficult to make a fair comparison if only Crystal used them.
Crystal’s expressiveness score is a strong win and a feat that should not be taken lightly. Thanks again @kostya for your huge work on LangArena.
I believe LangArena is good for all mainstream programming languages, not only for Crystal, because we can see with more precision how each language behaves in multiple use cases, to see which one is a better choice according to our needs.
Naturally it also shows how great Crystal is in many of them. IMHO, its combination of efficiency, performance and productivity is almost unmatched.