I have been experimenting with some concepts from Casey Muratori and just some other thoughts on performance, dependencies, general experimentation.
This has come together in this branch.
This fork as a few things.
- Z
- YAML
- Big Number
- CharCov
- SIMD
Why Pure Crystal Backends?
- Cross-compilation
- Static linking
- WASM
- embedded
- Docker to simplify libs
- just because it is cool
Honestly this is just fun and experimental. I remembered the story of Rubinius back in the day and I don’t even know if it was true but I remember one of the goals being to write parts of the stdlib in ruby and llvm.
Z
The goal was to make it as compliant as possible and make it performant. Mainly because of the c library using SIMD crystal cannot get close to zlib. Which got me to start messing with SIMD in crystal. I know this issue exists but I am just playing around with it.
| Z (pure Crystal) | stdlib/libz | Ratio | |
|---|---|---|---|
| Compress — Deflate (text) | 140 MB/s | 483 MB/s | 0.3x |
| Compress — Gzip (text) | 135 MB/s | 461 MB/s | 0.3x |
| Compress — Deflate (random) | 26 MB/s | 58 MB/s | 0.5x |
| Decompress — Deflate (text) | 2,910 MB/s | 4,749 MB/s | 0.6x |
| Decompress — Gzip (text) | 766 MB/s | 4,547 MB/s | 0.2x |
CharConv : Forum Post
Replaces libiconv with support for 150+ encodings: all ISO-8859-, CP, KOI8-* single-byte encodings, UTF-7/16/32, full CJK (Shift_JIS, EUC-JP, GB2312, GBK, GB18030, Big5, EUC-KR, EUC-TW), and stateful codecs
(ISO-2022-JP/CN/KR, HZ-GB-2312). Supports //IGNORE and //TRANSLIT modes.
| Conversion | charconv | system iconv | Ratio |
|---|---|---|---|
| ASCII → ASCII | 73 µs | 11.89 ms | 162x |
| UTF-8 → ISO-8859-1 | 3.43 ms | 14.62 ms | 4.3x |
| ISO-8859-1 → UTF-8 | 2.08 ms | 14.24 ms | 6.9x |
| UTF-8 → UTF-8 (mixed widths) | 4.92 ms | 11.98 ms | 2.4x |
| CP1252 → UTF-8 | 2.50 ms | 17.24 ms | 6.9x |
| UTF-16BE → UTF-8 | 3.73 ms | 10.83 ms | 2.9x |
| UTF-8 → UTF-16LE | 4.57 ms | 10.11 ms | 2.2x |
| Operation | 1 limb (19 dig) | 10 limbs (190 dig) | 50 limbs (950 dig) | 100 limbs (1.9k dig) | 1000 limbs (19k dig) |
|---|---|---|---|---|---|
| add | 0.77x | 0.98x | 1.12x | 1.09x | 1.18x |
| mul | 0.77x | 1.35x | 2.62x | 2.62x | 3.21x |
| div | 0.98x | 2.54x | 2.36x | 2.62x | |
| to_s | 1.95x | 3.36x | 5.16x | 5.37x | 4.58x |
YAML
| Fixture | Size | libyaml | pure Crystal | Ratio |
|---|---|---|---|---|
| trivial | 11 B | 333k ips | 429k ips | 1.29x faster |
| flat_mapping | 1.7 KB | 12.0k ips | 12.4k ips | 1.03x faster |
| flow_collections | 4.8 KB | 4.7k ips | 5.2k ips | 1.12x faster |
| nested | 32 KB | 1.02k ips | 989 ips | 0.97x |
| block_scalars | 1.6 KB | 28.9k ips | 24.5k ips | 0.85x |
| strings_heavy | 6.9 KB | 8.9k ips | 5.2k ips | 0.59x |
| large_config | 49 KB | 727 ips | 656 ips | 0.90x |
SIMD Vectors (SIMDVector(T, N))
- SIMDVectorType and SIMDVectorInstanceType
- LLVM vector instruction bindings
- Codegen: simd_splat, simd_extract, simd_insert, simd_binary, simd_compare
Finally
I am curious if people would use this, be interested in this, general feedback. I have some ideas for my own web framework and will probably see if I can include any of this in some of the ideas I have.
Thanks for reading this far!