I would like to write ML/AI apps natively in crystal.
I was wondering if there are any plans to add support for half-precision fp16 data types in crystal?
What about fp8 or brain float bf16/bf8 types?
many thanks,
-john
None so far.
There would need a thorough RFC to understand the feature, and how it would impact Crystal in general. For example, just on the codegen side, ignoring the whole impact on the runtime (stdlib):
LLVM supports IEEE754 half-precision floating points (f16). When the f16c CPU feature is enabled on x86_64, it will do the f16 ↔ f32 conversions in hardware. Now, hardware math on f16 is barely available (only the latest Intel Xeon supports avx512fp16), so math operations will always convert to f32, do the operation on f32, then convert back to f16.
LLVM doesn’t seem to support any 8-bit floating points. Unlike half-precision, there’s no IEEE754 standard, and there appears to be multiple formats in the wild.
Thanks for the helpful & clarifying response.
-john
16-bit bfloat is also supported in LLVM and it is apparently available on Apple M2+, however all the interesting parts probably require explicit SIMD in Crystal.
2 Likes
On the ARM front:
ARMv8.2-A introduces hardware conversions for IEEE754 half-precision conversions + optional math operations (FEAT_FP16) and optional support for conversions, multiply and matrix bfloat16 (FEAT_BF16). Weirdly FEAT_BF16 has become mandatory in ARMv8.6 but not FEAT_FP16 
1 Like