# BitArray size limitation

**URL:** <https://forum.crystal-lang.org/t/bitarray-size-limitation/4778>\
**Category:** Help & Support\
**Created:** [July 14, 2022, 2:41am UTC](https://forum.crystal-lang.org/t/bitarray-size-limitation/4778 "2022-07-14T02:41:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jzakiya](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jzakiya/32/278_2.png) [@jzakiya](https://forum.crystal-lang.org/u/jzakiya)\
**Post date:** [July 14, 2022, 2:41am UTC](https://forum.crystal-lang.org/t/bitarray-size-limitation/4778/1 "2022-07-14T02:41:16Z")

</div>

Currently bit array sizes are limited to `2**31 - 1 = 2147483647`  
I have an application needing \> 2\*\*32 bits, and it would be nice to use `bit_array` for it.

Please consider increasing the max limit, as I just upped my laptop memory to 40GB to handle it.

> <https://github.com/crystal-lang/crystal/blob/994c70b10/src/bit_array.cr#L18>

---

<div class="post-metadata">

**Author:** ![asterite](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/asterite/32/60_2.png) [@asterite](https://forum.crystal-lang.org/u/asterite)\
**Post date:** [July 14, 2022, 12:16pm UTC](https://forum.crystal-lang.org/t/bitarray-size-limitation/4778/2 "2022-07-14T12:16:46Z")

</div>

This has been discussed in the past. The proposed solution is that if you need it for 64 bits you copy the implementation and change `Int32` to `Int64`. You could also write a shard for it. Something like `BitArray64`.

---

<div class="post-metadata">

**Author:** ![jzakiya](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/jzakiya/32/278_2.png) [@jzakiya](https://forum.crystal-lang.org/u/jzakiya)\
**Post date:** [July 15, 2022, 2:55pm UTC](https://forum.crystal-lang.org/t/bitarray-size-limitation/4778/3 "2022-07-15T14:55:24Z")

</div>

Is it possible to create 2 versions in the std library, so that its sufficiently maintained and tested for new Crystal versions?

As hardware memory capacities increase, and as Crystal will eventually accommodate 128-bit values, it seems the better thing to do, to maintain verified compatibility across the board, is to add this feature as part of the std library.

---

<div class="post-metadata">

**Author:** ![RespiteSage](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/respitesage/32/793_2.png) [@RespiteSage](https://forum.crystal-lang.org/u/RespiteSage)\
**Post date:** [July 15, 2022, 4:27pm UTC](https://forum.crystal-lang.org/t/bitarray-size-limitation/4778/4 "2022-07-15T16:27:25Z")

</div>

I strongly disagree with the implication that as hardware memory capabilities increase programming languages ought to assume that programs should use more memory by default. However, I think that whole discussion is tangential to the two main issues here.

First, what you’re requesting is a maintenance nightmare. The core development team and everyone else who contributes are making steady, admirable progress in a number of useful features, and trying to maintain a separate standard library with a different default integer type would be, in my opinion, a whole lot of wasted effort. If you insist on using `BitArray`, it _might_ make sense to request a generic version that allows you to specify the size type. The implementation should be possible, though I expect it would require some slightly wonky macros.

Second, a `BitArray` becomes less and less useful for your prime sieve as the numbers you’re considering grow. Since the ratio of primes to non-primes goes to zero as you consider larger and larger numbers, you’re using less and less of your `BitArray` as the numbers grow. Consequently, you’d be served much better by a sparse implementation (which, crucially, is outside the scope of the standard library). I’m pretty sure I saw an interesting implementation of a sparse bit array/set/vector recently in another language, but for now you should take a look at [this Java implementation of a sparse BitSet](https://github.com/brettwooldridge/SparseBitSet). I think you’ll find the included PDF very interesting as well. Obviously, you would need to reimplement it in Crystal in order to use it, but it’s a single file in which the majority of the lines are comments. If I can find the other thing I saw recently, I’ll add it to this thread so you can take a look at it.

My general point is that your use case is specialized, so the needs you have in terms of data structures are also specialized. The standard library is good for a lot of things, but it can’t be ideal for every possible use case.

EDIT: Upon looking further at the linked sparse bit set implementation, I don’t think it’s appropriate for your application. I still think that a sparse representation will serve you better, and I’ll still update this thread if I find that other thing I saw before.

EDIT 2: I found what I remembered seeing: [Roaring Bitmaps](http://roaringbitmap.org/). However, it’s not sparse, just compressed, and the documentation explicitly indicates that it’s inappropriate for sparse cases.

---

<div class="post-metadata">

**Author:** ![straight-shoota](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/straight-shoota/32/36_2.png) [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)\
**Post date:** [July 15, 2022, 7:53pm UTC](https://forum.crystal-lang.org/t/bitarray-size-limitation/4778/5 "2022-07-15T19:53:51Z")

</div>

I don’t think the proposal was to have two versions of stdlib, but two versions of `BitArray` (32 and 64 bit) in stdlib.  
That should be reasonably maintainable, as well as a generic type.  
But considering that this is a relatively rare subject, I think it would perfectly fit in a shard.

---

<div class="post-metadata">

**Author:** ![RespiteSage](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/respitesage/32/793_2.png) [@RespiteSage](https://forum.crystal-lang.org/u/RespiteSage)\
**Post date:** [July 15, 2022, 8:06pm UTC](https://forum.crystal-lang.org/t/bitarray-size-limitation/4778/6 "2022-07-15T20:06:23Z")

</div>

> [@straight-shoota](#):
>
> I don’t think the proposal was to have two versions of stdlib, but two versions of `BitArray` (32 and 64 bit) in stdlib.

Ah, I see. I clearly misread the post originally. Sorry about that, @jzakiya.
