# Maximum Hash size reached - so low
**URL:** https://forum.crystal-lang.org/t/maximum-hash-size-reached-so-low/4796
**Category:** Help & Support
**Created:** [July 21, 2022, 8:46pm UTC](https://forum.crystal-lang.org/t/maximum-hash-size-reached-so-low/4796 "2022-07-21T20:46:00Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@pfischer](https://forum.crystal-lang.org/u/pfischer)
#### Post date: [July 21, 2022, 8:46pm UTC](https://forum.crystal-lang.org/t/maximum-hash-size-reached-so-low/4796/1 "2022-07-21T20:46:00Z")
Hello,
I can’t store `150_000_000` items even into a simple `Hash(Int32, Int32)`.
1. When I try to create `Hash` with precreated space - with `initial_capacity` in constructor - “**Arithmetic overflow (OverflowError)**” occurs in the `Hash` creation
2. When I ignore `initial_capacity`, “ **Maximum Hash size reached** ” error occurs while filling the `Hash` somwhere around `100_000_000` entries.
What…? Is `150_000_000` items really too much today? What about a billion entries? Why is maximum so low?
So I should implement my own `BigHash`?
Thanks! pf
---
### Author:  [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)
#### Post date: [July 21, 2022, 9:45pm UTC](https://forum.crystal-lang.org/t/maximum-hash-size-reached-so-low/4796/2 "2022-07-21T21:45:12Z")
This is unfortunately a limitation of the internal 32-bit index size.
See explanation here in the source code:
>
I’m afraid there is currently no alternative for collections using bigger size types in the standard library.
cf. [Data structers for large datasets · Issue #8523 · crystal-lang/crystal · GitHub](https://github.com/crystal-lang/crystal/issues/8523)
This is a long-known problem but a solution that involves changing stdlib’s size type is hard. And apparently this limitation is rarely an issue in practice.
---
### Author:  [@asterite](https://forum.crystal-lang.org/u/asterite)
#### Post date: [July 21, 2022, 11:18pm UTC](https://forum.crystal-lang.org/t/maximum-hash-size-reached-so-low/4796/3 "2022-07-21T23:18:52Z")
Maybe you could describe your use case or particular problem. There might be a way to model it without a huge hash.
---
### Author:  [@pfischer](https://forum.crystal-lang.org/u/pfischer)
#### Post date: [July 21, 2022, 11:52pm UTC](https://forum.crystal-lang.org/t/maximum-hash-size-reached-so-low/4796/4 "2022-07-21T23:52:27Z")
Imagine just a big in-memory index (object\_ids → position in a file + some other metadata) or something like this.
It looks like it won’t be a big problem to copy stdlib `Hash` and make it based on Int64 (unfortunately, it will not be able to implement `Enumerable` (because `size` in `Enumerable` is `Int32` etc).
---
### Author:  [@mavu](https://forum.crystal-lang.org/u/mavu)
#### Post date: [July 22, 2022, 9:21am UTC](https://forum.crystal-lang.org/t/maximum-hash-size-reached-so-low/4796/5 "2022-07-22T09:21:17Z")
Is my math off, or is that already 1.5gb of data if each entry has even only 10 bytes (total, including internaly used memory of the type)?
If you really need to work with such large indices of stuff, i would probably roll my own datatype.
That makes it easier later to do stuff like lazy loading of data or pagination when you run out of memory on the machine.
---
### Author:  [@jgaskins](https://forum.crystal-lang.org/u/jgaskins)
#### Post date: [July 24, 2022, 1:35am UTC](https://forum.crystal-lang.org/t/maximum-hash-size-reached-so-low/4796/6 "2022-07-24T01:35:56Z")
> [@pfischer](#):
>
> a big in-memory index (object\_ids → position in a file + some other metadata) or something like this.
Interesting, is this something that’s feasible to use Redis for?
I can imagine that populating that giant index all at once wouldn’t work for that, but if that mapping is accumulated over time, it might be a decent tradeoff since [it can hold 4 billion keys](https://redis.io/docs/getting-started/faq/#what-is-the-maximum-number-of-keys-a-single-redis-instance-can-hold-what-is-the-maximum-number-of-elements-in-a-hash-list-set-and-sorted-set).