# Add validity flags to String **URL:** \ **Category:** Crystal Contrib\ **Created:** [October 8, 2023, 8:13pm UTC](https://forum.crystal-lang.org/t/add-validity-flags-to-string/6065 "2023-10-08T20:13:43Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![funny-falcon](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/funny-falcon/32/159_2.png) [@funny-falcon](https://forum.crystal-lang.org/u/funny-falcon)\ **Post date:** [October 8, 2023, 8:13pm UTC](https://forum.crystal-lang.org/t/add-validity-flags-to-string/6065/1 "2023-10-08T20:13:43Z")
I wish to add `@flags : UInt8` to String to memoize: is it ascii? single byte optimizable? Valid utf8? Such information could be used to speedup searching. Unfortunately I failed to do it. I found couple of places in compiler infrastructure (LLVM typer and program.cr), but it was not enough to compile successfully anything’s (barks on dwarfs or something close to). Looks like there are more places to fix. Please, suggest where I should look to put more changes? ---
**Author:** ![straight-shoota](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/straight-shoota/32/36_2.png) [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)\ **Post date:** [October 8, 2023, 9:23pm UTC](https://forum.crystal-lang.org/t/add-validity-flags-to-string/6065/2 "2023-10-08T21:23:11Z")
There is an open issue about this proposal: > > > I just stumbled across this article in the current issue of Ruby Weekly: \[\*Code …Ranges: A Deeper Look at Ruby Strings\*\](https://shopify.engineering/code-ranges-ruby-strings). It's an interesting read about an internal optimization Ruby interpreters use to characterize a string. Basically, it caches the results of \`ascii\_only?\` and \`valid\_encoding?\`, thus they run in \*O(1)\* instead of \*O(N)\* (on repeated use). > > We could apply this optimization to Crystal's \`String\` class as well. It's even easier to implement because strings are immutable in Crystal (so we don't have to worry about invalidation) and there is only one valid encoding (UTF-8). It's also more efficient for string literals because the compiler can calculate code range values ahead of time. > > For implementing this, we need 2 bits in the string header to represent the four states: > \* ascii\_only > \* valid > \* broken > \* unknown > > We could try to squeeze that into the existing header values, which would reduce the effectively available string size a bit. So it would probably be considered a breaking change. > But I don't think there would be any problems with increasing the size of the string header. It just needs to be synchronized between compiler and runtime (which can be signalled via a compile time flag). I may have a branch with a prototype implementation for this laying around. I’ll have to look that up. ---
**Author:** ![funny-falcon](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/funny-falcon/32/159_2.png) [@funny-falcon](https://forum.crystal-lang.org/u/funny-falcon)\ **Post date:** [October 13, 2023, 5:10am UTC](https://forum.crystal-lang.org/t/add-validity-flags-to-string/6065/3 "2023-10-13T05:10:31Z")
Good day. Did you find the branch? ---
**Author:** ![straight-shoota](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/straight-shoota/32/36_2.png) [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)\ **Post date:** [October 13, 2023, 7:28am UTC](https://forum.crystal-lang.org/t/add-validity-flags-to-string/6065/4 "2023-10-13T07:28:40Z")
Sorry, this slipped my attention. Here it is: [Comparing master...feature/string-code\_range · crystal-lang/crystal · GitHub](https://github.com/crystal-lang/crystal/compare/feature/string-code_range) ---
**Author:** ![funny-falcon](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/funny-falcon/32/159_2.png) [@funny-falcon](https://forum.crystal-lang.org/u/funny-falcon)\ **Post date:** [October 13, 2023, 8:16am UTC](https://forum.crystal-lang.org/t/add-validity-flags-to-string/6065/5 "2023-10-13T08:16:14Z")
I see: I missed build\_string\_constant, and didn’t thought about trick in initialize\_header. Will try to cherry-pick your commit and go further. ---
**Author:** ![straight-shoota](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.crystal-lang.org/straight-shoota/32/36_2.png) [@straight-shoota](https://forum.crystal-lang.org/u/straight-shoota)\ **Post date:** [October 13, 2023, 9:17am UTC](https://forum.crystal-lang.org/t/add-validity-flags-to-string/6065/6 "2023-10-13T09:17:57Z")
Yeah, that same layout information repated across three different locations is a bit hard to track. I had previously investigated this for [Refactor String header layout reflection by straight-shoota · Pull Request #13335 · crystal-lang/crystal · GitHub](https://github.com/crystal-lang/crystal/pull/13335), so I know where to look. Not sure what I did back then to find this.