Using Regex to parse HTTP cookies

wontruefree · August 23, 2021, 5:36pm

I was looking through some of the PRs coming in and noticed HTTP cookies are parsed using regex. This seems like it would be more resource intensive then necessary for this task.

github.com

crystal-lang/crystal/blob/master/src/http/cookie.cr#L121-L151

    
      
          CookieName     = /[^()<>@,;:\\"\/\[\]?={} \t\x00-\x1f\x7f]+/
          CookieOctet    = /[!#-+\--:<-\[\]-~]/
          CookieValue    = /(?:"#{CookieOctet}*"|#{CookieOctet}*)/
          CookiePair     = /(?<name>#{CookieName})=(?<value>#{CookieValue})/
          DomainLabel    = /[A-Za-z0-9\-]+/
          DomainIp       = /(?:\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/
          Time           = /(?:\d{2}:\d{2}:\d{2})/
          Month          = /(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)/
          Weekday        = /(?:Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday)/
          Wkday          = /(?:Mon|Tue|Wed|Thu|Fri|Sat|Sun)/
          PathValue      = /[^\x00-\x1f\x7f;]+/
          DomainValue    = /(?:#{DomainLabel}(?:\.#{DomainLabel})?|#{DomainIp})+/
          Zone           = /(?:UT|GMT|EST|EDT|CST|CDT|MST|MDT|PST|PDT|[+-]?\d{4})/
          RFC1036Date    = /#{Weekday}, \d{2}-#{Month}-\d{2} #{Time} GMT/
          RFC1123Date    = /#{Wkday}, \d{1,2} #{Month} \d{2,4} #{Time} #{Zone}/
          IISDate        = /#{Wkday}, \d{1,2}-#{Month}-\d{2,4} #{Time} GMT/
          ANSICDate      = /#{Wkday} #{Month} (?:\d{2}| \d) #{Time} \d{4}/
          SaneCookieDate = /(?:#{RFC1123Date}|#{RFC1036Date}|#{IISDate}|#{ANSICDate})/
          ExtensionAV    = /(?<extension>[^\x00-\x1f\x7f]+)/
          HttpOnlyAV     = /(?<http_only>HttpOnly)/i

This file has been truncated. show original

Would it be worth using an implementation HTTP cookie parsing uses more native to Crystal to reduce memory and cpu usage?

This would make the code more complex and I am interested if there are any opinions on using regex to parse cookies. Are there any other concerns in using regex to parse cookies?

straight-shoota · August 23, 2021, 6:01pm

Yes, a custom parser is definitly better. I alread have an implementation laying around, it was just pushed to the back for more urgent tasks. The regexes work, so there’s no pressure.

I was also looking into extending the cookie test suite to make sure the parser works correctly in all kinds of edge cases. The current specs are a bit thin in my opinion.
There should be some kind of generic test suite for this. I think I translated some tests from Golang, but they were not that many either. Most I could find were tests for browser behaviour, which is a bit different (it does more than just serializing and deserializing).

hholst · March 31, 2022, 10:11pm

A change to something working should probably provide evidence of improvement too. The regexp libraries are very efficient and I would not take it for granted that a rewrite would be an improvement beyond taste and style.

Topic		Replies	Views
Poll: Experiences with the migration to PCRE2 Crystal Contrib	3	787	March 26, 2023
Documentation improvement: %r in regex Crystal Contrib	1	368	May 20, 2021
Important Update-Crystal is upgrading its Regex engine Official announces	5	613	March 24, 2023
String::gsub catch a group and use it Help & Support	7	105	November 1, 2024
Crystal not use same Regex engine as Ruby? Help & Support	2	497	September 23, 2022

Using Regex to parse HTTP cookies

Related topics