Hello Crystal community,
I saw that Crymongo received an update 4 months ago, but it is compatible with Mongo up to 4.2.
Since we are already at Mongo 8.3, I would like to know if there is anyone working on a Mongo driver.
I’m considering trying to contribute to Crymongo.
Ok. Temporarily forked bson.cr and cryomongo. Better help the community and unify efforts instead of splitting things.
The ongoing work at bson.cr fork is going well. I’ve been working on it for a day now, and I estimate I’ll finish it by the end of the day.
Sending the improvements to upstream will depends on @elbywan approval, and this is my goal, to return everything to the original repository as soon as the roadmap is concluded.
Later on I will move my efforts to cryomongo.
My understanding is that we cannot update bson.cr and publish it before also updating cryomongo, since they both need to be in sync of what the latest MongoDB expects.
Since I’m not at the level of Julien, I will start first focusing on Linux and local deployments, exclusively for MongoDB 8.0. Then I will probably need help to update the more complex things like integration with MongoDB Atlas and eventual retro-compatibility with previous MongoDB versions.
Since the current driver supports Mongo up to 4.2, one possibility in this update is to skip all intermediary versions before 8.0.
This is because, most probably, no one used cryomongo on newer versions of Mongo 4.2 anyway.
So one simpler strategy could be to document the compatibility of the existing version for Mongo 4.2 and the new version to 8.0.
Update: the complete update on the temporary fork at alumna/bson.cr is concluded. 
It is fully updated and test covered to the latest BSON specification used by MongoDB 8.0.
Lots of optimizations were done, including the long-awaited BSON::Decimal128 implementation in pure Crystal.
Starting now the work at Cryomongo
Update on Cryomongo: the work is now 50% 75% 90% 100% concluded (with basic auth).
The temporary fork at alumna/cryomongo partially works on MongoDB 8.0 and uses the official MongoDB Unified Test Format (UTF), in place of the previous specifications (now deprecated in favor of UTF).
All of of the added tests from UTF work, and I am was adding them gradually (see below). There is a simple roadmap and detailed one. Basically, this is the current one:
unified-test-format.md): new runner for UTFcrud.md): 100% compliant.retryable-writes.md): 100% compliant.retryable-reads.md): 100% compliant.transactions.md & transactions-convenient-api.md): 100% compliant.causal-consistency.md): 100% compliant.server-discovery-and-monitoring.md): 100% compliant.auth.md): Basic auth concluded. Beta usage.I’ll try to finish it this week, if possible.
Note: this would not be possible without the incredible work of @elbywan, so thank you Julien! I will inform when everything seems good to finally open a PR on the original/upstream repo.
Update:
Sessions, Causal Consistency and Snapshot Reads (MongoDB 5.0+) specifications now added.
6 of 8 categories from the roadmap for MongoDB 8.0 now concluded (75%).
Link: releases/tag/v0.8.0
Update:
Server Discovery and Monitoring (SDAM) specifications now added, including adjustments for MongoDB 6.0+.
This was probably the most difficult one.
7 of 8 categories concluded (~90%). Just one more.
Link: releases/tag/v0.10.0
Update: The fork is in beta state and welcome testers.
Basic authentication is finally done (last item from the beta roadmap).
To surpass beta, it will need performance optimizations and improved tests, specially multi-topology tests:
These items are the focus of my next steps.
Link: releases/tag/v0.12.0
Update: the fork is now in a stronger beta state (but still beta).
Multi-topology tests (standalone, replica set and sharded) now running and passing.
Cases that were previously non-supported are now printed as pending. Progress is more clear now for everyone that wants to get involved (or just to check).
Working right now on the load-balanced tests/topology. It should land today in the next 0.16.0 version.
Oh, and performance is much faster now.
Link: releases/tag/v0.15.0
Update: we are close. Phase 2 concluded. Just one more to arrive at production-grade level.
Load-balanced topology added together with standalone, replica set and sharded.
75+ additional specifications passing.
Link: releases/tag/v0.16.0
Small OT, since it has nothing to do with the Crystal driver, only the MongoDB itself:
Jepsen evaluated MongoDB version 4.2.6, and found that even at the strongest levels of read and write concern, it failed to preserve snapshot isolation. Instead, Jepsen observed read skew, cyclic information flow, duplicate writes, and internal consistency violations. Weak defaults meant that transactions could lose writes and allow dirty reads, even downgrading requested safety levels at the database and collection level.
Source: Jepsen: MongoDB 4.2.6
@Sija this is a good point to bring up. MongoDB was not perfect, even more in its early days.
But these safety-breaking bugs were solved right after, in 4.2.8 and 4.4.0 releases. And the article states that.
From MongoDB 5.0 and onwards, it changed the implicit default from w:1 to w:majority for most replica sets. IMHO, that alone removed the most common way that Jepsen showed possible data loss.
8.0, in which I’m working on, is even safer and cheaper in this regard.
Is MongoDB perfect? No. Is it production-grade when used as intended*? Yes, that’s how most large Atlas deployments run today.
*replica sets, explicit snapshot + majority for critical transactions
Good to know, thanks for the clarification!
Update: usable in production for MongoDB 8.0.
All mandatory phases concluded and release at v0.17.2.
@elbywan I don’t know if you are active here in the forum. I will send a PR after solving:
Please inform me if you prefer a PR now, and then we can solve those items in the official repo.