Karma — small key-counter DB (app)

Community
1

Hi! I am pleased to introduce a small key-counter db that solves the problem of storing counters. In simple terms, Karma is a partitioned hash tables for fast counter lookup wrapped in TCP Server.

In one of my projects there was a need to count the number of artifacts created by users (pages, articles, comments and similar things). Since the task is quite simple and Redis is absent in the project infrastructure, I decided to implement this functionality myself and named it Karma.

The project is in alpha version (although already in production) and can do the following things:

  • Quickly create a tree (analog of a table) and increment/decrement values, summarize, get a slice of values between dates and reset values.
  • Save its state to the file system and automatically load it into memory on reboot.
  • Save/load tree dumps by signal or by command from the client.

I will be glad if Karma will help someone else to solve the problem.

Link: GitHub - creadone/karma: Key-counter database

P.S. Thanks to Marcelo Boeira for the inspiring article: Why you should build your own NoSQL database | by Marcelo Boeira | Medium

10 Likes
2

I was needing inspiration for a microservices architecture application. This gives me the springboard to a logger/controller. Thanks for sharing.

1 Like
3

And a great logo too!

1 Like
4

For the exchange of data between the client and the server, I initially considered the option of RPC, but then I chose a simpler way. I recommend looking at this implementation RPC - GitHub - kostya/simple_rpc: RPC Server and Client for Crystal. Implements msgpack-rpc protocol.

5

Thanks to MidJourney for this. I used the following prompt:

mascot logo of cosmic goddess that holds tree with numbers on her palms, simple, vector, white background, by H. R. Giger --no shading detail

image
image496×529 87.3 KB

1 Like
6

So, only ruby version client available?

7

At the moment yes, but in the near future I plan to make clients for Crystal and Lua (Tarantool).

8

Oh, thanks for the inspiration. A little story:

The thing I’m trying to hack together in Crystal these days started out as a pair of Ruby scripts. Simple stuff, one script queries a webservice and puts the result in an sqlite database, another queries the database and pushes data to another webservice. Add in some token persisting and asking the user for how to map at the first run, and that’s it.

Very simple stuff, so I thought “I got two tables, I don’t need no stinking ORM”. I was probably into my fourth SQL query before I caved in and installed sequel.

Now, the Crystal version is a tad more advanced, it’s a web app where you log in using your Google account (so I don’t have to deal with login or passwords), authenticate with the webservices (more OAuth), set up the mapping and it’ll periodically do the push and pull.

Now I don’t need to store the data in the database, as I can just pull the data, map it, push it, and forget it. So I’m down to one table. So “I don’t need no stinking ORM”. Then I decided I needed somewhere to keep a log for each user… So I was back at looking at ORMs for Crystal.

Until you posted that link and started thinking (dangerous, I know). Why am I using an sqlite database in the first place? Because that’s what I’m used to, doing PHP every day, where the interpreter starts from scratch on each request. But with kemal, I can just keep the user list in memory, and just persist it to disk when shutting down. I use it as simple key-value store anyway. The logs can just be regular text files. I’ll probably perform better than querying a database all the time.

So I’ll be ripping out sqlite at some point, but at the moment I’m punting it until I get everything working. Experience has taught me that it’s more important to get something that works, than keep refactoring for all the new ideas that pop up.

You’re never too old to learn new tricks.

2 Likes