Search engine support for shards command?

Maybe I’m alone with this wish, but just as an idea and suggestion:
If there was a common API for search engines, and the shards command would use support it.

Currently I have to search for shards in my browser. The way how I would imagine it, is that I could setup my shards command with one or more of those shard search engines, similar like one can choose a search engine with their browser. Then I could just do a shards search xyz and it asks all the search engines for their results, and lists them. I think it would be nice if I wouldn’t have to leave the terminal, and to be able to search them all at once.
Would be also nice, if there was also an option, to list only unique results if some result appears in more than just one of my sources (probably just note that it was found via search engine A, B, and C).

Currently I often either miss existing shards completely, or I find and use old versions. Such a shards feature would help somewhat to avoid that. And maybe other would appreciate it as much as I would.

1 Like

Sounds like a cool little idea, could probably be done in Crystal too =]

1 Like

ref: New command: search · Issue #87 · crystal-lang/shards · GitHub

1 Like

In that issue, what is the “registry”?

See Registry related commands by ysbaddaden · Pull Request #85 · crystal-lang/shards · GitHub for reference on that. AFAIK it was an early experiment that wasn’t followed through.
Nowadays, as backend for search we can use shard indices such as and

1 Like

Does shardbox already have some api?
If not (I didn’t spot one when I quickly had a glance at your public code):
Do you want to (decide on) and offer one? Then one wouldn’t need to scrap all the information from the html which you wouldn’t have to generate either.

Yeah, I’ve been meaning to add an API, but it’s not implemented yet.

1 Like

That’s fine. Then I’m just going to scrap :wink:

Instead of going that route, would you be interested in contributing an API endpoint for shardbox?

Edit: It’s still rough and primitive and as much as I have noticed, it doesn’t work with alpine linux at all (at least there seem to be no issues on ubuntu), and while it supports only, only shards on github, and no versions, it’s maybe still useful to someone: here is some small tool. It doesn’t replace shards, it just let you search, add and remove shards to your shard.yml. Not more, not less. While I have dozens of ideas, don’t expect it to see any further development except some bug fixes or adjustments in case shardbox should change their page or if there is really some day a common api across the other search pages as well (but tbh: they all bring quite the same results anyway - far more than I noticed so far, so I’m satisfied with the new status quo).
The descriptions are sometimes cut off even though there is still space left in your terminal (in such cases you also won’t see the “…” which it shows otherwise) - that’s when there would have been some html tags, which I didn’t bother to take care of (the scraping is just one single regex - it’s really over simplified). But at least for me it seems to work well (I have actually found already a couple of shards, which I would have never noticed if I would have had to open a browser in order to do so).

Like I said - maybe it’s useful to someone :man_shrugging:

I have added a simple JSON API to in Add JSON API by straight-shoota · Pull Request #18 · shardbox/shardbox-web · GitHub

It’s now live on Currently available routes are

  • /search?q=:query
  • /shard/:name
  • /shard/:name/releases


The JSON format is just a quick draft, it might need some refinement and may change in the future. I think the overall structure should be good, though.