What's up with the Debian packages?

I upgraded to 1.9.2+dfsg-1 today (been a couple of months since last time) and now there’s no shards command anymore. And shards requires itself, so there’s a catch-22 for new users.

And I had to install libevent-dev in order to compile a simple program, shouldn’t that be a dependency then?

The old version wasn’t compiled with interactive (I was sad to discover), but the newest don’t even have play?

What’s up with that?

We don’t maintain the packages in the official Debian and Ubuntu repositories. What I do is putting something like below in /etc/apt/preferences.d/99crystal:

Package: *
Pin: release o=obs://build.opensuse.org/devel:languages:crystal/Debian_Unstable,n=Debian_Unstable,l=devel:languages:crystal,c=
Pin-Priority: 900

The package infos, by the way:

$ apt info crystal -a
Package: crystal
Version: 1.9.2+dfsg-1
Priority: optional
Section: devel
Maintainer: David Suárez <deiv@debian.org>
Installed-Size: 19.8 MB
Depends: pkg-config, libc6 (>= 2.34), libevent-2.1-7 (>= 2.1.8-stable), libgc1 (>= 1:7.4.2), libgcc-s1 (>= 3.3), libllvm14, libpcre2-8-0 (>= 10.22), libstdc++6 (>= 11), libgc-dev
Suggests: crystal-doc, crystal-samples
Homepage: https://crystal-lang.org/
Download-Size: 3,614 kB
APT-Sources: http://deb.debian.org/debian testing/main amd64 Packages
Description: compiler of the Crystal object-oriented programming language
 The Crystal's language syntax is inspired by Ruby, the language is statically
 type-checked but does not require that the type of variables or method
 arguments be specified.
 .
 The language has the following goals: (1) have the same syntax as Ruby, or at
 least as similar as possible; (2) statically type-checked but without having
 to specify the type of variables or method arguments; (3) be able to call C
 code by writing bindings to it in Crystal; (4) have compile-time evaluation
 and generation of code, to avoid boilerplate code; (5) compile to efficient
 native code.

Package: crystal
Version: 1.9.2-1+2.15
Priority: extra
Section: devel
Maintainer: Crystal Team <crystal@manas.tech>
Installed-Size: 133 MB
Provides: crystal1.9
Depends: gcc, pkg-config, libpcre3-dev, libpcre2-dev, libevent-dev
Recommends: libssl-dev, libz-dev, libxml2-dev, libgmp-dev, libyaml-dev
Conflicts: crystal
Homepage: https://crystal-lang.org
Download-Size: 31.8 MB
APT-Manual-Installed: yes
APT-Sources: http://download.opensuse.org/repositories/devel:languages:crystal/Debian_Unstable  Packages
Description: Crystal is a general-purpose, object-oriented programming language.
 With syntax inspired by Ruby, it is a compiled language with static type-checking,
 serving both, humans and computers.

Ah, faceslap didn’t realize Crystal was in the official repos now, and probably haven’t been bitten by it because I upgraded before the new version hit the official repos.

Apt still seems to think that the dfsg version is newer, so I’ll have to read up on apt again.

@HertzDevil
Does that pin work for you? Or are you running stable?

I had to bump the Pin-Priority to 1001 to allow downgrades, as 1.9.2+dfsg is always a higher version than 1.9.2 (had to dig into Debian package version internals to figure that one out)

So I ended up with:

Package: *
Pin: release o=obs://build.opensuse.org/devel:languages:crystal/Debian_Testing,n=Debian_Testing,l=devel:languages:crystal,c=
Pin-Priority: 1001

Using Package: crystal would probably be more correct, but I got shards back, so I’m done fiddling.

Trixie (testing), but did the pin beforehand when I still had -1+2.11 and an update brought both -1.2+12 (our package) and +dfsg-1 (their package), so I never had to downgrade anything

Ah, there be magic in them numbers… Works for me with 900 now.

Pinning could be a tad more user-friendly. Thanks for the help.

The magic is described in apt_preferences(5)

1 Like

I submitted a bug in the debian issue tracker. It would be good if Crystal is properly shipped out-of-the-box in debian-based systems. I’ll see how much engagement I get from the maintainer to drive this to completion… :crossed_fingers:

4 Likes

Hi @Xen,

Debian maintainer here. Sorry, but by mistake, we lost some recents changes done in previous versions. Im releasing a new debian version with the correct dependecies.

interpreter was disabled in previous versions because I found that was broken (at least in Debian), will enable it and test that is working right.

Playground, was disabled some versions ago because copyright isuess with the fonts (and posible will never be enabled).

About shards, it was never packaged into Debian, packaging it into Debian is in TODO.

Version 1.9.2+dfsg-2 is on the way.

3 Likes

I don’t understand, and googling doesn’t help me.

In the copyright file is states:

Files-Excluded: src/compiler/crystal/tools/playground/public/vendor/*
Comment: removed because there are distributed without the original source code

But as far as I can see, all the packages in vendor are MIT licensed if going to the original source. Shouldn’t it just be a matter of the core team committing the original LICENCE file from each project?

Not the problem here. Let me explain:

Files-Excluded: src/compiler/crystal/tools/playground/public/vendor/*
Comment: removed because there are distributed without the original source code

Only indicates that this files are excluded from the packaging original sources (in debian we look for pristine source code).

In https://github.com/crystal-lang/crystal/tree/master/src/compiler/crystal/tools/playground/public/vendor directory we have 5 vendored software packages inside the crystal source code. We just remove them as that packages are currently inside Debian (we don’t like to duplicate software inside the archive).

At first, playground was enabled because that five packages were found in the Debian archives, but at some point it was disabled because octicons package was removed from the archive. Look at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=883107 bug, point 2:

  1. the package is distributed under the MIT license but our users will
    not be able to make use of the freedoms this license is granting
    them (specifically, modify the files we ship) using just the tools
    in Debian main, instead they need tools outside of Debian. It is
    violating the spirit of the DFSG if we are shipping files that our
    users are unable to modify.

As octicons could not be used as a dependency and our rules are that we do not like to use precompiled or vendored software inside packages, the playground cannot be compiled until octicons will enter again the Debian archive.

Maybe this explanation could put a light on :slight_smile:

2 Likes

The new version is entering unstable now (Debian Package Tracker)… Could be good if I can get feedback on it :slight_smile:

The interpreter was enabled again.

About shards I’m looking into it and soon a new package named shards will enter debian :slight_smile:

5 Likes

Ok, now I remenber why interpreter gets disabled. We are getting test failures in the build (Build log for crystal (1.9.2+dfsg-2) on all):

  1) Crystal::Repl::Interpreter conversion interprets Float32#to_u64! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 72057594037927935
            got: 18446744073709551593

     # spec/compiler/interpreter/primitives_spec.cr:219

  2) Crystal::Repl::Interpreter conversion interprets Float64#to_u! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 4294967040
            got: 4294967273

     # spec/compiler/interpreter/primitives_spec.cr:219

  3) Crystal::Repl::Interpreter conversion interprets Float32#to_u16! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 65280
            got: 65513

     # spec/compiler/interpreter/primitives_spec.cr:219

  4) Crystal::Repl::Interpreter conversion interprets Float32#to_u! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 4294967040
            got: 4294967273

     # spec/compiler/interpreter/primitives_spec.cr:219

  5) Crystal::Repl::Interpreter conversion interprets Float64#to_u64! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 72057594037927935
            got: 18446744073709551593

     # spec/compiler/interpreter/primitives_spec.cr:219

  6) Crystal::Repl::Interpreter conversion interprets Float32#to_u32! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 4294967040
            got: 4294967273

     # spec/compiler/interpreter/primitives_spec.cr:219

  7) Crystal::Repl::Interpreter conversion interprets Float64#to_u8! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 0
            got: 233

     # spec/compiler/interpreter/primitives_spec.cr:219

  8) Crystal::Repl::Interpreter conversion interprets Float32#to_u8! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 0
            got: 233

     # spec/compiler/interpreter/primitives_spec.cr:219

  9) Crystal::Repl::Interpreter conversion interprets Float64#to_u16! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 65280
            got: 65513

     # spec/compiler/interpreter/primitives_spec.cr:219

 10) Crystal::Repl::Interpreter conversion interprets Float64#to_u32! (negative)
     Failure/Error: {% for target_type in %w(u8 i8 u16 i16 u32 i32 u i u64 i64 f32 f64).map(&.id) %}

       Expected: 4294967040
            got: 4294967273

     # spec/compiler/interpreter/primitives_spec.cr:219

In my machine im getting:

icr:4> -23.8_f32.to_u16
Unhandled exception: Arithmetic overflow (OverflowError)
  from :1:1 in '<top-level>'
icr:5> -23.8_f32.to_u8
Unhandled exception: Arithmetic overflow (OverflowError)
  from :1:1 in '<top-level>'

Could disable the failing specs, but I don’t want to go that way. Will upload a new version with the interpreter again disabled and posibly open a issue in github.

Anyway, any help here is welcome.

1 Like

@deiv
OK, sometimes Debian is giving me a headache.

So the Crystal playground is disabled because you can’t modify the font file it uses with the tools available in Debian…

I do appreciate that the Debian maintainer of the docker package does the work of splitting up the package and dynamically linking it, rather than just statically linking everything in as they do in the official package.

But crystal is using v3.5 of octicons, which is from 2016… The newest versions doesn’t even use font files anymore, but just uses the SVG files directly. So the chance of a variant of the package that Crystal can use, getting in Debian is, as you say, practically zero.

Which is mightily unfortunate as the playground is very helpful for the new user.

So, for the sake of the argument, if someone took it upon themselves to grab the SVG of the few icons the playground uses, put them in a directory with a note saying where they were obtained from and a copy of the license, replaced the usage of said icons to be inline SVG rather than CSS and fonts, and removed the octicons folder… Could that revive the playground in the Debian package?

I’m far from any lawyer, but I believe that this should be in accordance with the octicons license, and SVG is certainly editable with Debian tools. And until someone packages the SVG files of octicons one can make the argument that it’s far more efficient to include the few icons used rather than having to package the whole set.

Can you post the code that is producing the arithmetic overflows?

In Crystal, when you do an arithmetic operation a op b, where op is (+, -, *, /, //),
you want a to be the largest type variable so that the result of op is converted to it.

Also, with 1.9.0, Crystal fully supports U|Int128 values, so i128 and u128 should go in the lists.

this is already tracked here: Interpreter spec failures on Debian · Issue #13736 · crystal-lang/crystal · GitHub

The main point is that we can simply redistribute a compiled form of the font:

  • distributing compiled files, does not gives the security that are 100% the original source.
  • doing it will break the liberty of the user to do what he wants to do with the fonts.
  • etc

You solution on simply copy and paste the fonts, im not an expert lawyer too, could work if the application states clearly and includes the original copyright and a copy of the original license, as the MIT license elaborates:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

But really, including vendoring software inside another software is always a pain in the ass for package distribution. And for end users too. Theres not any gain If every app has it owns copy of his third dependencies.

I follow the logic, I just think it’s a tad pedantic when applied to an icon font. Even if you can “recompile” octicons, you can still replace it with a font of handdrawn icons without loss of functionality in crystal play.

But it’s how it is.

Excellent. Now the question is how the core team would feel about such a workarond. @beta-ziliani?

The Snapcraft folks would disagree. In general I agree with you, but when features get cut due to a few bloody icons, I think it goes a bit overboard. But if we can land a compromise everybody can live with, let’s do that.

2 Likes

Several points to make. First and foremost, I’m very thankful for the work deiv is doing (for free!) to help ship Crystal in Debian :bowing_man: :bowing_man: :bowing_man:

The second one, is that there’s already a fix for the octicon issue by @GeopJr : Update octicons to v19.5.0 by GeopJr · Pull Request #13738 · crystal-lang/crystal · GitHub, in which the svg file is used instead. I understand this should be compliant with Debian. @deiv when you find yourself some time, can you check this?

The third one: there’s also a fix about the failing test in the interpreter (Remove overflowing `Float#to_u!` interpreter primitive specs by HertzDevil · Pull Request #13737 · crystal-lang/crystal · GitHub).

And the last thing: these changes are going to be merged for Crystal 1.10. Moving forward, @deiv, do you want us to contact you after 1.10 is released to trigger a new build there? Or how would you want to go about releases?

2 Likes

Thanks @beta-ziliani and others for you help here too !

The icons, at first sight, looks good to me :slight_smile:. Dont worry about the releases, I will be watching.

4 Likes