Perform a uniq in a Array with specific conditions

Help & Support
1

Hi, I know in Ruby normally we can use block to perform specific uniq in a Array. But I feel like it doesn’t work in Crystal (or I use bad ?)

Some part of my code use this call:

dependenciesLevelArray.uniq! { |dependency| [   dependency.name,
                                                                        dependency.version,
                                                                        dependency.options] }

But the result is never what I expected. Because to explain, dependency isan instance of the class SoftwareDependency:

module ISM

    class SoftwareDependency

        property name : String
        property options : Array(String)

        def initialize
            @name = String.new
            @version = String.new
            @options = Array(String).new
        end

But when I perform this uniq, some duplicate dependencies with the same name and versions, but with differents options are fused in one with all options together ? It’s not completely wrong, but in some conditions, I can’t accept this behavior, how can I do that ?

Maybe it is better if I write dedicated function for that. I am just afraid to do a custom uniq function less fast.

2

Can you show an example of that?
I doubt this has anything to do with #uniq. It should behave as you expect it to (and as Ruby).

3

I tried in isolated file with a simple code, and uniq work normally, I think I messed a part of my code. I will investigate that.

Just a question. If I need some test conditions for the block with uniq, is it possible to do that, or it’s too heavy for a small block ?

4

Just to explain you I have 2 problems.

First I need to perform a uniq! in a multidimensional array. It is an array of arrays of SoftwareDependency. So for this point how can I proceed ? Because uniq need to do in the totality of the arrays , not per array.

The other problem I have is, when I perform this uniq!, the block I would like to pass need to remove duplicates element, but with conditions.

If the SoftwareDependency instances have same name, version and options, we remove the doubles.

If the instances have same names and versions, but differents options, there are 2 cases:
If the options of the doubles don’t contain any option named “Pass1”(can be an another number), I would like uniq fuse them into one item, and put together all enabled options
If it contain any Pass(number), and the pass numbers are different, I would like to keep the doubles

This is the related part:

matching = false
                    unavailableSoftware = false
                    unavailableSoftwaresArray = Array(ISM::SoftwareDependency).new
                    wrongArgument = ""

                    calculationStartingTime = Time.monotonic
                    frameIndex = 0
                    reverseAnimation = false

                    print ISM::Default::Option::SoftwareInstall::CalculationTitle
                    text = ISM::Default::Option::SoftwareInstall::CalculationWaitingText

                    matching, matchingSoftwaresArray, wrongArgument, calculationStartingTime, frameIndex, reverseAnimation = Ism.getRequestedSoftwares(   ARGV[2+Ism.debugLevel..-1].uniq,
                                                                                                                                        calculationStartingTime,
                                                                                                                                        frameIndex,
                                                                                                                                        reverseAnimation,
                                                                                                                                        ISM::Default::Option::SoftwareInstall::CalculationWaitingText)

                    #################################
                    #Get dependencies array by level#
                    #################################
                    currentDependenciesArray = Array(ISM::SoftwareDependency).new

                    matchingSoftwaresArray.each do |software|

                        calculationStartingTime, frameIndex, reverseAnimation = Ism.playCalculationAnimation(calculationStartingTime, frameIndex, reverseAnimation, text)

                        currentDependency = software.toSoftwareDependency
                        currentDependenciesArray << currentDependency
                    end

                    nextDependenciesArray = currentDependenciesArray
                    dependenciesLevelArray = currentDependenciesArray

                    dependencies = Array(ISM::SoftwareDependency).new
                    neededSoftwaresTree = Array(Array(ISM::SoftwareDependency)).new
                    neededSoftwares = Array(ISM::SoftwareDependency).new

                    matchingSoftwaresArray.clear

                    inextricableDependency = false

                    loop do

                        calculationStartingTime, frameIndex, reverseAnimation = Ism.playCalculationAnimation(calculationStartingTime, frameIndex, reverseAnimation, text)

                        currentDependenciesArray.each do |software|

                            animationVariables = Ism.playCalculationAnimation(calculationStartingTime, frameIndex, reverseAnimation, text)

                            calculationStartingTime = animationVariables[0]
                            frameIndex = animationVariables[1]
                            reverseAnimation = animationVariables[2]

                            dependencies = software.dependencies

                            if !dependencies.empty?
                                nextDependenciesArray = nextDependenciesArray + dependencies
                                dependenciesLevelArray = dependenciesLevelArray + dependencies
                            end
                        end

                        dependenciesLevelArray.uniq! { |dependency| [   dependency.name,
                                                                        dependency.version,
                                                                        dependency.options] }

                        if !dependenciesLevelArray.empty?
                            neededSoftwaresTree << dependenciesLevelArray.dup
                        end

                        if nextDependenciesArray.empty?
                            break
                        end

                        if neededSoftwaresTree.size != neededSoftwaresTree.uniq.size
                            inextricableDependency = true
                            neededSoftwaresTree = neededSoftwaresTree & neededSoftwaresTree.uniq
                            break
                        end

                        currentDependenciesArray = nextDependenciesArray.uniq

                        nextDependenciesArray.clear
                        dependenciesLevelArray.clear

                    end

                    if !inextricableDependency
                        neededSoftwaresTree.reverse.each do |level|
                            level.each do |dependency|
                                dependencyInformation = dependency.information
                                if !Ism.softwareIsInstalled(dependencyInformation)
                                    if dependencyInformation.name != ""
                                        matchingSoftwaresArray << dependencyInformation
                                    else
                                        unavailableSoftwaresArray << dependency
                                        unavailableSoftware = true
                                    end
                                end
                            end
                        end

                        unavailableSoftware ? unavailableSoftwaresArray.uniq! : matchingSoftwaresArray.uniq!
                    end
5

Hi, so after a long investigation, I patched few problems, but I have persistent problem.

So to explain I would like if a SoftwareDependency have a Pass option enabled, it is considered as an another dependency, even the name are equal. So to do that, I did a trick, I added a function named hiddenName, to compare with it, and not with name. Hidden name is the name + the passName.

But even I added that, I don’t have any duplicated SoftwareDependency.

Normally, SystemBase, Gcc and others have to be duplicate, because they have differents pass.

For example, before we install Gcc Pass2, I need the Gcc pass1.
But I have this result:

ism@calculate /usr/share/ism $ ism software install groff
ISM start to calculate depencies: Done !

SystemBase /0.2.0/ { Pass1 Pass2 Pass3 }
Binutils /2.37.0/ { Pass1 Pass2 }
Gcc /11.2.0/ { Pass1 Pass2 }
Linux-API-Headers /5.13.12/ { Pass1 }
Glibc /2.34.0/ { Pass1 }
Libstdc++ /11.2.0/ { Pass1 Pass2 }
M4 /1.4.19/ { Pass1 }
Ncurses /6.2.0/ { Pass1 }
Bash /5.1.8/ { Pass1 }
Coreutils /8.32.0/ { Pass1 }
Diffutils /3.8.0/ { Pass1 }
File /5.40.0/ { Pass1 }
Findutils /4.8.0/ { Pass1 }
Gawk /5.1.0/ { Pass1 }
Grep /3.7.0/ { Pass1 }
Gzip /1.10.0/ { Pass1 }
Make /4.3.0/ { Pass1 }
Patch /2.7.6/ { Pass1 }
Sed /4.8.0/ { Pass1 }
Tar /1.34.0/ { Pass1 }
Xz /5.2.5/ { Pass1 }
Gettext /0.21.0/ { Pass1 }
Bison /3.7.6/ { Pass1 }
Perl /5.34.0/ { Pass1 }
Python /3.9.6/ { Pass1 }
Texinfo /6.8.0/ { Pass1 }
Util-Linux /2.37.2/ { Pass1 }
Man-Pages /5.13.0/ { }
Iana-Etc /0.0.0-+20210611/ { }
Zlib /1.2.11/ { }
Bzip2 /1.0.8/ { }
Zstd /1.5.0/ { }
Readline /8.1.0/ { }
Bc /5.0.0/ { }
Flex /2.6.4/ { }
Tcl /8.6.11/ { }
Expect /5.45.4/ { }
DejaGnu /1.6.3/ { }
Gmp /6.2.1/ { }
Mpfr /4.1.0/ { }
Mpc /1.2.1/ { }
Attr /2.5.1/ { }
Acl /2.3.1/ { }
Libcap /2.53.0/ { }
Shadow /4.9.0/ { }
Pkg-Config /0.29.2/ { }
Psmisc /23.4.0/ { }
Libtool /2.4.6/ { }
Gdbm /1.20.0/ { }
Gperf /3.1.0/ { }
Expat /2.4.1/ { }
Inetutils /2.1.0/ { }
Less /590.0.0/ { }
XML-Parser /2.46.0/ { }
Intltool /0.51.0/ { }
Autoconf /2.71.0/ { }
Automake /1.16.4/ { }
Kmod /29.0.0/ { }
Libelf /0.185.0/ { }
Libffi /3.4.2/ { }
Openssl /1.1.1+l/ { }
Ninja /1.10.2/ { }
Meson /0.59.1/ { }
Check /0.15.2/ { }
Groff /1.22.4/ { }

65 new softwares will be install

Would you like to install these softwares ?[y/n]n

I implemented == function to compare properly, but maybe it’s useless ?

My full class implementation:

module ISM

    class SoftwareDependency

        property name : String
        property options : Array(String)

        def initialize
            @name = String.new
            @version = String.new
            @options = Array(String).new
        end

        def getEnabledPass : String
            @options.each do |option|
                if option.starts_with?(/Pass[0-9]/)
                    return option
                end
            end

            return String.new
        end

        def hiddenName : String
            passName = getEnabledPass
            return (passName == "" ? @name : @name+"-"+passName)
        end

        def version=(@version)
        end

        def includeComparators : Bool
            return @version.includes?("<") || @version.includes?(">")
        end

        def greaterComparator : Bool
            return @version[0] == ">" && @version[1] != "="
        end

        def lessComparator : Bool
            return @version[0] == "<" && @version[1] != "="
        end

        def greaterOrEqualComparator : Bool
            return @version[0..1] == ">="
        end

        def lessOrEqualComparator : Bool
            return @version[0..1] == "<="
        end

        def version
            availableVersion = @version

            if includeComparators
                if greaterComparator
                    temporaryAvailableVersion = SemanticVersion.parse(@version.tr("><=",""))

                    Ism.softwares.each do |software|
                        if @name == software.name
                            temporaryVersion = SemanticVersion.parse(@version.tr("><=",""))

                            software.versions.each do |versionInformation|
                                temporarySoftwareVersion = SemanticVersion.parse(versionInformation.version.tr("><=",""))
                                if temporaryVersion < temporarySoftwareVersion && temporaryAvailableVersion < temporarySoftwareVersion
                                    availableVersion = temporarySoftwareVersion.to_s
                                else
                                    availableVersion = @version
                                end
                            end

                        end
                    end
                end

                if lessComparator
                    temporaryAvailableVersion = SemanticVersion.parse(@version.tr("><=",""))

                    Ism.softwares.each do |software|
                        if @name == software.name
                            temporaryVersion = SemanticVersion.parse(@version.tr("><=",""))

                            software.versions.each do |versionInformation|
                                temporarySoftwareVersion = SemanticVersion.parse(versionInformation.version.tr("><=",""))
                                if temporaryVersion > temporarySoftwareVersion && temporaryAvailableVersion > temporarySoftwareVersion
                                    availableVersion = temporarySoftwareVersion.to_s
                                else
                                    availableVersion = @version
                                end
                            end
                        end
                    end
                end

                if greaterOrEqualComparator
                    temporaryAvailableVersion = SemanticVersion.parse(@version.tr("><=",""))

                    Ism.softwares.each do |software|
                        if @name == software.name
                            temporaryVersion = SemanticVersion.parse(@version.tr("><=",""))

                            software.versions.each do |versionInformation|
                                temporarySoftwareVersion = SemanticVersion.parse(versionInformation.version.tr("><=",""))
                                if temporaryVersion <= temporarySoftwareVersion && temporaryAvailableVersion <= temporarySoftwareVersion
                                    availableVersion = temporarySoftwareVersion.to_s
                                else
                                    availableVersion = @version
                                end
                            end
                        end
                    end
                end

                if lessOrEqualComparator
                    temporaryAvailableVersion = SemanticVersion.parse(@version.tr("><=",""))

                    Ism.softwares.each do |software|
                        if @name == software.name
                            temporaryVersion = SemanticVersion.parse(@version.tr("><=",""))

                            software.versions.each do |versionInformation|
                                temporarySoftwareVersion = SemanticVersion.parse(versionInformation.version.tr("><=",""))
                                if temporaryVersion >= temporarySoftwareVersion && temporaryAvailableVersion >= temporarySoftwareVersion
                                    availableVersion = temporarySoftwareVersion.to_s
                                else
                                    availableVersion = @version
                                end
                            end
                        end
                    end
                end
            end

            return availableVersion
        end

        def information : ISM::SoftwareInformation
            dependencyInformation = Ism.getSoftwareInformation(@name,@version)

            @options.each do |option|
                dependencyInformation.enableOption(option)
            end

            return dependencyInformation
        end

        def dependencies : Array(ISM::SoftwareDependency)
            return information.dependencies
        end

        def == (other : ISM::SoftwareDependency) : Bool
            return hiddenName == other.hiddenName &&
            version == other.version &&
            @options == other.options
        end

    end

end

Depend of this class as well:

module ISM

  class SoftwareInformation

    record Option,
        name : String,
        description : String,
        active : Bool,
        dependencies : Array(Dependency),
        downloadLinks : Array(String),
        md5sums : Array(String) do
        include JSON::Serializable
    end
    
    record Dependency,
        name : String,
        version : String,
        options : Array(String) do
        include JSON::Serializable
    end
    
    record Information,
        port : String,
        name : String,
        version : String,
        architectures : Array(String),
        description : String,
        website : String,
        downloadLinks : Array(String),
        md5sums : Array(String),
        patchesLinks : Array(String),
        installedFiles : Array(String),
        dependencies : Array(Dependency),
        options : Array(Option) do
        include JSON::Serializable
    end

    property port : String
    property name : String
    property version : String
    property architectures : Array(String)
    property description : String
    property website : String
    property downloadLinks : Array(String)
    property md5sums : Array(String)
    property patchesLinks : Array(String)
    setter options : Array(ISM::SoftwareOption)
    property installedFiles : Array(String)
    setter dependencies : Array(ISM::SoftwareDependency)

    def initialize
        @port = String.new
        @name = String.new
        @version = String.new
        @architectures = Array(String).new
        @description = String.new
        @website = String.new
        @downloadLinks = Array(String).new
        @md5sums = Array(String).new
        @patchesLinks = Array(String).new
        @installedFiles = Array(String).new
        @dependencies = Array(ISM::SoftwareDependency).new
        @options = Array(ISM::SoftwareOption).new
    end

    def getEnabledPass : String
        @options.each do |option|
            if option.isPass && option.active
                return option.name
            end
        end

        return String.new
    end

    def loadInformationFile(loadInformationFilePath : String)
        begin
            information = Information.from_json(File.read(loadInformationFilePath))
        rescue error : JSON::ParseException
            puts    "#{ISM::Default::SoftwareInformation::FileLoadProcessSyntaxErrorText1 +
                    loadInformationFilePath +
                    ISM::Default::SoftwareInformation::FileLoadProcessSyntaxErrorText2 +
                    error.line_number.to_s}".colorize(:yellow)
            return
        end

        @port = information.port
        @name = information.name
        @version = information.version
        @architectures = information.architectures
        @description = information.description
        @website = information.website
        @downloadLinks = information.downloadLinks
        @md5sums = information.md5sums
        @patchesLinks = information.patchesLinks
        @installedFiles = information.installedFiles

        information.dependencies.each do |data|
            dependency = ISM::SoftwareDependency.new
            dependency.name = data.name
            dependency.version = data.version
            dependency.options = data.options
            @dependencies << dependency
        end

        information.options.each do |data|
            dependenciesArray = Array(ISM::SoftwareDependency).new
            data.dependencies.each do |dependency|
                temporary = ISM::SoftwareDependency.new
                temporary.name = dependency.name
                temporary.version = dependency.version
                temporary.options = dependency.options
                dependenciesArray << temporary
            end

            option = ISM::SoftwareOption.new
            option.name = data.name
            option.description = data.description
            option.active = data.active
            option.dependencies = dependenciesArray
            option.downloadLinks = data.downloadLinks
            option.md5sums = data.md5sums
            @options << option
        end

    end

    def writeInformationFile(writeInformationFilePath : String)
        path = writeInformationFilePath.chomp(writeInformationFilePath[writeInformationFilePath.rindex("/")..-1])

        if !Dir.exists?(path)
            Dir.mkdir_p(path)
        end


        dependenciesArray = Array(Dependency).new
        @dependencies.each do |data|
            dependenciesArray << Dependency.new(data.name,data.version,data.options)
        end

        optionsArray = Array(Option).new
        @options.each do |data|
            optionsDependenciesArray = Array(Dependency).new
            data.dependencies.each do |dependencyData|
                dependency = Dependency.new(dependencyData.name,dependencyData.version,dependencyData.options)
                optionsDependenciesArray << dependency
            end

            optionsArray << Option.new(data.name,data.description,data.active,optionsDependenciesArray,data.downloadLinks,data.md5sums)
        end

        information = Information.new(  @port,
                                        @name,
                                        @version,
                                        @architectures,
                                        @description,
                                        @website,
                                        @downloadLinks,
                                        @md5sums,
                                        @patchesLinks,
                                        @installedFiles,
                                        dependenciesArray,
                                        optionsArray)

        file = File.open(writeInformationFilePath,"w")
        information.to_json(file)
        file.close
    end

    def versionName
        return @name+"-"+@version
    end

    def builtSoftwareDirectoryPath
        return "#{ISM::Default::Path::BuiltSoftwaresDirectory}#{@port}/#{@name}/#{@version}/"
    end

    def filePath : String
        return Ism.settings.rootPath +
               ISM::Default::Path::SoftwaresDirectory +
               @port + "/" +
               @name + "/" +
               @version + "/" +
               ISM::Default::Filename::Information
    end

    def requireFilePath : String
        return Ism.settings.rootPath +
               ISM::Default::Path::SoftwaresDirectory +
               @port + "/" +
               @name + "/" +
               @version + "/" +
               @version + ".cr"
    end

    def settingsFilePath : String
        return  Ism.settings.rootPath +
                ISM::Default::Path::SettingsSoftwaresDirectory +
                @name + "/" +
                @version + "/" +
                ISM::Default::Filename::SoftwareSettings
    end

    def installedFilePath : String
        return Ism.settings.rootPath +
               ISM::Default::Path::InstalledSoftwaresDirectory +
               @port + "/" +
               @name + "/" +
               @version + "/" +
               ISM::Default::Filename::Information
    end

    def options : Array(ISM::SoftwareOption)
        if File.exists?(settingsFilePath)
            settingsInformation = Information.from_json(File.read(settingsFilePath))
            settingsOptions = Array(ISM::SoftwareOption).new

            settingsInformation.options.each do |data|
                dependenciesArray = Array(ISM::SoftwareDependency).new
                data.dependencies.each do |dependency|
                    temporary = ISM::SoftwareDependency.new
                    temporary.name = dependency.name
                    temporary.version = dependency.version
                    temporary.options = dependency.options
                    dependenciesArray << temporary
                end

                option = ISM::SoftwareOption.new
                option.name = data.name
                option.description = data.description
                option.active = data.active
                option.dependencies = dependenciesArray
                option.downloadLinks = data.downloadLinks
                option.md5sums = data.md5sums
                settingsOptions << option
            end

            return settingsOptions
        end

        return @options
    end

    def option(optionName : String) : Bool
        @options.each do |option|
            if optionName == option.name
                return option.active
            end
        end

        return false
    end

    def enableOption(optionName : String)
        @options.each_with_index do |option, index|
            if optionName == option.name
                if option.isPass
                    currentEnabledPass = getEnabledPass

                    if passEnabled && currentEnabledPass != optionName
                        disableOption(currentEnabledPass)
                    end

                    @options[index].active = true
                end
            end
        end
    end

    def disableOption(optionName : String)
        @options.each_with_index do |option, index|
            if optionName == option.name
                @options[index].active = false
            end
        end
    end

    def passEnabled : Bool
        @options.each do |option|
            if option.isPass
                return true
            end
        end

        return false
    end

    def dependencies : Array(ISM::SoftwareDependency)
        passEnabled = false
        dependenciesArray = Array(ISM::SoftwareDependency).new

        @options.each do |option|
            if option.isPass && option.active
                dependenciesArray = option.dependencies
                passEnabled = true
                break
            else
                if option.active
                    dependenciesArray = dependenciesArray+option.dependencies
                end
            end
        end

        return (passEnabled ? dependenciesArray : @dependencies+dependenciesArray)
    end

    def toSoftwareDependency : ISM::SoftwareDependency
        softwareDependency = ISM::SoftwareDependency.new

        softwareDependency.name = @name
        softwareDependency.version = @version

        @options.each do |option|
            if option.active
                softwareDependency.options << option.name
            end
        end

        return softwareDependency
    end

    def == (other : ISM::SoftwareInformation) : Bool
        return @name == other.name &&
            @version == other.version &&
            @options == other.options
    end

  end

end

If you need more details, ask me or you can look directly the repository: ISM Repository

The dependencies calculation is here: Dependencies calculation implementation

6

I patched the bugs, but definitely now, the problem come from the array.uniq method.

First I need from this method to perform uniq when items have same hidden names, version and options.
The other thing, how can I perform a uniq in an array of array of class ?

This is my biggest problem actually ?

7

Can you put together a minimal reproduction of your issue? It’s not clear to me what the problem with #uniq is and trying to decipher that from 500 lines of a program I don’t know is a bit much for those trying to help.

EDIT: Minimal meaning like a handful of lines where you have a variable with a small array of data, you use #uniq on it, and show the actual versus expected output.

2 Likes
8

Yeah no problem, sorry I do my best to explain.

So in one word, if I have an multidimensional array of SoftwareDependency like that (this is just a purely simple example):

arr = [ [SoftwareDependency.new("Gcc","11.2.0",["Pass1"]),SoftwareDependency.new("Bash","5.1.8",[])],
        [SoftwareDependency.new("Gcc","11.2.0",["Pass1"]),SoftwareDependency.new("Ble","0.3.0",[])],
        [SoftwareDependency.new("Gcc","11.2.0",["Pass2"]),SoftwareDependency.new("Firefox","108.0",["openrc"])],
        [SoftwareDependency.new("Gcc","11.2.0",[])]]

If I perform a uniq, I would like this result, so no any duplication in the entire array:

[   [SoftwareDependency.new("Gcc","11.2.0",["Pass1"]),SoftwareDependency.new("Bash","5.1.8",[])],
            [SoftwareDependency.new("Ble","0.3.0",[])],
            [SoftwareDependency.new("Gcc","11.2.0",["Pass2"]),SoftwareDependency.new("Firefox","108.0",["openrc"])],
            [SoftwareDependency.new("Gcc","11.2.0",[])]]]

The special thing is, when uniq is performed, I want it to compare with the method hiddenName from SoftwareDependency class, not the member name, and compare options and version.

If we follow my example, when we create the item, it’s like that:
SoftwareDependency.new(name,version,options)
Normally option in my code is an array of an another class, but I wrote a String to make the example more simple.

I accept the duplication only if options have differents Pass in the option, not when the options are differents.

If 2 softwaredependency are same name and version but differents option (but no pass), I want them to fuse in one, with the options fusionned

This is why I implemented a function hiddenName, to know the current enabled Pass option of a software, because hiddenName return the name of the software plus the enabled pass name.

Only one pass can be enabled, not multiple pass together.

It’s a bit complicate I agree

Is it clear ?

9

I think I follow what you’re wanting now, but I’m not so sure you can actually use #uniq since the uniqueness needs to be global between all nested arrays. There may be a better way to handle it, but I came up with this which seems to work:

lookup = Hash(SoftwareDependency, Bool).new

arr.each do |deps|
  deps.each do |d|
    if lookup.has_key? d
      deps.delete d
    else
      lookup[d] = true
    end
  end
end

https://play.crystal-lang.org/#/r/eney

It’ll remove any dependencies that it sees more than once.

EDIT: If you’re using a class then your #== method is needed:

class SoftwareDependency 
  getter name : String
  getter version : String
  getter options : Array(String)
  
  def initialize(@name, @version, @options); end
  
  def ==(other : self)
    @name == other.name && @version == other.version && @options == other.options
  end
end
10

Hmmmm your answer is very useful for me. To be honest I almost never use Hash, but I realize how much it can be useful now.

I think I will rewrite completely the function is in charge of calculate the dependencies. I think it will be definitely better to use a Hash to do that.

I have a question now, because I have an idea.

If I would like to make easy to calculate easily the dependencies order for the softwares, is it a good idea if I make a dedicated class the dependencies ?

I mean make an array of this dedicated class, and define in this class how to now if a dependency is greater or lower, greater or lower mean for a dependency if the dependency is higher or lower in the dependencies tree, and when this is define properly, you just need to call the sort function from the array class.

What do you think ?

This is an example of what I think (to sort the dependencies), but I have an error. And I am a bit annoyed of what to return if 2 dependencies are just different, without greater or lower:

class Dependency

    property name : String
    property dependencies : Array(String)

    def initialize(@name,@dependencies)
    end

    def <=>(other : Dependency)
        if other.dependencies.includes?(@name)
            return 1
        end
        if @dependencies.includes?(other.name)
            return -1
        end
        if @name == other.name
            return 0
        end
    end

end

arr = [ Dependency.new("Gcc",["Binutils","Libstdc++"]),
        Dependency.new("SystemBase",[] of String),
        Dependency.new("Binutils",["SystemBase"]),
        Dependency.new("Libstdc++",["Binutils"])]

puts arr.sort

Normally, when sort is called, which function need to be redefined? This one : <=> ? Is it the same for uniq ?

11

Guys, I found usefull information about my bug, if somebody have an idea. I think something is wrong in my implementation, but the bug is just crazy

I made a new algorithm to calculate the dependencies, I will show you my test code, the result is just crazy. I don’t understand why the value of the options off my array of SoftwareDependency change.

This is the test code:

def getDependenciesTable(softwareList : Array(ISM::SoftwareInformation)) : Hash(ISM::SoftwareDependency,Array(ISM::SoftwareDependency))

                puts softwareList
                #dependenciesTable = Hash(ISM::SoftwareDependency,Array(ISM::SoftwareDependency)).new
                keysTable = Array(ISM::SoftwareDependency).new
                valuesTable = Array(Array(ISM::SoftwareDependency)).new

                puts "TEST"
                puts "--------------------------------"

                softwareList.each do |software|

                    dependencies = getRequiredDependencies(software)

                    puts software.name
                    #puts software.version
                    puts software.options
                    #puts "software.toSoftwareDependency"
                    #puts software.toSoftwareDependency.name
                    puts "---"
                    puts software.toSoftwareDependency.version
                    puts software.toSoftwareDependency.options

                    #keysTable.push(software.toSoftwareDependency)
                    #valuesTable.push(dependencies)

                    #dependencies.each do |dependency|
                        #keysTable.push(dependency)
                        #valuesTable.push(getRequiredDependencies(dependency.information))
                    #end

                end

                return keysTable.zip(valuesTable).to_h#dependenciesTable
            end

In the output, the first array is the value of the argument before I call the function, the other one is under the function. And the last output is the value of each element when I perform a each. Why the values changes ???

Arrray before function
[#<ISM::SoftwareInformation:0x7fd68c0c1bd0 @port="Utilities-Main", @name="File", @version="5.40.0", @architectures=["x86_64"], @description="Command-line tool that tells what kind of data a file contains", @website="https://www.darwinsys.com/file/", @downloadLinks=["https://astron.com/pub/file/file-5.40.tar.gz"], @md5sums=["72540ea1cc8c6e1dee35d6100ec66589"], @patchesLinks=[], @options=[#<ISM::SoftwareOption:0x7fd68c0ee440 @name="Pass1", @description="Enable the phase 1 of the cross toolchain building", @active=false, @dependencies=[#<ISM::SoftwareDependency:0x7fd68c0bf1e0 @name="Diffutils", @options=["Pass1"], @version="3.8.0">], @downloadLinks=[], @md5sums=[]>], @installedFiles=[], @dependencies=[#<ISM::SoftwareDependency:0x7fd68c0bf210 @name="Zstd", @options=[], @version="1.5.0">]>]
Array under the function
[#<ISM::SoftwareInformation:0x7fd68c0c1bd0 @port="Utilities-Main", @name="File", @version="5.40.0", @architectures=["x86_64"], @description="Command-line tool that tells what kind of data a file contains", @website="https://www.darwinsys.com/file/", @downloadLinks=["https://astron.com/pub/file/file-5.40.tar.gz"], @md5sums=["72540ea1cc8c6e1dee35d6100ec66589"], @patchesLinks=[], @options=[#<ISM::SoftwareOption:0x7fd68c0ee440 @name="Pass1", @description="Enable the phase 1 of the cross toolchain building", @active=false, @dependencies=[#<ISM::SoftwareDependency:0x7fd68c0bf1e0 @name="Diffutils", @options=["Pass1"], @version="3.8.0">], @downloadLinks=[], @md5sums=[]>], @installedFiles=[], @dependencies=[#<ISM::SoftwareDependency:0x7fd68c0bf210 @name="Zstd", @options=[], @version="1.5.0">]>]
TEST
--------------------------------
File
[#<ISM::SoftwareOption:0x7fd68c0ee440 @name="Pass1", @description="Enable the phase 1 of the cross toolchain building", @active=true, @dependencies=[#<ISM::SoftwareDependency:0x7fd68c0bf1e0 @name="Diffutils", @options=["Pass1"], @version="3.8.0">], @downloadLinks=[], @md5sums=[]>]
---
5.40.0
["Pass1"]
Done !

The final question is, why the options changes :dotted_line_face:?
If you need again more informations about the implementation, it’s here: GitHub - Fulgurance/ISM: Ingenius System Manager

It’s like this function alter the value of software. Because when I comment the call of getRequiredDependencies, the array stay okay:

def getRequiredDependencies(software : ISM::SoftwareInformation) : Array(ISM::SoftwareDependency)
                dependencies = Hash(String,SoftwareDependency).new

                currentDependencies = software.dependencies
                nextDependencies = Array(ISM::SoftwareDependency).new

                #puts "------------------"
                #puts software.toSoftwareDependency.hiddenName
                #puts software.options

                loop do

                    if currentDependencies.empty?
                        break
                    end

                    currentDependencies.each do |dependency|

                        #Inextricable dependencies or need multiple version or just need to fusion options
                        if dependencies.has_key? dependency.hiddenName

                            #Inextricable dependencies
                            if dependencies[dependency.hiddenName] == dependency

                                exit 1
                            else
                                #Multiple versions of single software requested
                                if dependencies[dependency.hiddenName].version != dependency.version


                                    exit 1
                                #When versions are equal but options are differents
                                else
                                #Fusion of all options and add it as well to the nextDependencies

                                end
                            end
                        else
                            dependencies[dependency.hiddenName] = dependency
                            nextDependencies += dependency.dependencies

                            #puts "---------------"
                            #puts dependency.hiddenName
                            #puts dependency.dependencies

                        end

                    end

                    currentDependencies = nextDependencies.dup
                    nextDependencies.clear

                end

                #puts "------------------"
                #puts software.toSoftwareDependency.hiddenName
                #puts software.name
                #puts software.version
                #puts software.options
                #puts dependencies.values

                return dependencies.values
            end
12

I think I’m back to not following what the problem is now. Can you make a more minimal example that I can copy/paste into my editor and run? Ideally with some clear comments of what exactly the issue is. Because from that output I’m not sure what the problem is.

1 Like
13

I found more accurately what is happen when the bug appear.

So to explain I have this function:

def getRequiredDependencies(software : ISM::SoftwareInformation) : Array(ISM::SoftwareDependency)
                dependencies = Hash(String,SoftwareDependency).new

                currentDependencies = software.dependencies
                nextDependencies = Array(ISM::SoftwareDependency).new

                loop do

                    if currentDependencies.empty?
                        break
                    end

                    currentDependencies.each do |dependency|

                        #Inextricable dependencies or need multiple version or just need to fusion options
                        if dependencies.has_key? dependency.hiddenName

                            #Inextricable dependencies
                            if dependencies[dependency.hiddenName] == dependency

                                exit 1
                            else
                                #Multiple versions of single software requested
                                if dependencies[dependency.hiddenName].version != dependency.version


                                    exit 1
                                #When versions are equal but options are differents
                                else
                                #Fusion of all options and add it as well to the nextDependencies


                                end
                            end
                        else
                            dependencies[dependency.hiddenName] = dependency
                            nextDependencies += dependency.dependencies
                        end

                    end

                    currentDependencies = nextDependencies.dup
                    nextDependencies.clear

                end

                return dependencies.values
            end

This function return the dependencies tree of a given software. It work in a single software.
But now a bug appear.

Now I made this function to get the dependencies of a list of software.

def getDependenciesTable(softwareList : Array(ISM::SoftwareInformation)) : Hash(ISM::SoftwareDependency,Array(ISM::SoftwareDependency))
                dependenciesTable = Hash(ISM::SoftwareDependency,Array(ISM::SoftwareDependency)).new

                softwareList.each do |software|
                    dependenciesTable[software.toSoftwareDependency] = getRequiredDependencies(software)
                end

                return dependenciesTable
            end

When this list contain only one software, it work, but if I have more than 1 item, the result of getRequiredDependencies is wrong.

Look. Result when I request only one software:

ism@calculate /usr/share/ism $ ism software install file
File
[#<ISM::SoftwareDependency:0x7f1cacf2e210 @name="Zstd", @options=[], @version="1.5.0">, #<ISM::SoftwareDependency:0x7f1cab71f2d0 @name="Xz", @options=[], @version="5.2.5">, #<ISM::SoftwareDependency:0x7f1cab71f420 @name="Bzip2", @options=[], @version="1.0.8">, #<ISM::SoftwareDependency:0x7f1cab71f6f0 @name="Zlib", @options=[], @version="1.2.11">, #<ISM::SoftwareDependency:0x7f1cacf2e870 @name="Glibc", @options=[], @version="2.34.0">, #<ISM::SoftwareDependency:0x7f1cab6f8c00 @name="Iana-Etc", @options=[], @version="0.0.0-+20210611">, #<ISM::SoftwareDependency:0x7f1cab71fd20 @name="Man-Pages", @options=[], @version="5.13.0">, #<ISM::SoftwareDependency:0x7f1cab6f8720 @name="SystemBase", @options=["Pass3"], @version="0.2.0">, #<ISM::SoftwareDependency:0x7f1cacf2e630 @name="Util-Linux", @options=["Pass1"], @version="2.37.2">, #<ISM::SoftwareDependency:0x7f1cab6f8150 @name="Texinfo", @options=["Pass1"], @version="6.8.0">, #<ISM::SoftwareDependency:0x7f1cab6f8240 @name="Python", @options=["Pass1"], @version="3.9.6">, #<ISM::SoftwareDependency:0x7f1cab71fba0 @name="Perl", @options=["Pass1"], @version="5.34.0">, #<ISM::SoftwareDependency:0x7f1cab71fcc0 @name="Bison", @options=["Pass1"], @version="3.7.6">, #<ISM::SoftwareDependency:0x7f1cacf2e330 @name="Gettext", @options=["Pass1"], @version="0.21.0">, #<ISM::SoftwareDependency:0x7f1cab6f8e40 @name="Libstdc++", @options=["Pass2"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7f1cab71f870 @name="SystemBase", @options=["Pass2"], @version="0.2.0">, #<ISM::SoftwareDependency:0x7f1cacf2e660 @name="Gcc", @options=["Pass2"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7f1cab6f8e10 @name="Binutils", @options=["Pass2"], @version="2.37.0">, #<ISM::SoftwareDependency:0x7f1cacf2e4b0 @name="Xz", @options=["Pass1"], @version="5.2.5">, #<ISM::SoftwareDependency:0x7f1cab71f3c0 @name="Tar", @options=["Pass1"], @version="1.34.0">, #<ISM::SoftwareDependency:0x7f1cab71f4e0 @name="Sed", @options=["Pass1"], @version="4.8.0">, #<ISM::SoftwareDependency:0x7f1cab6f84b0 @name="Patch", @options=["Pass1"], @version="2.7.6">, #<ISM::SoftwareDependency:0x7f1cab6f86f0 @name="Make", @options=["Pass1"], @version="4.3.0">, #<ISM::SoftwareDependency:0x7f1cab6f8a80 @name="Gzip", @options=["Pass1"], @version="1.10.0">, #<ISM::SoftwareDependency:0x7f1cab71f600 @name="Grep", @options=["Pass1"], @version="3.7.0">, #<ISM::SoftwareDependency:0x7f1cab6f8c90 @name="Gawk", @options=["Pass1"], @version="5.1.0">, #<ISM::SoftwareDependency:0x7f1cab6f8f30 @name="Findutils", @options=["Pass1"], @version="4.8.0">, #<ISM::SoftwareDependency:0x7f1cacf2e0f0 @name="File", @options=["Pass1"], @version="5.40.0">, #<ISM::SoftwareDependency:0x7f1cacf2e1e0 @name="Diffutils", @options=["Pass1"], @version="3.8.0">, #<ISM::SoftwareDependency:0x7f1cacf2e690 @name="Coreutils", @options=["Pass1"], @version="8.32.0">, #<ISM::SoftwareDependency:0x7f1cacf2e390 @name="Bash", @options=["Pass1"], @version="5.1.8">, #<ISM::SoftwareDependency:0x7f1cacf2e7e0 @name="Ncurses", @options=["Pass1"], @version="6.2.0">, #<ISM::SoftwareDependency:0x7f1cab71f240 @name="M4", @options=["Pass1"], @version="1.4.19">, #<ISM::SoftwareDependency:0x7f1cab6f88a0 @name="Libstdc++", @options=["Pass1"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7f1cab71f8a0 @name="Glibc", @options=["Pass1"], @version="2.34.0">, #<ISM::SoftwareDependency:0x7f1cab6f8ba0 @name="Linux-API-Headers", @options=["Pass1"], @version="5.13.12">, #<ISM::SoftwareDependency:0x7f1cab71f540 @name="Gcc", @options=["Pass1"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7f1cab6f8ea0 @name="Binutils", @options=["Pass1"], @version="2.37.0">, #<ISM::SoftwareDependency:0x7f1cacf2e4e0 @name="SystemBase", @options=["Pass1"], @version="0.2.0">]

When I request 2 softwares. You will see the dependencies of File are differents:

ism@calculate /usr/share/ism $ ism software install diffutils file
Diffutils
[#<ISM::SoftwareDependency:0x7ff29b4e65d0 @name="Check", @options=[], @version="0.15.2">, #<ISM::SoftwareDependency:0x7ff29b4e6180 @name="Coreutils", @options=[], @version="8.32.0">, #<ISM::SoftwareDependency:0x7ff29b4e6390 @name="Meson", @options=[], @version="0.59.1">, #<ISM::SoftwareDependency:0x7ff299cb0930 @name="Ninja", @options=[], @version="1.10.2">, #<ISM::SoftwareDependency:0x7ff299cb0810 @name="Python", @options=[], @version="3.9.6">, #<ISM::SoftwareDependency:0x7ff299cd6bd0 @name="Openssl", @options=[], @version="1.1.1+l">, #<ISM::SoftwareDependency:0x7ff299cb0d20 @name="Libffi", @options=[], @version="3.4.2">, #<ISM::SoftwareDependency:0x7ff299cd6a20 @name="Libelf", @options=[], @version="0.185.0">, #<ISM::SoftwareDependency:0x7ff299cd6ae0 @name="Kmod", @options=[], @version="29.0.0">, #<ISM::SoftwareDependency:0x7ff299cb0a20 @name="Automake", @options=[], @version="1.16.4">, #<ISM::SoftwareDependency:0x7ff29b4e65a0 @name="Autoconf", @options=[], @version="2.71.0">, #<ISM::SoftwareDependency:0x7ff29b4e66c0 @name="Intltool", @options=[], @version="0.51.0">, #<ISM::SoftwareDependency:0x7ff299cd6e40 @name="XML-Parser", @options=[], @version="2.46.0">, #<ISM::SoftwareDependency:0x7ff299cd69c0 @name="Perl", @options=[], @version="5.34.0">, #<ISM::SoftwareDependency:0x7ff299cd6cc0 @name="Less", @options=[], @version="590.0.0">, #<ISM::SoftwareDependency:0x7ff299cb0960 @name="Inetutils", @options=[], @version="2.1.0">, #<ISM::SoftwareDependency:0x7ff299cd6090 @name="Expat", @options=[], @version="2.4.1">, #<ISM::SoftwareDependency:0x7ff299cd6b10 @name="Gperf", @options=[], @version="3.1.0">, #<ISM::SoftwareDependency:0x7ff299cb0d50 @name="Gdbm", @options=[], @version="1.20.0">, #<ISM::SoftwareDependency:0x7ff299cd6f90 @name="Libtool", @options=[], @version="2.4.6">, #<ISM::SoftwareDependency:0x7ff299cd66f0 @name="Bash", @options=[], @version="5.1.8">, #<ISM::SoftwareDependency:0x7ff29b4e67b0 @name="Grep", @options=[], @version="3.7.0">, #<ISM::SoftwareDependency:0x7ff299cb0c90 @name="Bison", @options=[], @version="3.7.6">, #<ISM::SoftwareDependency:0x7ff29b4e6300 @name="Gettext", @options=[], @version="0.21.0">, #<ISM::SoftwareDependency:0x7ff299cb0e40 @name="Psmisc", @options=[], @version="23.4.0">, #<ISM::SoftwareDependency:0x7ff299cb0570 @name="Sed", @options=[], @version="4.8.0">, #<ISM::SoftwareDependency:0x7ff299cb04b0 @name="Ncurses", @options=[], @version="6.2.0">, #<ISM::SoftwareDependency:0x7ff299cd6240 @name="Pkg-Config", @options=[], @version="0.29.2">, #<ISM::SoftwareDependency:0x7ff299cb05d0 @name="Gcc", @options=[], @version="11.2.0">, #<ISM::SoftwareDependency:0x7ff299cb0ea0 @name="Shadow", @options=[], @version="4.9.0">, #<ISM::SoftwareDependency:0x7ff299cb09c0 @name="Libcap", @options=[], @version="2.53.0">, #<ISM::SoftwareDependency:0x7ff299cd6ba0 @name="Acl", @options=[], @version="2.3.1">, #<ISM::SoftwareDependency:0x7ff29b4e63f0 @name="Attr", @options=[], @version="2.5.1">, #<ISM::SoftwareDependency:0x7ff29b4e6510 @name="Mpc", @options=[], @version="1.2.1">, #<ISM::SoftwareDependency:0x7ff299cd63c0 @name="Mpfr", @options=[], @version="4.1.0">, #<ISM::SoftwareDependency:0x7ff299cd6300 @name="Gmp", @options=[], @version="6.2.1">, #<ISM::SoftwareDependency:0x7ff299cd6ed0 @name="Binutils", @options=[], @version="2.37.0">, #<ISM::SoftwareDependency:0x7ff29b4e64b0 @name="DejaGnu", @options=[], @version="1.6.3">, #<ISM::SoftwareDependency:0x7ff29b4e6720 @name="Expect", @options=[], @version="5.45.4">, #<ISM::SoftwareDependency:0x7ff29b4e62a0 @name="Tcl", @options=[], @version="8.6.11">, #<ISM::SoftwareDependency:0x7ff299cb0300 @name="Flex", @options=[], @version="2.6.4">, #<ISM::SoftwareDependency:0x7ff29b4e6000 @name="Bc", @options=[], @version="5.0.0">, #<ISM::SoftwareDependency:0x7ff29b4e6450 @name="M4", @options=[], @version="1.4.19">, #<ISM::SoftwareDependency:0x7ff299cb08a0 @name="Readline", @options=[], @version="8.1.0">, #<ISM::SoftwareDependency:0x7ff299cd6030 @name="File", @options=[], @version="5.40.0">, #<ISM::SoftwareDependency:0x7ff29b4e61e0 @name="Zstd", @options=[], @version="1.5.0">, #<ISM::SoftwareDependency:0x7ff299cd62a0 @name="Xz", @options=[], @version="5.2.5">, #<ISM::SoftwareDependency:0x7ff299cd63f0 @name="Bzip2", @options=[], @version="1.0.8">, #<ISM::SoftwareDependency:0x7ff299cd66c0 @name="Zlib", @options=[], @version="1.2.11">, #<ISM::SoftwareDependency:0x7ff29b4e6810 @name="Glibc", @options=[], @version="2.34.0">, #<ISM::SoftwareDependency:0x7ff299cb0bd0 @name="Iana-Etc", @options=[], @version="0.0.0-+20210611">, #<ISM::SoftwareDependency:0x7ff299cd6cf0 @name="Man-Pages", @options=[], @version="5.13.0">, #<ISM::SoftwareDependency:0x7ff299cb06f0 @name="SystemBase", @options=["Pass3"], @version="0.2.0">, #<ISM::SoftwareDependency:0x7ff29b4e6600 @name="Util-Linux", @options=["Pass1"], @version="2.37.2">, #<ISM::SoftwareDependency:0x7ff299cb0120 @name="Texinfo", @options=["Pass1"], @version="6.8.0">, #<ISM::SoftwareDependency:0x7ff299cb0210 @name="Python", @options=["Pass1"], @version="3.9.6">, #<ISM::SoftwareDependency:0x7ff299cd6b70 @name="Perl", @options=["Pass1"], @version="5.34.0">, #<ISM::SoftwareDependency:0x7ff299cd6c90 @name="Bison", @options=["Pass1"], @version="3.7.6">, #<ISM::SoftwareDependency:0x7ff29b4e62d0 @name="Gettext", @options=["Pass1"], @version="0.21.0">, #<ISM::SoftwareDependency:0x7ff299cb0e10 @name="Libstdc++", @options=["Pass2"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7ff299cd6840 @name="SystemBase", @options=["Pass2"], @version="0.2.0">, #<ISM::SoftwareDependency:0x7ff29b4e6630 @name="Gcc", @options=["Pass2"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7ff299cb0de0 @name="Binutils", @options=["Pass2"], @version="2.37.0">, #<ISM::SoftwareDependency:0x7ff29b4e6420 @name="Xz", @options=["Pass1"], @version="5.2.5">, #<ISM::SoftwareDependency:0x7ff299cd6390 @name="Tar", @options=["Pass1"], @version="1.34.0">, #<ISM::SoftwareDependency:0x7ff299cd64b0 @name="Sed", @options=["Pass1"], @version="4.8.0">, #<ISM::SoftwareDependency:0x7ff299cb0480 @name="Patch", @options=["Pass1"], @version="2.7.6">, #<ISM::SoftwareDependency:0x7ff299cb06c0 @name="Make", @options=["Pass1"], @version="4.3.0">, #<ISM::SoftwareDependency:0x7ff299cb0a50 @name="Gzip", @options=["Pass1"], @version="1.10.0">, #<ISM::SoftwareDependency:0x7ff299cd65d0 @name="Grep", @options=["Pass1"], @version="3.7.0">, #<ISM::SoftwareDependency:0x7ff299cb0c60 @name="Gawk", @options=["Pass1"], @version="5.1.0">, #<ISM::SoftwareDependency:0x7ff299cb0f00 @name="Findutils", @options=["Pass1"], @version="4.8.0">, #<ISM::SoftwareDependency:0x7ff29b4e60c0 @name="File", @options=["Pass1"], @version="5.40.0">, #<ISM::SoftwareDependency:0x7ff29b4e61b0 @name="Diffutils", @options=["Pass1"], @version="3.8.0">, #<ISM::SoftwareDependency:0x7ff29b4e6570 @name="Coreutils", @options=["Pass1"], @version="8.32.0">, #<ISM::SoftwareDependency:0x7ff29b4e6360 @name="Bash", @options=["Pass1"], @version="5.1.8">, #<ISM::SoftwareDependency:0x7ff29b4e66f0 @name="Ncurses", @options=["Pass1"], @version="6.2.0">, #<ISM::SoftwareDependency:0x7ff299cd6210 @name="M4", @options=["Pass1"], @version="1.4.19">, #<ISM::SoftwareDependency:0x7ff299cb0870 @name="Libstdc++", @options=["Pass1"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7ff299cd6870 @name="Glibc", @options=["Pass1"], @version="2.34.0">, #<ISM::SoftwareDependency:0x7ff299cb0b70 @name="Linux-API-Headers", @options=["Pass1"], @version="5.13.12">, #<ISM::SoftwareDependency:0x7ff299cd6510 @name="Gcc", @options=["Pass1"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7ff299cb0e70 @name="Binutils", @options=["Pass1"], @version="2.37.0">, #<ISM::SoftwareDependency:0x7ff29b4e6480 @name="SystemBase", @options=["Pass1"], @version="0.2.0">]
File
[#<ISM::SoftwareDependency:0x7ff29b4e61b0 @name="Diffutils", @options=["Pass1"], @version="3.8.0">, #<ISM::SoftwareDependency:0x7ff29b4e6570 @name="Coreutils", @options=["Pass1"], @version="8.32.0">, #<ISM::SoftwareDependency:0x7ff29b4e6360 @name="Bash", @options=["Pass1"], @version="5.1.8">, #<ISM::SoftwareDependency:0x7ff29b4e66f0 @name="Ncurses", @options=["Pass1"], @version="6.2.0">, #<ISM::SoftwareDependency:0x7ff299cd6210 @name="M4", @options=["Pass1"], @version="1.4.19">, #<ISM::SoftwareDependency:0x7ff299cb0870 @name="Libstdc++", @options=["Pass1"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7ff299cd6870 @name="Glibc", @options=["Pass1"], @version="2.34.0">, #<ISM::SoftwareDependency:0x7ff299cb0b70 @name="Linux-API-Headers", @options=["Pass1"], @version="5.13.12">, #<ISM::SoftwareDependency:0x7ff299cd6510 @name="Gcc", @options=["Pass1"], @version="11.2.0">, #<ISM::SoftwareDependency:0x7ff299cb0e70 @name="Binutils", @options=["Pass1"], @version="2.37.0">, #<ISM::SoftwareDependency:0x7ff29b4e6480 @name="SystemBase", @options=["Pass1"], @version="0.2.0">]

I suspect getRequiredDependencies to alterate it argument software, I am almost sure it is that, but I dont know how it is possible.

Because I did a test, and after a getRequiredDependencies, the array is alterated

14

I am preparing a code example at Crystal Play for you to reproduce the bug. Just give me one second

1 Like
15

So, I uploaded an archive with the project adapted to be tested.

You can run the crystal interpreter or compile it before you test, as you wish.

So under the project, if you run in a terminal:

crystal Main.cr software install file

You will get an output of all needed softwares for the software File.
If you do the same, with 2 arguments as well, you will have again a result:

crystal Main.cr software install file diffutils

So, just forget what I said in the previous post, I prefer you make your own opinion about the bug.

But the output show clearly something is wrong, because normally the expected result is not this one.

To explain you, a Software have dependencies, and optional dependencies.

For example if I would like to enable optional dependencies, I need to enables some options of the software, that’s why SoftwareInformation contain an array of SoftwareOption.

Normally, for normal software, if you enable an option, you will have the normal dependencies (without options enabled) + the new dependencies required by the enabled option.

The other case is, you have a special option named Pass(x), x is a number.

This is for special software needed to be rebuild differently. Because if you build a full Linux system from scratch, for example Gcc need to be build 3 times in total. That why I implemented this kind of option.

So the behavior of the dependencies calculation change when this option is here.

If s software for example need Gcc with the option Pass1, the needed dependencies will be ONLY the dependencies needed by the option Pass1, nothing more or less.

So in the input you can see clearly something wrong, because first, File need Zstd in the database, but just before File, we can see Iana-Etc (it doesn’t make sense)

And some software in the list are build 2 times in succession, but it’s not possible. For example before we install Glibc, we need Iana-Etc, and before long time ago, we need Glibc Pass1

Diffutils with Pass1 is duplicated as well.

Anyway. If you need to understand the project, all informations are stored in json file (Information.json), under the directory run in the project.

The main implementation is under the directory ISM.
Under this one, Default contain all defaults values, and Options contain the implementation of all ISM commands.
So the command:

software install

Is under ISM/Options/Software/Install

Let me know if you need more detail again

EDIT: You will get an error is you try due to persmission denied. Just edit the file ISM/CommandLineSettings at line 49 and 53, and replace “/” by “./”

16

Ah, well an archive of your project is not what I had in mind for a “minimal example” :sweat_smile:. I did run those two commands you provided and can definitely see Diffutils-3.8.0["Pass1"] in there twice. But, not to be rude, I’m not about to dive into this project just to figure out why.

However, if you could do something more like Carcin where you have an isolated, actually minimal, example of the one method that the problem is in, and define a clear actual versus expected output, that would be a lot easier for people to help out with.

In regards to this, are you sure it’s actually a problem you see zstd and lana-etc listed? IDK where/how you’re determining dependencies, but for Arch at least Arch Linux - file 5.45-1 (x86_64) you can see it has a required dependency on zstd, and depends on glibc which depends on filesystem which depends on lana-etc. So these are in fact NOT optional dependencies.

17

Yeah I was thinking you would like something smaller. but to be honest, at this level of the development, now it’s difficult to make something small.

The database is huge as well. I will try to take a time to give you a small example.

For Arch, you can’t really compare, it’s a binary distribution, and they don’t manage the compilation complexity for the dependencies, specially from scratch.

Yes SoftwareOption is for optional dependencies, but the option Pass are different. When ism see a Pass option, their dependencies are not optionals.

18

I did a minimalistic example man, It was a pain, but I reduced all classes as the minimal possible.

https://play.crystal-lang.org/#/r/eoax/

Almost at the end of the code, you have the result you should have in comment.
I just hope this minimal code have the same behaviour of my project. I think normaly yes

19

Not to throw stones from the peanut gallery, but I looked at your github repo and you don’t have any specs (tests) defined. When I’ve run into unexpected behavior, defining tests and seeing how they fail has been invaluable to understanding and fixing the code.

1 Like
20

Do you mean it’s possible to deploy test with git ? To be honest I am not very familiar with git functionalities