Problem for Process.run to read some syntax

Hi, today I am writing to you because a lot of time, I am facing a problem to integrate properly some Linux call with Process.run.

I think as well it’s the origin of some bugs in my project.

Today, I was coding just a small script to perform a migration for some files. I was just testing a function to make sure it work properly.

My function:

def repack(filename)
    process = Process.run("tar",args: ["cvfJ",filename,"*"],shell: true,error: :inherit,chdir: tempDirectory)

    puts "tar cvfJ #{filename} * (#{tempDirectory})"
end

Unfortunately, it doesn’t work, and have this strange error:

zohran@alienware-m17-r3 ~/Downloads $ crystal Repack.cr
tar: *: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
tar cvfJ Acl.tar.xz * (/home/zohran/Work/temp/)

The very strange thing is, if I copy past this command to execute it in a shell, it just work …

What is wrong ? I feel like the translation to crystal with some characters don’t work very well, like special characters like the star *

Don’t you need to include a - before the cvfJ options?

1 Like

No, I checked already.

Look if I type in the terminal, it work:

zohran@alienware-m17-r3 ~/Work/temp $ tar cvfJ Acl.tar.xz *
acl-2.3.1/
acl-2.3.1/build-aux/
acl-2.3.1/build-aux/ar-lib
acl-2.3.1/build-aux/compile
acl-2.3.1/build-aux/config.guess
acl-2.3.1/build-aux/config.rpath
acl-2.3.1/build-aux/config.sub
acl-2.3.1/build-aux/depcomp
acl-2.3.1/build-aux/install-sh
acl-2.3.1/build-aux/ltmain.sh
acl-2.3.1/build-aux/missing
acl-2.3.1/build-aux/test-driver
acl-2.3.1/doc/
acl-2.3.1/doc/extensions.txt
acl-2.3.1/doc/CHANGES
acl-2.3.1/doc/COPYING
acl-2.3.1/doc/COPYING.LGPL
acl-2.3.1/doc/libacl.txt
acl-2.3.1/doc/PORTING
acl-2.3.1/doc/Makemodule.am
acl-2.3.1/doc/TODO
acl-2.3.1/examples/
acl-2.3.1/examples/Makemodule.am
acl-2.3.1/examples/copy-acl.c
acl-2.3.1/examples/copyperm.c
acl-2.3.1/examples/get-acl.c
acl-2.3.1/examples/Makefile
acl-2.3.1/examples/README
acl-2.3.1/examples/set-acl.c
acl-2.3.1/include/
acl-2.3.1/include/libacl.h
acl-2.3.1/include/acl.h
acl-2.3.1/include/acl_ea.h
acl-2.3.1/include/misc.h
acl-2.3.1/include/walk_tree.h
acl-2.3.1/include/Makemodule.am
acl-2.3.1/include/config.h.in
acl-2.3.1/libacl/
acl-2.3.1/libacl/Makemodule.am
acl-2.3.1/libacl/acl_add_perm.c
acl-2.3.1/libacl/acl_calc_mask.c
acl-2.3.1/libacl/acl_clear_perms.c
acl-2.3.1/libacl/acl_copy_entry.c
acl-2.3.1/libacl/acl_copy_ext.c
acl-2.3.1/libacl/acl_copy_int.c
acl-2.3.1/libacl/acl_create_entry.c
acl-2.3.1/libacl/acl_delete_def_file.c
acl-2.3.1/libacl/acl_delete_entry.c
acl-2.3.1/libacl/acl_delete_perm.c
acl-2.3.1/libacl/acl_dup.c
acl-2.3.1/libacl/acl_free.c
acl-2.3.1/libacl/acl_from_text.c
acl-2.3.1/libacl/acl_get_entry.c
acl-2.3.1/libacl/acl_get_fd.c
acl-2.3.1/libacl/acl_get_file.c
acl-2.3.1/libacl/acl_get_perm.c
acl-2.3.1/libacl/acl_get_permset.c
acl-2.3.1/libacl/acl_get_qualifier.c
acl-2.3.1/libacl/acl_get_tag_type.c
acl-2.3.1/libacl/acl_init.c
acl-2.3.1/libacl/acl_set_fd.c
acl-2.3.1/libacl/acl_set_file.c
acl-2.3.1/libacl/acl_set_permset.c
acl-2.3.1/libacl/acl_set_qualifier.c
acl-2.3.1/libacl/acl_set_tag_type.c
acl-2.3.1/libacl/acl_size.c
acl-2.3.1/libacl/acl_to_text.c
acl-2.3.1/libacl/acl_valid.c
acl-2.3.1/libacl/acl_check.c
acl-2.3.1/libacl/acl_cmp.c
acl-2.3.1/libacl/acl_entries.c
acl-2.3.1/libacl/acl_equiv_mode.c
acl-2.3.1/libacl/acl_error.c
acl-2.3.1/libacl/acl_extended_fd.c
acl-2.3.1/libacl/acl_extended_file.c
acl-2.3.1/libacl/acl_extended_file_nofollow.c
acl-2.3.1/libacl/acl_from_mode.c
acl-2.3.1/libacl/acl_to_any_text.c
acl-2.3.1/libacl/__acl_extended_file.c
acl-2.3.1/libacl/__acl_from_xattr.c
acl-2.3.1/libacl/__acl_reorder_obj_p.c
acl-2.3.1/libacl/__acl_to_any_text.c
acl-2.3.1/libacl/__acl_to_xattr.c
acl-2.3.1/libacl/__apply_mask_to_mode.c
acl-2.3.1/libacl/__libobj.c
acl-2.3.1/libacl/perm_copy_fd.c
acl-2.3.1/libacl/perm_copy_file.c
acl-2.3.1/libacl/libobj.h
acl-2.3.1/libacl/libacl.h
acl-2.3.1/libacl/byteorder.h
acl-2.3.1/libacl/__acl_from_xattr.h
acl-2.3.1/libacl/__acl_to_xattr.h
acl-2.3.1/libacl/perm_copy.h
acl-2.3.1/libacl/__acl_extended_file.h
acl-2.3.1/libmisc/
acl-2.3.1/libmisc/Makemodule.am
acl-2.3.1/libmisc/high_water_alloc.c
acl-2.3.1/libmisc/next_line.c
acl-2.3.1/libmisc/quote.c
acl-2.3.1/libmisc/unquote.c
acl-2.3.1/libmisc/walk_tree.c
acl-2.3.1/m4/
acl-2.3.1/m4/gettext.m4
acl-2.3.1/m4/iconv.m4
acl-2.3.1/m4/intlmacosx.m4
acl-2.3.1/m4/lib-ld.m4
acl-2.3.1/m4/lib-link.m4
acl-2.3.1/m4/lib-prefix.m4
acl-2.3.1/m4/libtool.m4
acl-2.3.1/m4/ltoptions.m4
acl-2.3.1/m4/ltsugar.m4
acl-2.3.1/m4/ltversion.m4
acl-2.3.1/m4/lt~obsolete.m4
acl-2.3.1/m4/nls.m4
acl-2.3.1/m4/package_attrdev.m4
acl-2.3.1/m4/po.m4
acl-2.3.1/m4/progtest.m4
acl-2.3.1/m4/visibility_hidden.m4
acl-2.3.1/man/
acl-2.3.1/man/man1/
acl-2.3.1/man/man1/chacl.1
acl-2.3.1/man/man1/getfacl.1
acl-2.3.1/man/man1/setfacl.1
acl-2.3.1/man/man1/Makemodule.am
acl-2.3.1/man/man3/
acl-2.3.1/man/man3/acl_add_perm.3
acl-2.3.1/man/man3/acl_calc_mask.3
acl-2.3.1/man/man3/acl_check.3
acl-2.3.1/man/man3/acl_clear_perms.3
acl-2.3.1/man/man3/acl_cmp.3
acl-2.3.1/man/man3/acl_copy_entry.3
acl-2.3.1/man/man3/acl_copy_ext.3
acl-2.3.1/man/man3/acl_copy_int.3
acl-2.3.1/man/man3/acl_create_entry.3
acl-2.3.1/man/man3/acl_delete_def_file.3
acl-2.3.1/man/man3/acl_delete_entry.3
acl-2.3.1/man/man3/acl_delete_perm.3
acl-2.3.1/man/man3/acl_dup.3
acl-2.3.1/man/man3/acl_entries.3
acl-2.3.1/man/man3/acl_equiv_mode.3
acl-2.3.1/man/man3/acl_error.3
acl-2.3.1/man/man3/acl_extended_fd.3
acl-2.3.1/man/man3/acl_extended_file.3
acl-2.3.1/man/man3/acl_extended_file_nofollow.3
acl-2.3.1/man/man3/acl_free.3
acl-2.3.1/man/man3/acl_from_mode.3
acl-2.3.1/man/man3/acl_from_text.3
acl-2.3.1/man/man3/acl_get_entry.3
acl-2.3.1/man/man3/acl_get_fd.3
acl-2.3.1/man/man3/acl_get_file.3
acl-2.3.1/man/man3/acl_get_perm.3
acl-2.3.1/man/man3/acl_get_permset.3
acl-2.3.1/man/man3/acl_get_qualifier.3
acl-2.3.1/man/man3/acl_get_tag_type.3
acl-2.3.1/man/man3/acl_init.3
acl-2.3.1/man/man3/acl_set_fd.3
acl-2.3.1/man/man3/acl_set_file.3
acl-2.3.1/man/man3/acl_set_permset.3
acl-2.3.1/man/man3/acl_set_qualifier.3
acl-2.3.1/man/man3/acl_set_tag_type.3
acl-2.3.1/man/man3/acl_size.3
acl-2.3.1/man/man3/acl_to_any_text.3
acl-2.3.1/man/man3/acl_to_text.3
acl-2.3.1/man/man3/acl_valid.3
acl-2.3.1/man/man3/Makemodule.am
acl-2.3.1/man/man5/
acl-2.3.1/man/man5/acl.5
acl-2.3.1/man/man5/Makemodule.am
acl-2.3.1/man/Makemodule.am
acl-2.3.1/test/
acl-2.3.1/test/nfs/
acl-2.3.1/test/nfs/nfsacl.test
acl-2.3.1/test/nfs/nfs-dir.test
acl-2.3.1/test/root/
acl-2.3.1/test/root/getfacl.test
acl-2.3.1/test/root/permissions.test
acl-2.3.1/test/root/restore.test
acl-2.3.1/test/root/setfacl.test
acl-2.3.1/test/Makemodule.am
acl-2.3.1/test/test_passwd.c
acl-2.3.1/test/test_group.c
acl-2.3.1/test/make-tree
acl-2.3.1/test/malformed-restore-double-owner.acl
acl-2.3.1/test/run
acl-2.3.1/test/runwrapper
acl-2.3.1/test/sort-getfacl-output
acl-2.3.1/test/test.passwd
acl-2.3.1/test/test.group
acl-2.3.1/test/cp.test
acl-2.3.1/test/getfacl-lfs.test
acl-2.3.1/test/getfacl-noacl.test
acl-2.3.1/test/getfacl-recursive.test
acl-2.3.1/test/malformed-restore.test
acl-2.3.1/test/misc.test
acl-2.3.1/test/sbits-restore.test
acl-2.3.1/test/setfacl-X.test
acl-2.3.1/test/utf8-filenames.test
acl-2.3.1/tools/
acl-2.3.1/tools/Makemodule.am
acl-2.3.1/tools/chacl.c
acl-2.3.1/tools/getfacl.c
acl-2.3.1/tools/user_group.c
acl-2.3.1/tools/user_group.h
acl-2.3.1/tools/do_set.c
acl-2.3.1/tools/do_set.h
acl-2.3.1/tools/parse.c
acl-2.3.1/tools/parse.h
acl-2.3.1/tools/sequence.c
acl-2.3.1/tools/sequence.h
acl-2.3.1/tools/setfacl.c
acl-2.3.1/Makefile.am
acl-2.3.1/configure
acl-2.3.1/configure.ac
acl-2.3.1/aclocal.m4
acl-2.3.1/Makefile.in
acl-2.3.1/libacl.pc.in
acl-2.3.1/ABOUT-NLS
acl-2.3.1/README
acl-2.3.1/TODO
acl-2.3.1/exports
acl-2.3.1/po/
acl-2.3.1/po/Makefile.in.in
acl-2.3.1/po/remove-potcdate.sin
acl-2.3.1/po/quot.sed
acl-2.3.1/po/boldquot.sed
acl-2.3.1/po/en@quot.header
acl-2.3.1/po/en@boldquot.header
acl-2.3.1/po/insert-header.sin
acl-2.3.1/po/Rules-quot
acl-2.3.1/po/Makevars
acl-2.3.1/po/POTFILES.in
acl-2.3.1/po/de.po
acl-2.3.1/po/es.po
acl-2.3.1/po/fr.po
acl-2.3.1/po/gl.po
acl-2.3.1/po/pl.po
acl-2.3.1/po/sv.po
acl-2.3.1/po/en@boldquot.po
acl-2.3.1/po/en@quot.po
acl-2.3.1/po/de.gmo
acl-2.3.1/po/es.gmo
acl-2.3.1/po/fr.gmo
acl-2.3.1/po/gl.gmo
acl-2.3.1/po/pl.gmo
acl-2.3.1/po/sv.gmo
acl-2.3.1/po/en@boldquot.gmo
acl-2.3.1/po/en@quot.gmo
acl-2.3.1/po/acl.pot
acl-2.3.1/po/stamp-po
acl-2.3.1/po/LINGUAS

Process.run look a bit buggy :dotted_line_face:

I think what’s going on is it works in your shell because the * is a glob that expands to all files in your current dir, which is essentially passing a list of filenames to tar. However, Crystal isn’t able to do that same sort of glob expansion by default, so it’s passing a literal *. So I think you just want to pass "." instead of "*" to let tar itself find all the files and not your shell and should be good to go.

1 Like

* is all files in the current folder, . is the current folder, i consider not same?

Since we provide the shell: true option, why can’t we use *?

Another issue is, chdir will happen before the tar command is running, so, even * work, it will tar all files in the tempDirectory.

1 Like

One more question please, what is the means of :inherit of stderr/stdout? where is the document for it? thanks.

1 Like

Okay, I think I figured out the actual solution. My understanding is shell: true changes what the command parameter of Process.run does. I.e.

So in this case, being on shell: true mode, you actually don’t want to provide your args via the args array, but instead as part of the command itself. This would then run it thru /bin/sh which does the proper expanding of the glob. Probably wouldn’t be a bad idea passing the filename to Process.quote if its user supplied as well.

https://crystal-lang.org/api/1.8.2/Process/Redirect.html

1 Like

Process::Redirect - Crystal 1.8.2

Thanks.

So, the correct behavior usage should be:

process = Process.run("tar cJvf /tempDirectory/Acl.tar.xz *", shell: true, error: :inherit)

BTW: tar option can be used like cJvf without -, but, if you include a long option, e.g. --strip-components=1, you must prefix with - for all options.

2 Likes

I will try all of that, but all of this explanation start to help me now to understand why I can’t debug some call with crystal now.

Thanks a lot. It wasn’t easy to found why this bug occured

So if I wish to use args:, I need to set shell: to false right ?

Because if I set it to false:

def repack(filename)
    process = Process.run("tar",args: ["cvfJ",filename,"*"],error: :inherit,shell: false,chdir: tempDirectory)

    puts "tar cvfJ #{filename} * (#{tempDirectory})"
end

Result:

zohran@alienware-m17-r3 ~/Downloads $ crystal Repack.cr 
tar: *: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
tar cvfJ Acl.tar.xz * (/home/zohran/Work/temp/)

I tried. Set shell: to true ou false doesn’t change the problem. So I will try without args. But it’s a bit annoying for me, because if it’s like that, I will probably need to change a lot of parts in my code

Update:
With shell: true and don’t use args, it work. So definitely, set shell: false with args don’t work properly

If you want shell expansion, you need to use the shell. A command goes through the shell if the entire command is provided in the first argument and shell: true. That’s how it works.
If you want to use an argument array, you may try to expand the array into the command string (command: "tar #{[cvfJ, filename, "*"].join(" ")}", shell: true).
If you don’t want to use the shell, you can pass the filenames directly into the program instead of letting the shell expand them: args: ["cvfJ", filename, Dir.glob("*").join(" ")], shell: false.

1 Like

Thanks all for your help. It solved all last problems I had actually in my project
:smiling_face_with_three_hearts: