OPNsense Forum

English Forums => General Discussion => Topic started by: Taomyn on September 23, 2020, 09:23:00 am

Title: How do I extract content of a FreeBSD package?
Post by: Taomyn on September 23, 2020, 09:23:00 am
Hi, I have a FreeBSD package file, .pkg, which I'd like to run on the firewall but I don't want to fully install it. Is there a way to extract the content of the package so I can execute it directly? I can access the content with 7-zip but that gives me an installer file not the main executable.


Annoyingly both the Windows and Linux versions are just the executables in an archive file, and only the FreeBSD version has been packed this way.
Title: Re: How do I extract content of a FreeBSD package?
Post by: franco on September 23, 2020, 10:23:08 am
What do you mean ".pkg" ? The file extension for packages is commonly ".txz" which is a ... tar archive :)

# tar -xzf -C /to/dir my.txz

See https://pkg.opnsense.org/FreeBSD:12:amd64/20.7/latest/All/

That being said pkg supports different tar types through libarchive, but the one used is txz for maximum compression.


Cheers,
Franco
Title: Re: How do I extract content of a FreeBSD package?
Post by: Taomyn on September 23, 2020, 10:26:54 am
What do you mean ".pkg" ?


It's for this: https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-freebsd.pkg
Title: Re: How do I extract content of a FreeBSD package?
Post by: fabian on September 23, 2020, 11:58:57 am
You can check what kind of compression was used using the file command but in case of pkg it is likely just a tar.
Title: Re: How do I extract content of a FreeBSD package?
Post by: franco on September 23, 2020, 12:05:23 pm
Yay, unsolicited random "package" links. I have no idea what we are looking at here.


Cheers,
Franco
Title: Re: How do I extract content of a FreeBSD package?
Post by: Taomyn on September 23, 2020, 12:10:47 pm
Yay, unsolicited random "package" links. I have no idea what we are looking at here.


Cheers,
Franco


Unsolicited? Random? I'm not a bot ;-)
Title: Re: How do I extract content of a FreeBSD package?
Post by: franco on September 23, 2020, 12:12:10 pm
The problem is:

1. not sure what this file is as it does not come from a FreeBSD repo nor does it look like one.
2. there is no context, download page, etc. describing what this does when keeping (1.) in mind.


Cheers,
Franco
Title: Re: How do I extract content of a FreeBSD package?
Post by: Taomyn on September 23, 2020, 12:17:55 pm
The problem is:

1. not sure what this file is as it does not come from a FreeBSD repo nor does it look like one.
2. there is no context, download page, etc. describing what this does when keeping (1.) in mind.


Cheers,
Franco


Isn't the context in the original post? This is the page link if it will make you happy https://www.speedtest.net/apps/cli


Anyway, back to the actual topic.


When I use 7-zip to extract it it reports:


Code: [Select]
Warning
Can not open the files as [Xar] archive
The file is open as [xz] archive


When I examine the file it starts with "+COMPACT_MANIFEST"


I'll see if I can find out more about this Xar format.
Title: Re: How do I extract content of a FreeBSD package?
Post by: franco on September 23, 2020, 12:24:55 pm
I must have not seen the link... ;)

This helps give context:

% sudo pkg update && sudo pkg install -g libidn2 ca_root_nss
% sudo pkg add "https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-freebsd.pkg"

So it's a package... This is simple then:

% fetch "https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-freebsd.pkg"
% tar tf download_file\?file_path=ookla-speedtest-1.0.0-freebsd.pkg
+COMPACT_MANIFEST
+MANIFEST
/usr/local/bin/speedtest
/usr/share/man/man5/speedtest.5

Look it has a binary and a manual page...

% pkg info -F download_file\?file_path=ookla-speedtest-1.0.0-freebsd.pkg
speedtest-1.0.0.2-1.5ae238b
Name           : speedtest
Version        : 1.0.0.2-1.5ae238b
Origin         : fpm/speedtest
Architecture   : (null)
Prefix         : /
Categories     :
Licenses       :
Maintainer     : support@ookla.com
WWW            : http://example.com/no-uri-given
Comment        : no description given
Annotations    :
Flat size      : 0.00B
Description    :
no description given

% pkg info -l -F download_file\?file_path=ookla-speedtest-1.0.0-freebsd.pkg
speedtest-1.0.0.2-1.5ae238b:
   /usr/local/bin/speedtest
   /usr/share/man/man5/speedtest.5


Cheers,
Franco
Title: Re: How do I extract content of a FreeBSD package?
Post by: Taomyn on September 23, 2020, 12:32:51 pm
Thank-you Franco, that should give something to play with
Title: Re: How do I extract content of a FreeBSD package?
Post by: Taomyn on September 23, 2020, 03:24:46 pm
Sorry to be a bother, but do you know why this:


Code: [Select]
set test_output_1='"Eltrona Interdiffusion s.a. - Luxembourg","10779","1.673","0.586","0","60262263","32162522","564697440","243358734","https://www.speedtest.net/result/c/d3cb54fb-6f9f-4dca-ac5f-3fa5798e06d3"'
echo $test_output_1 | awk -v FPAT='([^,]*)|(\"[^\"]+\")' '{for (i=1; i<=NF;i++) print i, "<" $i ">"}'


works on my Federa machine



Code: [Select]
1 <"Eltrona Interdiffusion s.a. - Luxembourg">
2 <"10779">
3 <"1.673">
4 <"0.586">
5 <"0">
6 <"60262263">
7 <"32162522">
8 <"564697440">
9 <"243358734">
10 <"https://www.speedtest.net/result/c/d3cb54fb-6f9f-4dca-ac5f-3fa5798e06d3">


but on OPNsense it doesn't?


Code: [Select]
1 <"Eltrona>
2 <Interdiffusion>
3 <s.a.>
4 <->
5 <Luxembourg","10779","1.673","0.586","0","60262263","32162522","564697440","243358734","https://www.speedtest.net/result/c/d3cb54fb-6f9f-4dca-ac5f-3fa5798e06d3">


I'm sure it's something simple and related to the shell, but it has me stumped.

Title: Re: How do I extract content of a FreeBSD package?
Post by: fabian on September 23, 2020, 04:19:04 pm
Maybe a different awk version. FreeBSD and GNU differ sometimes. You can use the GNU version of the tools using "g" as a prefix like "gmake" instead of "make". The reason behind is that the FreeBSD people don't like the GPL so the internals have been written from scratch which are the versions without the "g". The GNU versions are likely not pre-installed so you may have to do that by yourself.
Title: Re: How do I extract content of a FreeBSD package?
Post by: Taomyn on September 24, 2020, 12:41:27 pm

So I switched from CSV to TSV thinking that maybe handling a tab character would be easier and as a test I just tried this and it still does not work:



Code: [Select]
echo $test_output_1 | awk '{gsub("\t","@")} {print}'
Is there actually something wrong with awk in the build because it's not working even for things it shows in it's man page:



Code: [Select]

SYNOPSIS
     awk [-safe] [-V] [-d[n]] [-F fs] [-v var=value] [prog | -f progfile]
         file ...


DESCRIPTION
     awk scans each input file for lines that match any of a set of patterns
     specified literally in prog or in one or more files specified as -f
     progfile.  With each pattern there can be an associated action that will
     be performed when a line of a file matches the pattern.  Each line is
     matched against the pattern portion of every pattern-action statement;
     the associated action is performed for each matched pattern.  The file
     name `-' means the standard input.  Any file of the form var=value is
     treated as an assignment, not a filename, and is executed at the time it
     would have been opened if it were a filename.


     The options are as follows:


     -d[n]   Debug mode.  Set debug level to n, or 1 if n is not specified.  A
             value greater than 1 causes awk to dump core on fatal errors.


     -F fs   Define the input field separator to be the regular expression fs.


     -f progfile
             Read program code from the specified file progfile instead of
             from the command line.


     -safe   Disable file output (print >, print >>), process creation (cmd |
             getline, print |, system) and access to the environment (ENVIRON;
             see the section on variables below).  This is a first (and not
             very reliable) approximation to a "safe" version of awk.


     -V      Print the version number of awk to standard output and exit.


     -v var=value
             Assign value to variable var before prog is executed; any number
             of -v options may be present.


     The input is normally made up of input lines (records) separated by
root@bart:/var/prtg/scriptsxml # awk -V
awk: unknown option -V ignored


awk: no program given