Package/environment management

[Justin Poliey]

There needs to be a much better way to manage Io packages, at least those that are Io-based. It's still pretty annoying to go through the whole make addons/make install process for developing addons with C source, but there needs to be an easier way to manage Io packages that use pure Io.

Here are some already existing projects:

http://github.com/oleganza/iopackage/
IoPackage is a package management system that works on Git repositories, and I don't doubt it could work with other packages too. It hasn't been touched since February, so I think it's going stale. Maybe a totally new project is best here?

http://github.com/jdp/boid
Boid is an environment management system that works like virtualenv (http://pypi.python.org/pypi/virtualenv). What it does is create isolated environments of Io packages, making it so you can use multiple versions of a package in different scenarios depending on your needs.

I also had the idea that IOIMPORT environment variable should be respected by Importer, and directories in that variable will be added to the search path for Io imports. There is already a pull request waiting for that.

What does everyone think?

[Samuel A. Falvo II]

On Mon, Nov 2, 2009 at 9:57 PM, Justin Poliey <jdp34@...> wrote:
> What does everyone think?

I haven't looked at the above, but I'm smitten with the approach
GoboLinux manages its own packages, and it could be applied to other
environments too.

Let's imagine we define an environment variable IOPKGS, which points
to the base path for all things Io-package related.

Then, for any given Io package P, you have ${IOPKGS}/P/${P_VERSION} as
a containing directory for the various source files and necessary
subdirectories (e.g., in the case of GoboLinux, you see bin, etc, lib,
usr, etc. in here).

Then, each has a symbolic link ${IOPKGS}/P/current which points to
${IOPKGS}/P/${P_VERSION_OF_YOUR_CHOICE}

In this manner, you are able to maintain as many different versions of
P as you want.

Next, we define a single directory containing symbolic links to all
the installed packages' binaries.

The package management system defines scripts (Io or shell, it doesn't
really matter) which refreshes these symbolic links.  This lets Io
look in a _single_ directory for all installed packages, while
preserving the ability to manage packages on their own terms.

The only disadvantage to this system, as described, is its dependency
on POSIX filesystem semantics.  You can work around this by, for
example, replacing that "big dump of symlinks" directory described
above with a simple text file, with each line pointing to a package
binary, etc.

Subdirectories would be supported, of course, so if you have
${IOPKGS}/P/bin/blah/foo.io, your symbolic link will point to
blah/foo.io and not just foo.io.  This eases namespace management.

--
Samuel A. Falvo II

[Justin Poliey]

> I haven't looked at the above, but I'm smitten with the approach
> GoboLinux manages its own packages, and it could be applied to other
> environments too.
>
> Let's imagine we define an environment variable IOPKGS, which points
> to the base path for all things Io-package related.
>
> Then, for any given Io package P, you have ${IOPKGS}/P/${P_VERSION} as
> a containing directory for the various source files and necessary
> subdirectories (e.g., in the case of GoboLinux, you see bin, etc, lib,
> usr, etc. in here).
>
> Then, each has a symbolic link ${IOPKGS}/P/current which points to
> ${IOPKGS}/P/${P_VERSION_OF_YOUR_CHOICE}
>
> In this manner, you are able to maintain as many different versions of
> P as you want.

And that works amazingly. But what if you want to have a specific
environment where a package depends on a specific older version of
another package though? You should be able to set up sandboxes for
each configuration.

I have to admit that my main concern when thinking about all this was
not necessarily packages in terms of tools and programs, but more like
pure-Io modules with the occasional binary. The current state of
making Io modules isn't very good, I feel it should be much easier.

> Next, we define a single directory containing symbolic links to all
> the installed packages' binaries.

Exactly, but instead of a multitude of symlinks to each individual
binary, how about a symlink to the path that contains all the
binaries, i.e., $IOPKGS/active/bin, where /active/ is a symlink to a
specific environment/sandbox.

> The package management system defines scripts (Io or shell, it doesn't
> really matter) which refreshes these symbolic links. This lets Io
> look in a _single_ directory for all installed packages, while
> preserving the ability to manage packages on their own terms.

I was thinking that would be $IOPKGS/active/io, with a directory
structure a little like:

Foo.io
Foo/
   Subfile.io
   AnotherSubfile.io
Bar.io
Bar/
   Subfile.io

Allowing Foo.io and Bar.io to include those relative files with
doRelativeFile, and the importer would search for the Foo and Bar
protos in that $IOPKGS/active/io top-level.

Cheers
Justin

[Danya Alexeyevsky]

On Tue, Nov 3, 2009 at 9:14 AM, Samuel A. Falvo II <sam.falvo@...> wrote:
> On Mon, Nov 2, 2009 at 9:57 PM, Justin Poliey <jdp34@...> wrote:
>> What does everyone think?
>
> I haven't looked at the above, but I'm smitten with the approach
> GoboLinux manages its own packages, and it could be applied to other
> environments too.

I totally agree with Justin that Io _Needs_ a more flexible package
managment system.

I'd rather support the idea of IOIMPORT (but I'd prefere it to be
called IOPATH to be consistent with PATH, LD_LIBRARY_PATH,
PYTHON_PATH, --ClassPath, etc). Approach with one directory with
symlinks works well for systems administrator, but does not work if
you have globally installed Io, you don't have root access and still
want to add some addons.

-- Danya.

[Samuel A. Falvo II]

On Mon, Nov 2, 2009 at 11:50 PM, Danya Alexeyevsky <me.dendik@...> wrote:
> I'd rather support the idea of IOIMPORT (but I'd prefere it to be
> called IOPATH to be consistent with PATH, LD_LIBRARY_PATH,
> PYTHON_PATH, --ClassPath, etc). Approach with one directory with
> symlinks works well for systems administrator, but does not work if
> you have globally installed Io, you don't have root access and still
> want to add some addons.

You are mistaken; you still need IOPATH to use the symlinked approach
too.  You can still have a centralized IOPKGS path and a user-specific
IOPKGS (or environment-specific, as others suggest).  Link those paths
into IOPATH as appropriate.

These need not be competing systems.

--
Samuel A. Falvo II

[Steve Dekorte]

On 2009-11-02, at 9:57 PM, Justin Poliey wrote:

> http://github.com/oleganza/iopackage/
> IoPackage is a package management system that works on Git  
> repositories, and I don't doubt it could work with other packages  
> too. It hasn't been touched since February, so I think it's going  
> stale. Maybe a totally new project is best here?
>
> http://github.com/jdp/boid
> Boid is an environment management system that works like virtualenv (http://pypi.python.org/pypi/virtualenv 
> ). What it does is create isolated environments of Io packages,  
> making it so you can use multiple versions of a package in different  
> scenarios depending on your needs.

I agree that one is needed and like the idea of it integrated with  
github. For example, you could type:

        iopackage johnsmith/foobar

and it would grab it from:

        git://github.com/johnsmith/foobar

compile and install it. Git versions/branches could also be supported,  
etc. If we can get the basics working, I'd be happy to move all the  
current addons into their own guthub repos.

Thoughts?

[Rich Collins]

I think you should group some addons into a standard library.  People  
using Io for the first time might be put off that they need to figure  
out how to "iopackage stevedekorte/socket" to fetch a url.

On Nov 3, 2009, at 5:08 PM, Steve Dekorte wrote:

>
> On 2009-11-02, at 9:57 PM, Justin Poliey wrote:
> > http://github.com/oleganza/iopackage/
> > IoPackage is a package management system that works on Git
> > repositories, and I don't doubt it could work with other packages
> > too. It hasn't been touched since February, so I think it's going
> > stale. Maybe a totally new project is best here?
> >
> > http://github.com/jdp/boid
> > Boid is an environment management system that works like  
> virtualenv (http://pypi.python.org/pypi/virtualenv
> > ). What it does is create isolated environments of Io packages,
> > making it so you can use multiple versions of a package in different
> > scenarios depending on your needs.
>
> I agree that one is needed and like the idea of it integrated with
> github. For example, you could type:
>
> iopackage johnsmith/foobar
>
> and it would grab it from:
>
> git://github.com/johnsmith/foobar
>
> compile and install it. Git versions/branches could also be supported,
> etc. If we can get the basics working, I'd be happy to move all the
> current addons into their own guthub repos.
>
> Thoughts?
>
>
>

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/iolanguage/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/iolanguage/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:iolanguage-digest@...
    mailto:iolanguage-fullfeatured@...

<*> To unsubscribe from this group, send an email to:
    iolanguage-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/