Do you still want CTM?

[Stephen Montgomery-Smith-3]

I have just been informed by my system administrator that the computer
that creates the CTM deltas may be put behind a firewall.  There may be
ways to get around it, but it will be quite a hassle on my end.

My question is this - are there people who still depend on CTM for their
updates?  Or is it time to close down CTM?  Or is there someone else who
wants to take it over?

I should add that my system administrator is tremendously supportive
towards the FreeBSD project, and I am sure that he would be a big help
to finding ways to keep CTM going, if it turns out that enough people
really need it.  But I don't want to go through that headache if there
aren't enough people to warrant its continued existence.

Please reply to ctm-users, but this is being sent to ctm-announce as
well, just in case some people are there who are not on ctm-users.

Thanks, Stephen
_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Peter Jeremy-3]

On 2009-Aug-03 21:46:06 -0500, Stephen Montgomery-Smith <stephen@...> wrote:
>My question is this - are there people who still depend on CTM for their
>updates?  Or is it time to close down CTM?  Or is there someone else who
>wants to take it over?

I don't have CVSup access at work and currently rely on CTM delta
emails.  I also use CTM (via FTP) at home because that was easy to
setup and means my trees at home and work are synchronised.

I could switch to CVSup at home without too much hassle but loss of
CTM would be a serious hassle at work - the easiest solution would
probably be for me to setup something fairly similar to a CTM server
at home and mail it to work.

As for formally taking over CTM, I don't think I'm in a position to
do so but would appreciate some more details:
- How much disk space is required?
- How resource intensive is building the deltas?  If it's a dedicated
  box, what CPU/RAM does it have?
- What are the bandwidth requirements?
- How are the deltas forwarded out to the mail and FTP servers?

--
Peter Jeremy

[Stephen Montgomery-Smith-3]

Peter Jeremy wrote:
> On 2009-Aug-03 21:46:06 -0500, Stephen Montgomery-Smith <stephen@...> wrote:
>> My question is this - are there people who still depend on CTM for their
>> updates?  Or is it time to close down CTM?  Or is there someone else who
>> wants to take it over?
>
> I don't have CVSup access at work and currently rely on CTM delta
> emails.  I also use CTM (via FTP) at home because that was easy to
> setup and means my trees at home and work are synchronised.

I use CTM for synchronization, just as you do.

>
> I could switch to CVSup at home without too much hassle but loss of
> CTM would be a serious hassle at work - the easiest solution would
> probably be for me to setup something fairly similar to a CTM server
> at home and mail it to work.

It might be more efficient to use CVSUP at home, and then synchronize
with work using rsync over vpn (that is, if you have vpn available to you).

>
> As for formally taking over CTM, I don't think I'm in a position to
> do so but would appreciate some more details:
> - How much disk space is required?

?????? I am doing a "du -s" right now, but the directory structure of
the complete cvs is pretty complicated, and it is taking a long time to
finish.  For now, I am going to guess that it is perhaps a little less
than 10G.

> - How resource intensive is building the deltas?  If it's a dedicated
>   box, what CPU/RAM does it have?

I use my desktop, which is a fairly old DELL, 32 bit, 1G RAM, 2.6GHz.
But the big bottleneck is the hard drive.

When CTM is running, it really uses the disk heavily, and I try to time
it to when I am not wanting to use the computer for regular activities
(like surfing the web).  Each CTM run takes about 2 hours, and this
happens 3 times a day.  I use fairly modern SATA drives, and that makes
a huge difference.  Most of the time is spent updating cvs-cur.

> - What are the bandwidth requirements?

You need enough bandwidth to be able to cvsup 3 times a day, and so that
ftp-master can fetch the deltas.  The big deltas are the
cvs-cur*xEmpty's, which are about 1G each in size, and new ones are
created about once per month.

> - How are the deltas forwarded out to the mail and FTP servers?

The deltas are mailed directly from my computer to the mailing lists,
using sendmail.

The FTP servers get the deltas via ftp-master fetching them via rsync.
It does this about every 4 hours.

_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Andre Albsmeier]

On Tue, 04-Aug-2009 at 13:17:53 +1000, Peter Jeremy wrote:
> On 2009-Aug-03 21:46:06 -0500, Stephen Montgomery-Smith <stephen@...> wrote:
> >My question is this - are there people who still depend on CTM for their
> >updates?  Or is it time to close down CTM?  Or is there someone else who
> >wants to take it over?
>
> I don't have CVSup access at work and currently rely on CTM delta
> emails.  I also use CTM (via FTP) at home because that was easy to
> setup and means my trees at home and work are synchronised.

Same here ;-)

>
> I could switch to CVSup at home without too much hassle but loss of
> CTM would be a serious hassle at work - the easiest solution would

Exactly the same here. Additionally, I use a script which
applies the deltas and filters out changes and local
modifications. I am doing it this way since FreeBSD-2.1.5 ;-).

A few years ago, there was some disruption in the delta generation
and I had to use CVSup at home and carry stuff over at work
temporarily. It was a big hack and luckily delta generation was
restored. This showed me how much I loved CTM ;-).

If CTM went away the world would not end but I would miss
it heavily for sure.

Thanks a lot for all your efforts being put into CTM!

        -Andre

> probably be for me to setup something fairly similar to a CTM server
> at home and mail it to work.
>
> As for formally taking over CTM, I don't think I'm in a position to
> do so but would appreciate some more details:
> - How much disk space is required?
> - How resource intensive is building the deltas?  If it's a dedicated
>   box, what CPU/RAM does it have?
> - What are the bandwidth requirements?
> - How are the deltas forwarded out to the mail and FTP servers?
>
> --
> Peter Jeremy

_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Andre Albsmeier]

On Mon, 03-Aug-2009 at 23:03:59 -0500, Stephen Montgomery-Smith wrote:

> Peter Jeremy wrote:
> > On 2009-Aug-03 21:46:06 -0500, Stephen Montgomery-Smith <stephen@...> wrote:
> >...
>
> ?????? I am doing a "du -s" right now, but the directory structure of
> the complete cvs is pretty complicated, and it is taking a long time to
> finish.  For now, I am going to guess that it is perhaps a little less
> than 10G.
>
> > - How resource intensive is building the deltas?  If it's a dedicated
> >   box, what CPU/RAM does it have?
>
> I use my desktop, which is a fairly old DELL, 32 bit, 1G RAM, 2.6GHz.
> But the big bottleneck is the hard drive.
>
> When CTM is running, it really uses the disk heavily, and I try to time
> it to when I am not wanting to use the computer for regular activities
> (like surfing the web).  Each CTM run takes about 2 hours, and this
> happens 3 times a day.  I use fairly modern SATA drives, and that makes
> a huge difference.  Most of the time is spent updating cvs-cur.

Maybe there is room for improvement by spreading the work
over various disks intelligently. I don't know how the
delta generation works but I think stuff has to be checked
out and compared to the former version. So if one got three
disks:

1. holding the cvs Repository
2. checked out version #1
3. checked out version #2

things might get a big boost.

>
> > - What are the bandwidth requirements?
>
> You need enough bandwidth to be able to cvsup 3 times a day, and so that
> ftp-master can fetch the deltas.  The big deltas are the
> cvs-cur*xEmpty's, which are about 1G each in size, and new ones are
> created about once per month.

One could argue about the need of distributing cvs-cur and src-cur
and stay with the -STABLE branches and -ports. But this is just
my opinion and others' needs may be different ;-).

Also, it might be enough to run the process only once a day.

>
> > - How are the deltas forwarded out to the mail and FTP servers?
>
> The deltas are mailed directly from my computer to the mailing lists,
> using sendmail.

How many people are there on the lists? This might also give an
idea about the necessity supporting cvs-cur and src-cur.

Chances are low, but, anyway, I will ask a friend who is well
connected if we could arrange something...

Thanks,

        -Andre

>
> The FTP servers get the deltas via ftp-master fetching them via rsync.
> It does this about every 4 hours.
>
> _______________________________________________
> ctm-users@... mailing list
> http://lists.freebsd.org/mailman/listinfo/ctm-users
> To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

--
Failure is not an option -- it comes bundled with Windows.
_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Joe Kraft-3]

> On Tue, 04-Aug-2009 at 13:17:53 +1000, Peter Jeremy wrote:
> > On 2009-Aug-03 21:46:06 -0500, Stephen Montgomery-Smith
<stephen@...> wrote:
> > >My question is this - are there people who still depend on CTM for
their
> > >updates?  Or is it time to close down CTM?  Or is there someone else
who
> > >wants to take it over?
> >
> > I don't have CVSup access at work and currently rely on CTM delta
> > emails.  I also use CTM (via FTP) at home because that was easy to
> > setup and means my trees at home and work are synchronised.
>
> Same here ;-)

Me too.  No access through the firewall at work to run CVSup or FTP so
e-mail is the way to go.

>
> >
> > I could switch to CVSup at home without too much hassle but loss of
> > CTM would be a serious hassle at work - the easiest solution would
>
> Exactly the same here. Additionally, I use a script which
> applies the deltas and filters out changes and local
> modifications. I am doing it this way since FreeBSD-2.1.5 ;-).
>
> If CTM went away the world would not end but I would miss
> it heavily for sure.
>

Occasionally I drop an e-mail and have to transfer something manually from
home and it's a PITA for sure.

> Thanks a lot for all your efforts being put into CTM!
>
> -Andre
My thanks also.

Joe.

_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Tom Russo-2]

On Mon, Aug 03, 2009 at 08:46:06PM -0600, Stephen Montgomery-Smith <stephen@...> wrote:
> I have just been informed by my system administrator that the computer
> that creates the CTM deltas may be put behind a firewall.  There may be
> ways to get around it, but it will be quite a hassle on my end.
>
> My question is this - are there people who still depend on CTM for their
> updates?  Or is it time to close down CTM?  Or is there someone else who
> wants to take it over?

I still depend on CTM at work, where I can't do CVSup due to restrictive
firewall and no chance of getting it opened up for this purpose.

--
Thomas Russo      *  tvrusso@...          *  Tel: [phone]
Dept 1437, Electrical & Microsystems Modeling    *  FAX: [phone]
Mail Stop 0316, Sandia National Laboratories     * PAGE: [phone]
Albuquerque, NM 87185-0316                       * CELL: [phone]

Give a man a program and you frustrate him for a day.  Teach him to program
and you frustrate him for life.  --- Anonymous

             EXCEPTIONAL SERVICE IN THE NATIONAL INTEREST
  Sandia is a multiprogram laboratory operated for the United States
 Department of Energy by Sandia Corporation, a Lockheed Martin Company.


_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Stephen Montgomery-Smith-3]

Stephen Montgomery-Smith wrote:
> I have just been informed by my system administrator that the computer
> that creates the CTM deltas may be put behind a firewall.

I had more conversations with my systems administrator.

1.  This "firewall" will not come into effect for several months, so
there is no big hurry to get everything sorted out.

2.  The main problem I will have with the firewall is getting the deltas
out to the ftp sites.  Right now, ftp-master uses rsync to get the
deltas from my machine.  But I am told that if there were some site I
could rsync TO, then that would work just fine behind the firewall.
That is, I act as the client, and the freebsd site acts as the rsync
server, to which I copy the files.

Does anyone know if freebsd has any such mechanism?

Thanks, Stephen
_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Julian H. Stacey-3]

I use & like CTM 'cos it's effecient push technology, whereas CVSup is
pull technology.

With increasingly common cheap flat rate DSL / fast connections,
that's less important to more end users than once it was.

But I guess for those who may touch base briefly with a laptop, then
whizz off traveling somewhere, it could still be a bonus there too.

Plus if we ever really got mainstream interest in BSD,
CVSup servers wouldnt scale well to supply 100 times more clients,
whereas CTM to 100 times more recipients would make little difference.

Cheers,
Julian
--
Julian Stacey: BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com
  Mail ASCII plain text not HTML & Base64.      http://asciiribbon.org
  Virused Microsoft PCs cause spam.             http://berklix.com/free/
_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Peter Jeremy-3]

On 2009-Aug-04 10:21:46 -0500, Stephen Montgomery-Smith <stephen@...> wrote:
>1.  This "firewall" will not come into effect for several months, so
>there is no big hurry to get everything sorted out.

That's good.

>2.  The main problem I will have with the firewall is getting the deltas
>out to the ftp sites.  Right now, ftp-master uses rsync to get the
>deltas from my machine.  But I am told that if there were some site I
>could rsync TO, then that would work just fine behind the firewall.
>That is, I act as the client, and the freebsd site acts as the rsync
>server, to which I copy the files.
>
>Does anyone know if freebsd has any such mechanism?

I'd suggest writing to admin@... and/or hubs@... and
discuss it with them.

On 2009-Aug-03 23:03:59 -0500, Stephen Montgomery-Smith <stephen@...> wrote:
>The FTP servers get the deltas via ftp-master fetching them via rsync.
>It does this about every 4 hours.

Ths disk I/O would be a nuisance but that point rules me out - my
ISP's AUP doesn't allow me to run servers.

--
Peter Jeremy

[Stephen Montgomery-Smith-3]

Peter Jeremy wrote:
 > On 2009-Aug-04 10:21:46 -0500, Stephen Montgomery-Smith
<stephen@...> wrote:


 >> 2.  The main problem I will have with the firewall is getting the
deltas
 >> out to the ftp sites.  Right now, ftp-master uses rsync to get the
 >> deltas from my machine.  But I am told that if there were some site I
 >> could rsync TO, then that would work just fine behind the firewall.
 >> That is, I act as the client, and the freebsd site acts as the rsync
 >> server, to which I copy the files.
 >>
 >> Does anyone know if freebsd has any such mechanism?
 >
 > I'd suggest writing to admin@... and/or hubs@... and
 > discuss it with them.


I have sent a message to freebsd-hubs, and I am awaiting a reply.

Stephen

_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Stephen Montgomery-Smith-3]

Andre Albsmeier wrote:

> Maybe there is room for improvement by spreading the work
> over various disks intelligently. I don't know how the
> delta generation works but I think stuff has to be checked
> out and compared to the former version. So if one got three
> disks:
>
> 1. holding the cvs Repository
> 2. checked out version #1
> 3. checked out version #2
>
> things might get a big boost.

I have actually tried this, and it did help a little.  But switching
from PATA to SATA made the biggest difference.  And if I had some kind
of RAID, it would help enormously.


> One could argue about the need of distributing cvs-cur and src-cur
> and stay with the -STABLE branches and -ports. But this is just
> my opinion and others' needs may be different ;-).

The src-cur (which I actually use myself) puts rather little extra
overhead on the system.  It is like another stable branch.

The vast majority of the effort is spent on updating cvs-cur.  If I
discovered that no-one wants this branch, that would be a huge saving of
time.

> Also, it might be enough to run the process only once a day.
>
>>> - How are the deltas forwarded out to the mail and FTP servers?
>> The deltas are mailed directly from my computer to the mailing lists,
>> using sendmail.
>
> How many people are there on the lists? This might also give an
> idea about the necessity supporting cvs-cur and src-cur.
>
> Chances are low, but, anyway, I will ask a friend who is well
> connected if we could arrange something...
>
> Thanks,
>
> -Andre
>
>> The FTP servers get the deltas via ftp-master fetching them via rsync.
>> It does this about every 4 hours.
>>
>> _______________________________________________
>> ctm-users@... mailing list
>> http://lists.freebsd.org/mailman/listinfo/ctm-users
>> To unsubscribe, send any mail to "ctm-users-unsubscribe@..."
>

_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Peter Jeremy-3]

On 2009-Aug-04 18:00:29 -0500, Stephen Montgomery-Smith <stephen@...> wrote:
>The vast majority of the effort is spent on updating cvs-cur.  If I
>discovered that no-one wants this branch, that would be a huge saving of
>time.

Just to be difficult, that's the branch I use - I find it most
convenient to replicate the repository and then I can pull out
anything I want.

Maybe you can get some statistics on the number of subscribers to each
of the ctm mailing lists as well as statistics for ctm downloads via FTP.

--
Peter Jeremy

[Stephen Montgomery-Smith-3]

Peter Jeremy wrote:
> On 2009-Aug-04 18:00:29 -0500, Stephen Montgomery-Smith <stephen@...> wrote:
>> The vast majority of the effort is spent on updating cvs-cur.  If I
>> discovered that no-one wants this branch, that would be a huge saving of
>> time.
>
> Just to be difficult, that's the branch I use - I find it most
> convenient to replicate the repository and then I can pull out
> anything I want.

I'll keep cvs-cur.

>
> Maybe you can get some statistics on the number of subscribers to each
> of the ctm mailing lists as well as statistics for ctm downloads via FTP.

Someday, perhaps, I'll do this as well.
_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Klaus T Aehlig]

Dear Stephen,

> My question is this - are there people who still depend on CTM for their
> updates?

I guess I'm not technically depending on CTM, as I'm sure I might be able to
find other ways of achieving the same goal. Nevertheless I am using CTM, as it
is the most convenient way and the one best suited to my needs. So I certainly
don't want to lose CTM. Is there anything I can do to help keep it going?

Best,
Klaus


PS:
> Please reply to ctm-users, but this is being sent to ctm-announce as
> well, just in case some people are there who are not on ctm-users.

Please CC me on direct replies, as I'm not subscribed to ctm-users.
_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Julian H. Stacey-3]

> >The vast majority of the effort is spent on updating cvs-cur.  
> Just to be difficult, that's the branch I use - I find it most

My personal most used in order: ports-cur, cvs-cur, src-cur

>  as well as statistics for ctm downloads via FTP.

Difficult, likely not worth the effort: ... mirror sites.

Cheers,
Julian
--
Julian Stacey: BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com
  Mail ASCII plain text not HTML & Base64.      http://asciiribbon.org
  Virused Microsoft PCs cause spam.             http://berklix.com/free/
_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."

[Andre Albsmeier]

On Wed, 05-Aug-2009 at 12:39:46 +0200, Julian H. Stacey wrote:
> > >The vast majority of the effort is spent on updating cvs-cur.  
> > Just to be difficult, that's the branch I use - I find it most
>
> My personal most used in order: ports-cur, cvs-cur, src-cur

It seems we have to stick with all of them. My prefs, also
in order with the most important first, would be:

src-X (where X are the -STABLE branches), ports-cur, src-cur

        -Andre

>
> >  as well as statistics for ctm downloads via FTP.
>
> Difficult, likely not worth the effort: ... mirror sites.
>
> Cheers,
> Julian
_______________________________________________
ctm-users@... mailing list
http://lists.freebsd.org/mailman/listinfo/ctm-users
To unsubscribe, send any mail to "ctm-users-unsubscribe@..."