Help with Git hanging bugs

View: New views
8 Messages — Rating Filter:   Alert me  

Help with Git hanging bugs

by Chad Woolley :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

We've taken a shot at fixing this with a timeout loop, but it still
happens sometimes.    It only happens in certain scenarios, and is
hard (impossible for me so far) to reproduce.  I've recorded the info
I have in the comments of the ticket.  Any help or further info is
welcome:

https://cruisecontrolrb.lighthouseapp.com/projects/9150/tickets/229-sometimes-git-hangs

Thanks,
-- Chad
_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers

Re: Help with Git hanging bugs

by Chad Woolley :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, Jul 28, 2009 at 6:35 PM, Chad Woolley <thewoolleyman@...> wrote:
> We've taken a shot at fixing this with a timeout loop, but it still
> happens sometimes.    It only happens in certain scenarios, and is
> hard (impossible for me so far) to reproduce.  I've recorded the info
> I have in the comments of the ticket.  Any help or further info is
> welcome:
>
> https://cruisecontrolrb.lighthouseapp.com/projects/9150/tickets/229-sometimes-git-hangs

Matt Duncan has helped track this down, but the fix will involve the
use of the SystemTimer native gem.  I plan to use Bundler to manage
this.  Please see the ticket if you'd like to discuss:

https://cruisecontrolrb.lighthouseapp.com/projects/9150-cruise-control-rb/tickets/229-sometimes-git-hangs#ticket-229-14

-- Chad
_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers

Re: Help with Git hanging bugs

by Alexey Verkhovsky-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

All I can say is: ugh, timeout.rb rearing its ugly head again... :(

I don't like the sound of automaticaly installing native gems. This screams "maintenance nightmare" to me (think Windoze). Why don't we leave "SystemTimer not installed" as a fairly visible warning (written to the dashboard, and linked to a documentation page that explains the whole thing, perhaps)? This way, if a build gets stuck, people will easily find out why and take corrective action.

--Alex

On Mon, Oct 5, 2009 at 12:16 AM, Chad Woolley <thewoolleyman@...> wrote:
Matt Duncan has helped track this down, but the fix will involve the
use of the SystemTimer native gem.  I plan to use Bundler to manage
this.  Please see the ticket if you'd like to discuss:
 

_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers

Re: Help with Git hanging bugs

by Chad Woolley :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Oct 5, 2009 at 9:03 AM, Alexey Verkhovsky
<alexey.verkhovsky@...> wrote:
> I don't like the sound of automaticaly installing native gems. This screams
> "maintenance nightmare" to me (think Windoze). Why don't we leave
> "SystemTimer not installed" as a fairly visible warning (written to the
> dashboard, and linked to a documentation page that explains the whole thing,
> perhaps)? This way, if a build gets stuck, people will easily find out why
> and take corrective action.

We definitely can and should do that, but I'd still like to have a
fail-safe installation attempt, just for my own selfish automation
purposes.
_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers

Re: Help with Git hanging bugs

by Alexey Verkhovsky-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Oct 5, 2009 at 10:53 AM, Chad Woolley <thewoolleyman@...> wrote:
We definitely can and should do that, but I'd still like to have a
fail-safe installation attempt

This would still suck on:

(1) computers behind paranoid firewalls (by adding network timeout to every attempt to start an instance, which may or may not be infinite).
(2) computers that don't have gcc or ruby-dev (increasing startup time)
(3) you'll need to address what happens when several CC processes start from the same install at the same time (because, say, I like to run builder processes separately from the dashboard).
(4) possibly, computers that run CC.rb under nologin accounts.
(5) for people who don't want the system_timer gem for any reason whatsoever.
(6) ???

Seriously, a shrink-wrapped product should not silently install stuff into any global library locations, IMHO.

--
Alexey Verkhovsky
http://alex-verkhovsky.blogspot.com/
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]


_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers

Re: Help with Git hanging bugs

by Chad Woolley :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Oct 5, 2009 at 10:08 AM, Alexey Verkhovsky
<alexey.verkhovsky@...> wrote:

> (1) computers behind paranoid firewalls (by adding network timeout to every
> attempt to start an instance, which may or may not be infinite).
> (2) computers that don't have gcc or ruby-dev (increasing startup time)
> (3) you'll need to address what happens when several CC processes start from
> the same install at the same time (because, say, I like to run builder
> processes separately from the dashboard).
> (4) possibly, computers that run CC.rb under nologin accounts.
> (5) for people who don't want the system_timer gem for any reason
> whatsoever.
> (6) ???
>
> Seriously, a shrink-wrapped product should not silently install stuff into
> any global library locations, IMHO.

Good points.  One note though - with Bundler, it doesn't install to a
global location, it is installed under vendor/gems.  We can also do a
platform check, plus there is a 'placebo' stub gem on Windows.

Maybe we can make the bundling automated, but only if there is an
"opt-in" flag in cruise config?  Or maybe do it on all supported
platforms unless there is an "opt-out" flag set?
_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers

Re: Help with Git hanging bugs

by Matt Duncan :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

As long as it isn't installing to a global location, my vote would be an opt-out for supported platforms.  As it is now, the timeout config option is basically broken if a system call is what's hanging.

On Mon, Oct 5, 2009 at 1:17 PM, Chad Woolley <thewoolleyman@...> wrote:
On Mon, Oct 5, 2009 at 10:08 AM, Alexey Verkhovsky
> (1) computers behind paranoid firewalls (by adding network timeout to every
> attempt to start an instance, which may or may not be infinite).
> (2) computers that don't have gcc or ruby-dev (increasing startup time)
> (3) you'll need to address what happens when several CC processes start from
> the same install at the same time (because, say, I like to run builder
> processes separately from the dashboard).
> (4) possibly, computers that run CC.rb under nologin accounts.
> (5) for people who don't want the system_timer gem for any reason
> whatsoever.
> (6) ???
>
> Seriously, a shrink-wrapped product should not silently install stuff into
> any global library locations, IMHO.

Good points.  One note though - with Bundler, it doesn't install to a
global location, it is installed under vendor/gems.  We can also do a
platform check, plus there is a 'placebo' stub gem on Windows.

Maybe we can make the bundling automated, but only if there is an
"opt-in" flag in cruise config?  Or maybe do it on all supported
platforms unless there is an "opt-out" flag set?
_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers


_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers

Re: Help with Git hanging bugs

by Alexey Verkhovsky-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Oct 5, 2009 at 11:17 AM, Chad Woolley <thewoolleyman@...> wrote:
One note though - with Bundler, it doesn't install to a
global location, it is installed under vendor/gems

OK, that's much easier to swallow :) Sorry for making a bogus argument.
 
Maybe we can make the bundling automated, but only if there is an
"opt-in" flag in cruise config?
 
That doesn't make sense (not any better than ).

Or maybe do it on all supported
platforms unless there is an "opt-out" flag set?
 
I could probably live with that. With my "skeptical tester" hat on, I still expect bad things to happen though. See points (1), (2), (3), (4), (5) and - especially! - (6). By the time we address all those, it may well be so much effort that it doesn't make sense. Please, think about it.

--
Alexey Verkhovsky
http://alex-verkhovsky.blogspot.com/
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]


_______________________________________________
Cruisecontrolrb-developers mailing list
Cruisecontrolrb-developers@...
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-developers