[Discussion] Continuum 2.0 Roadmap

[Trygve Laugstøl apache]

Rahul Thakur wrote:
>
>> Overall I think the core of Continuum should be re-though to be more
>> pluggable. In particular a workflow engine should be in the middle of
>> the execution to orchestrate any steps involved with building a
>> project. This is one of the places where people should be able to plug
>> in their own steps and functionality.
> Is this close to what you are thinking?
> http://www.eclipse.org/alf/index.php

After a 30 second look, I don't think so. More like this:

   http://www.jboss.com/products/jbpm

--
Trygve

[brettporter]

Ok, I've created two wiki's:

http://cwiki.apache.org/confluence/display/CONTINUUM/Index
(exported to: http://cwiki.apache.org/CONTINUUM/)

This one is open to all users to edit, so is the traditional wiki/
cookbook area.

http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Index
(exported to: http://cwiki.apache.org/CONTINUUMDEV/)

This one is editable by developers only (accepts comments from  
anyone). This is for the roadmap and design docs. I only granted  
access to a few people that I could easily find - if you need to edit,  
just let me or a confluence admin know.

So once this is migrated, we can point the Codehaus ones at this.

I have not yet set up an rsync to people.apache.org - if there's a  
place we should house this under the Continuum site we can do that.  
Also, the template probably needs some modifications.

Cheers,
Brett

On 07/02/2008, at 6:04 AM, Rahul Thakur wrote:

>
> If everyone is happy to keep the history till date on codehaus wiki,  
> I can help copy stuff across to Apache wiki :-)
>
> Rahul
>
>
> Brett Porter wrote:
>> We can create such a wiki any time - the challenge is converting  
>> existing content. If someone is happy to lose history and do it by  
>> hand, it can be done straight away.
>>
>> On 06/02/2008, at 9:25 PM, Rahul Thakur wrote:
>>
>>> Some good points emerging from this discussion! :-)
>>>
>>> Would it be a nice idea to put following on wiki:
>>> 1)  State goals/philosophy for C2 in light of lessons learnt from  
>>> 1.x development - lean, mean, extensible (~add any other here~)
>>> 2)  Document *all* features/requirements we want to see in C2 on  
>>> wiki (even if they are already present in 1.x!).
>>> 3)  Draw a proposed architecture.
>>> 4)  Assign items in (1) a priority/weight. Add use-cases to each  
>>> item in (1) to determine this.
>>> 5)  Group the priortised requirements/features into milestones.
>>> 6)  Start cutting code.
>>>
>>> Thoughts?
>>>
>>> PS: Codehaus wiki seems to be very slow. Any chance we can have a  
>>> space created on Apache wiki? Or, I guess it will have to wait for  
>>> TLP vote.
>>>
>>> Cheers,
>>> Rahul
>>>
>>> Brett Porter wrote:
>>>> This looks very exciting, and agree with most of the thread that  
>>>> follows. I'm just going to reply in summary - most of my thoughts  
>>>> are actually non-technical :)
>>>>
>>>> Regarding databases: I don't think xml files are the solution  
>>>> (except for the configuration where it makes a lot more sense :)  
>>>> - the data needs to be queryable. I think Andy made a good point  
>>>> in his comment on the roadmap - we need to look at the actual  
>>>> problems. Here's what I think needs to be improved:
>>>> - better centralisation of access. The architecture of Continuum  
>>>> bleeds JDO decisions all through the code since you access lazy  
>>>> stuff for the first time in obscure places.
>>>> - I think this might be that the model is too complicated (sorry,  
>>>> my fault) - it assumes complex relationships are handled easily.  
>>>> It seems to be going ok these days, but I feel it would be hard  
>>>> to modify.
>>>> I haven't looked at Rahul's branch yet, but I think we should  
>>>> consider a more decoupled database (ie, lookup build results for  
>>>> a project but keep them separate in the model to avoid the need  
>>>> to lazyload 90% of the time), and more centralised database  
>>>> logic. I would consider JPA just because it gives more options in  
>>>> terms of an implementation. It is quite easy to use from a  
>>>> development standpoint. But we also need to consider what  
>>>> functionality is needed up front - I think high on the list needs  
>>>> to be migrations between versions. Also, We are probably going to  
>>>> need to store more data in the future, and be able to query it  
>>>> (particularly historical datapoints).
>>>>
>>>> On the container: I would prefer to move off Plexus simply  
>>>> because it's a moving target and it's a barrier to entry for new  
>>>> developers.
>>>>
>>>> Now, my more general observations. Firstly, the roadmap doesn't  
>>>> appear to have any features - these are all technology changes.  
>>>> Some of that might be cool and a feature in itself, but I think  
>>>> there needs to be a balance between evolution, features, and  
>>>> bugfixing. I would also emphasise that features should be  
>>>> creative new things Continuum can do (for which we've had plenty  
>>>> of ideas), not just catching up to other CI servers :)
>>>>
>>>> I think the first part of the roadmap is key - separating the  
>>>> layers out, and basically building Continuum to be lightweight  
>>>> and distributed from the ground up. I hope that's the focus of  
>>>> the development. Note this also impacts the database as it should  
>>>> store much less information on builder machines (it can ship  
>>>> history back to the main server).
>>>>
>>>> I also think that supporting plugins is a good idea - it has been  
>>>> a huge bonus in other apps and in Maven itself. I'd like to  
>>>> investigate using OSGi for this.
>>>>
>>>> But by far the biggest question I have is what happened to 1.2? I  
>>>> think Continuum needs to set a target to achieve, but get there  
>>>> in gradual steps that at each stage sees a production release.  
>>>> The long 1.1 cycle really set Continuum back - a lot of it was  
>>>> changing features, but there was also a lot of changing  
>>>> technologies :) I don't think Continuum will survive another year-
>>>> and-a-half release cycle. So the start could be to break all the  
>>>> actions out (plexus, not webwork) into services and add some  
>>>> features, then the next release could adjust the database model  
>>>> and add some other features. And as we split these things out we  
>>>> make sure they are nicely documented and tested.
>>>>
>>>> That's my thoughts :)
>>>>
>>>> Cheers,
>>>> Brett
>>>>
>>>> On 30/01/2008, at 9:34 AM, Emmanuel Venisse wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> I started a document [1] with my ideas about Continuum 2.
>>>>> As you can see in this doc, I want to add lot of things in the  
>>>>> next version.
>>>>>
>>>>> Feel free to comment on it.
>>>>>
>>>>>
>>>>> [1]
>>>>> http://docs.codehaus.org/display/CONTINUUM/Continuum+2.0+Design+Discussion
>>>>>
>>>>> Emmanuel
>>>>
>>>>
>>
>>

--
Brett Porter
brett@...
http://blogs.exist.com/bporter/

[Wendy Smoak-3]

On Feb 12, 2008 10:01 PM, Brett Porter <brett@...> wrote:
> Ok, I've created two wiki's:
...
> http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Index
> (exported to: http://cwiki.apache.org/CONTINUUMDEV/)
>
> This one is editable by developers only (accepts comments from
> anyone). This is for the roadmap and design docs. I only granted
> access to a few people that I could easily find - if you need to edit,
> just let me or a confluence admin know.

Why would we not want to allow the community to participate in roadmap
and design docs?

The only reason I can think of to restrict access is to make sure we
have a CLA for content we intend to redistribute.

--
Wendy

[brettporter]

On 13/02/2008, at 4:04 PM, Wendy Smoak wrote:

> On Feb 12, 2008 10:01 PM, Brett Porter <brett@...> wrote:
>> Ok, I've created two wiki's:
> ...
>> http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Index
>> (exported to: http://cwiki.apache.org/CONTINUUMDEV/)
>>
>> This one is editable by developers only (accepts comments from
>> anyone). This is for the roadmap and design docs. I only granted
>> access to a few people that I could easily find - if you need to  
>> edit,
>> just let me or a confluence admin know.
>
> Why would we not want to allow the community to participate in roadmap
> and design docs?
>
> The only reason I can think of to restrict access is to make sure we
> have a CLA for content we intend to redistribute.

Both good points - I was following what we had in Maven already - what  
do others think - shall we just open it up? Or do we not even need the  
DEV space?

- Brett

--
Brett Porter
brett@...
http://blogs.exist.com/bporter/

[rahul.thakur.xdev]

+1 as long as editing it requires a login :-)

Should I hold off the migration from Codehaus?

Rahul

On Feb 13, 2008 6:32 PM, Brett Porter <brett@...> wrote:

>
>
> On 13/02/2008, at 4:04 PM, Wendy Smoak wrote:
>
> > On Feb 12, 2008 10:01 PM, Brett Porter <brett@...> wrote:
> >> Ok, I've created two wiki's:
> > ...
> >> http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Index
> >> (exported to: http://cwiki.apache.org/CONTINUUMDEV/)
> >>
> >> This one is editable by developers only (accepts comments from
> >> anyone). This is for the roadmap and design docs. I only granted
> >> access to a few people that I could easily find - if you need to
> >> edit,
> >> just let me or a confluence admin know.
> >
> > Why would we not want to allow the community to participate in roadmap
> > and design docs?
> >
> > The only reason I can think of to restrict access is to make sure we
> > have a CLA for content we intend to redistribute.
>
> Both good points - I was following what we had in Maven already - what
> do others think - shall we just open it up? Or do we not even need the
> DEV space?
>
> - Brett
>
> --
> Brett Porter
> brett@...
>
> http://blogs.exist.com/bporter/
>
>

[brettporter]

no, permissions changes are non-destructive :)

On 13/02/2008, at 6:33 PM, Rahul Thakur wrote:

> +1 as long as editing it requires a login :-)
>
> Should I hold off the migration from Codehaus?
>
> Rahul
>
> On Feb 13, 2008 6:32 PM, Brett Porter <brett@...> wrote:
>>
>>
>> On 13/02/2008, at 4:04 PM, Wendy Smoak wrote:
>>
>>> On Feb 12, 2008 10:01 PM, Brett Porter <brett@...> wrote:
>>>> Ok, I've created two wiki's:
>>> ...
>>>> http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Index
>>>> (exported to: http://cwiki.apache.org/CONTINUUMDEV/)
>>>>
>>>> This one is editable by developers only (accepts comments from
>>>> anyone). This is for the roadmap and design docs. I only granted
>>>> access to a few people that I could easily find - if you need to
>>>> edit,
>>>> just let me or a confluence admin know.
>>>
>>> Why would we not want to allow the community to participate in  
>>> roadmap
>>> and design docs?
>>>
>>> The only reason I can think of to restrict access is to make sure we
>>> have a CLA for content we intend to redistribute.
>>
>> Both good points - I was following what we had in Maven already -  
>> what
>> do others think - shall we just open it up? Or do we not even need  
>> the
>> DEV space?
>>
>> - Brett
>>
>> --
>> Brett Porter
>> brett@...
>>
>> http://blogs.exist.com/bporter/
>>
>>

--
Brett Porter
brett@...
http://blogs.exist.com/bporter/

[Emmanuel Venisse-2]

Thanks Brett.

I'm +1 to open it.

Emmanuel

On Feb 13, 2008 8:43 AM, Brett Porter <brett@...> wrote:

> no, permissions changes are non-destructive :)
>
> On 13/02/2008, at 6:33 PM, Rahul Thakur wrote:
>
> > +1 as long as editing it requires a login :-)
> >
> > Should I hold off the migration from Codehaus?
> >
> > Rahul
> >
> > On Feb 13, 2008 6:32 PM, Brett Porter <brett@...> wrote:
> >>
> >>
> >> On 13/02/2008, at 4:04 PM, Wendy Smoak wrote:
> >>
> >>> On Feb 12, 2008 10:01 PM, Brett Porter <brett@...> wrote:
> >>>> Ok, I've created two wiki's:
> >>> ...
> >>>> http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Index
> >>>> (exported to: http://cwiki.apache.org/CONTINUUMDEV/)
> >>>>
> >>>> This one is editable by developers only (accepts comments from
> >>>> anyone). This is for the roadmap and design docs. I only granted
> >>>> access to a few people that I could easily find - if you need to
> >>>> edit,
> >>>> just let me or a confluence admin know.
> >>>
> >>> Why would we not want to allow the community to participate in
> >>> roadmap
> >>> and design docs?
> >>>
> >>> The only reason I can think of to restrict access is to make sure we
> >>> have a CLA for content we intend to redistribute.
> >>
> >> Both good points - I was following what we had in Maven already -
> >> what
> >> do others think - shall we just open it up? Or do we not even need
> >> the
> >> DEV space?
> >>
> >> - Brett
> >>
> >> --
> >> Brett Porter
> >> brett@...
> >>
> >> http://blogs.exist.com/bporter/
> >>
> >>
>
> --
> Brett Porter
> brett@...
> http://blogs.exist.com/bporter/
>
>

[rahul.thakur.xdev]

I have created an page for Continuum 2.0 related stuff (treat this as a
dashboard with links to related C2 docs).

http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Draft+-+Continuum+2.0

The other content will keep moving in the background.

Cheers,
Rahul


Emmanuel Venisse wrote:

> Thanks Brett.
>
> I'm +1 to open it.
>
> Emmanuel
>
> On Feb 13, 2008 8:43 AM, Brett Porter <brett@...> wrote:
>
>  
>> no, permissions changes are non-destructive :)
>>
>> On 13/02/2008, at 6:33 PM, Rahul Thakur wrote:
>>
>>    
>>> +1 as long as editing it requires a login :-)
>>>
>>> Should I hold off the migration from Codehaus?
>>>
>>> Rahul
>>>
>>> On Feb 13, 2008 6:32 PM, Brett Porter <brett@...> wrote:
>>>      
>>>> On 13/02/2008, at 4:04 PM, Wendy Smoak wrote:
>>>>
>>>>        
>>>>> On Feb 12, 2008 10:01 PM, Brett Porter <brett@...> wrote:
>>>>>          
>>>>>> Ok, I've created two wiki's:
>>>>>>            
>>>>> ...
>>>>>          
>>>>>> http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Index
>>>>>> (exported to: http://cwiki.apache.org/CONTINUUMDEV/)
>>>>>>
>>>>>> This one is editable by developers only (accepts comments from
>>>>>> anyone). This is for the roadmap and design docs. I only granted
>>>>>> access to a few people that I could easily find - if you need to
>>>>>> edit,
>>>>>> just let me or a confluence admin know.
>>>>>>            
>>>>> Why would we not want to allow the community to participate in
>>>>> roadmap
>>>>> and design docs?
>>>>>
>>>>> The only reason I can think of to restrict access is to make sure we
>>>>> have a CLA for content we intend to redistribute.
>>>>>          
>>>> Both good points - I was following what we had in Maven already -
>>>> what
>>>> do others think - shall we just open it up? Or do we not even need
>>>> the
>>>> DEV space?
>>>>
>>>> - Brett
>>>>
>>>> --
>>>> Brett Porter
>>>> brett@...
>>>>
>>>> http://blogs.exist.com/bporter/
>>>>
>>>>
>>>>        
>> --
>> Brett Porter
>> brett@...
>> http://blogs.exist.com/bporter/
>>
>>
>>    
>
>  

[Deng Ching-2]

Sorry, just caught up with my mails today..

Anyway, +1 on the things in the wiki. All the ideas are exciting :) As what
have been mentioned already in the thread, I agree that it would be easier
and more manageable to implement these plans in milestones not just in one
blow.

And if Continuum will be switching databases, I'd go with JPA. Comparing my
experience in using JPA and JPOX, I'd say that it has been more pleasant
with JPA. I also think switching from Plexus to a different framework would
attract more contributors as a lot of people outside the Maven community
aren't really very familiar on how to use Plexus.

Thanks,
Deng

On Jan 30, 2008 6:34 AM, Emmanuel Venisse <emmanuel@...> wrote:

> Hi
>
> I started a document [1] with my ideas about Continuum 2.
> As you can see in this doc, I want to add lot of things in the next
> version.
>
> Feel free to comment on it.
>
>
> [1]
> http://docs.codehaus.org/display/CONTINUUM/Continuum+2.0+Design+Discussion
>
> Emmanuel
>

[Erik Drolshammer-3]

Maria Odea Ching wrote:
> Anyway, +1 on the things in the wiki. All the ideas are exciting :) As what
> have been mentioned already in the thread, I agree that it would be easier
> and more manageable to implement these plans in milestones not just in one
> blow.

Smaller iterations and more frequent releases is a very good idea. If
the transition from 1.0.3 to 1.1 had been done in smaller iterations I
think Continuum would have been a lot more popular today than it is now.
(Frequent releases seem to be one of the things that attract people to
Hudson...)


> And if Continuum will be switching databases, I'd go with JPA. Comparing my
> experience in using JPA and JPOX, I'd say that it has been more pleasant
> with JPA. I also think switching from Plexus to a different framework would
> attract more contributors as a lot of people outside the Maven community
> aren't really very familiar on how to use Plexus.

If I could choose between Jpox and JPA and Plexus and Spring, I'd chosen
JPA and Spring in a heartbeat. However, if you haven't got any really
_need_ for functionality only provided by JPA or Spring/whatever, the
value added to Continuum compared to the cost of implementation is not
worth it IMHO.


I also want to address the wish to "attract more contributors". As some
of you might know, I have developed an extension to Continuum that I now
call Backward Compatibility Tester [1]. I thus feel that I can give
feedback with regards to how it is to start developing on Continuum.
IMHO the most serious showstoppers are

1. I'm unable to run tests that extend AbstractContinuumTest in my IDE
(IntelliJ)
2. I'm unable to run tests that extend AbstractContinuumTest in my IDE
(IntelliJ)
3. I'm unable to run tests that extend AbstractContinuumTest in my IDE
(IntelliJ)

4. Lack of documentation. E.g., I haven't found a single design
diagram/description that explains how the 30~ modules interact...

5. There are 30 or so modules in the same maven project.
5.1 The build is horribly slow (6min 28secs on a Intel Core2 6300 CPU @
1.86GHz with 2GB ram).
5.2 It is hard to grasp the underlaying architecture.
5.3. Are really all these modules needed to check out some code from a
repository and run mvn clean install?
Yes, I try to suggest that some of the functionality should be moved to
separate projects. Perhaps a plugin-based architecture is the solution,
but I think much can be achieved by refactoring to a smaller set of
libraries that different products can use. (This might be a first step
towards Continuum with distributed builds.)
Perhaps continuum-model, continuum-xmlrpc or maven-continuum-plugin can
be split out to separate maven projects?

6. JPOX is buggy, hard to debug and difficult to work with.


[1] https://bct.dev.java.net/ (only a prototype, not production ready)



--
Regards
Erik Drolshammer
Sherriff @ #continuum and #maven

[Rahul Thakur-4]

Hello Everyone,

I have re-organized the document on the cwiki.apache.org
http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Continuum+2.0+Roadmap*

*and, moved the items into their own child pages. I think we should have
a template to lend some structure to requirements captured and expand them.

Any suggestions?

Cheers,
Rahul


Emmanuel Venisse wrote:

> Hi
>
> I started a document [1] with my ideas about Continuum 2.
> As you can see in this doc, I want to add lot of things in the next version.
>
> Feel free to comment on it.
>
>
> [1]
> http://docs.codehaus.org/display/CONTINUUM/Continuum+2.0+Design+Discussion
>
> Emmanuel
>
>    

[rahul.thakur.xdev]

Hi,

I have re-organised and updated content related to Continuum 2.0 Roadmap
here:
http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Draft+-+Continuum+2.0+Roadmap

Would appreciate if others can review/update/comment as appropriate.

Also, I think we start cutting out concrete JIRA tasks from those
umbrella issues listed on that page above and assign them fix versions.

Thanks,

Rahul



Emmanuel Venisse wrote:

> Hi
>
> I started a document [1] with my ideas about Continuum 2.
> As you can see in this doc, I want to add lot of things in the next version.
>
> Feel free to comment on it.
>
>
> [1]
> http://docs.codehaus.org/display/CONTINUUM/Continuum+2.0+Design+Discussion
>
> Emmanuel
>
>    

[Emmanuel Venisse-2]

Thanks Rahul.

Emmanuel

On Feb 20, 2008 4:44 AM, Rahul Thakur <rahul.thakur.xdev@...> wrote:

> Hi,
>
> I have re-organised and updated content related to Continuum 2.0 Roadmap
> here:
>
> http://cwiki.apache.org/confluence/display/CONTINUUMDEV/Draft+-+Continuum+2.0+Roadmap
>
> Would appreciate if others can review/update/comment as appropriate.
>
> Also, I think we start cutting out concrete JIRA tasks from those
> umbrella issues listed on that page above and assign them fix versions.
>
> Thanks,
>
> Rahul
>
>
>
> Emmanuel Venisse wrote:
> > Hi
> >
> > I started a document [1] with my ideas about Continuum 2.
> > As you can see in this doc, I want to add lot of things in the next
> version.
> >
> > Feel free to comment on it.
> >
> >
> > [1]
> >
> http://docs.codehaus.org/display/CONTINUUM/Continuum+2.0+Design+Discussion
> >
> > Emmanuel
> >
> >
>
>

[rahul.thakur.xdev]

Another feature (rather features) that i would like to see is around
Change tracking/audit.

I would like to add to the feature list - integration with some of
popular Change management/ Bug tracking systems, such that user can see
issues fixed in a build.

On a related note, I think we are already capturing changes in the SCM
but we should present them in the separate view for more visibility.

WDYT?

Cheers,
Rahul


Emmanuel Venisse wrote:

> Hi
>
> I started a document [1] with my ideas about Continuum 2.
> As you can see in this doc, I want to add lot of things in the next version.
>
> Feel free to comment on it.
>
>
> [1]
> http://docs.codehaus.org/display/CONTINUUM/Continuum+2.0+Design+Discussion
>
> Emmanuel
>
>    

[brettporter]

On 21/02/2008, at 9:57 AM, Rahul Thakur wrote:

>
> Another feature (rather features) that i would like to see is around  
> Change tracking/audit.
>
> I would like to add to the feature list - integration with some of  
> popular Change management/ Bug tracking systems, such that user can  
> see issues fixed in a build.

I think just keep adding them and make this a collection page rather  
than a 2.0 page. Then they can gather volunteers, more info. Then  
split out the really focused ones that you're planning to work on  
right now.

>
>
> On a related note, I think we are already capturing changes in the  
> SCM but we should present them in the separate view for more  
> visibility.

+1

>
>
> WDYT?
>
> Cheers,
> Rahul
>
>
> Emmanuel Venisse wrote:
>> Hi
>>
>> I started a document [1] with my ideas about Continuum 2.
>> As you can see in this doc, I want to add lot of things in the next  
>> version.
>>
>> Feel free to comment on it.
>>
>>
>> [1]
>> http://docs.codehaus.org/display/CONTINUUM/Continuum+2.0+Design+Discussion
>>
>> Emmanuel
>>
>>
>

--
Brett Porter
brett@...
http://blogs.exist.com/bporter/

[Emmanuel Venisse-2]

+1, maybe as a UI plugin

Emmanuel

On Wed, Feb 20, 2008 at 11:57 PM, Rahul Thakur <rahul.thakur.xdev@...>
wrote:

>
> Another feature (rather features) that i would like to see is around
> Change tracking/audit.
>
> I would like to add to the feature list - integration with some of
> popular Change management/ Bug tracking systems, such that user can see
> issues fixed in a build.
>
> On a related note, I think we are already capturing changes in the SCM
> but we should present them in the separate view for more visibility.
>
> WDYT?
>
> Cheers,
> Rahul
>
>
> Emmanuel Venisse wrote:
> > Hi
> >
> > I started a document [1] with my ideas about Continuum 2.
> > As you can see in this doc, I want to add lot of things in the next
> version.
> >
> > Feel free to comment on it.
> >
> >
> > [1]
> >
> http://docs.codehaus.org/display/CONTINUUM/Continuum+2.0+Design+Discussion
> >
> > Emmanuel
> >
> >
>
>