making sure archiva goes exactly where you want it to

[EJ Ciramella-3]

Hello list,
 
I'm struggling with a problem where other repositories that we're
proxing for are scanned for our internal things we've installed into our
own repository.
 
Is there an easy way to say, "if you're looking for package com.foo.bar,
only ever go to <internal repository>"?

[brettporter]

On 05/09/2009, at 12:10 AM, EJ Ciramella wrote:

> Hello list,
>
> I'm struggling with a problem where other repositories that we're
> proxing for are scanned for our internal things we've installed into  
> our
> own repository.
>
> Is there an easy way to say, "if you're looking for package  
> com.foo.bar,
> only ever go to <internal repository>"?

Not quite, but the reverse.

Add com/foo/bar/** to the blacklist for the repositories you don't  
want it to go to.

- Brett

[Wendy Smoak-3]

On Fri, Sep 4, 2009 at 7:10 AM, EJ Ciramella<eciramella@...> wrote:

> I'm struggling with a problem where other repositories that we're
> proxing for are scanned for our internal things we've installed into our
> own repository.
>
> Is there an easy way to say, "if you're looking for package com.foo.bar,
> only ever go to <internal repository>"?

You should be able to accomplish this by using white/black lists on
the proxy connectors.

http://archiva.apache.org/docs/1.2.2/adminguide/proxy-connectors.html

--
Wendy

[EJ Ciramella-3]

So that's kind of a pain, no?  Having to list things that should be
coming from the codehaus repository in all the blacklists of every other
repository?

Is this truly the only way to do this?

-----Original Message-----
From: Brett Porter [mailto:brett@...] On Behalf Of Brett
Porter
Sent: Friday, September 04, 2009 10:17 AM
To: users@...
Subject: Re: making sure archiva goes exactly where you want it to


On 05/09/2009, at 12:10 AM, EJ Ciramella wrote:

> Hello list,
>
> I'm struggling with a problem where other repositories that we're
> proxing for are scanned for our internal things we've installed into
> our own repository.
>
> Is there an easy way to say, "if you're looking for package
> com.foo.bar, only ever go to <internal repository>"?

Not quite, but the reverse.

Add com/foo/bar/** to the blacklist for the repositories you don't want
it to go to.

- Brett

[brettporter]

On 08/09/2009, at 10:33 PM, EJ Ciramella wrote:

> So that's kind of a pain, no?  Having to list things that should be
> coming from the codehaus repository in all the blacklists of every  
> other
> repository?
>
> Is this truly the only way to do this?

I see you're point and personally I'd rather all these rules were in a  
central location.

The best approach to the current situation is to use whitelists on  
every repository (in which case blacklists for those not in that list  
are not necessary). Only on repositories such as central where you  
generally want everything except corporate artifacts do I use a  
blacklist typically.

- Brett

>
> -----Original Message-----
> From: Brett Porter [mailto:brett@...] On Behalf Of Brett
> Porter
> Sent: Friday, September 04, 2009 10:17 AM
> To: users@...
> Subject: Re: making sure archiva goes exactly where you want it to
>
>
> On 05/09/2009, at 12:10 AM, EJ Ciramella wrote:
>
>> Hello list,
>>
>> I'm struggling with a problem where other repositories that we're
>> proxing for are scanned for our internal things we've installed into
>> our own repository.
>>
>> Is there an easy way to say, "if you're looking for package
>> com.foo.bar, only ever go to <internal repository>"?
>
> Not quite, but the reverse.
>
> Add com/foo/bar/** to the blacklist for the repositories you don't  
> want
> it to go to.
>
> - Brett

[EJ Ciramella-3]

How many repositories do you have mapped?  

We have four internal ones:

"bertha" - for our release and project branches
"snapshot" - unused for now, but will house, you guessed it, snapshots
"test" - for testing deployments
"thirdparty" - (proxy) for storing all thirdparty artifacts we need to
install

And maybe 6 - 8 remote repositories that feed "thirdparty"

Regularly, the codehaus ones get flaky and even tho we've set 2 second
timeouts, each third party artifact has to time out.  That's pretty
horrendous when a few projects have 20 DIRECT dependencies let alone
hundreds of transitive ones.

Does this paint a better picture?  I don't feel like the black/white
lists are what we need, especially when we have 100% of the artifacts
the build is looking for already in "thirdparty".  Is there a better way
to do this?  What we've been doing is when a remote repository gets
wonky, we'll remove it and let the artifacts get served from our local
stash, but that stinks if someone is actively adding new features to a
module and requires new/different dependnencies to be looked up and
downloaded.

How do people normally accomplish this kinda thing?

-----Original Message-----
From: Brett Porter [mailto:brett@...] On Behalf Of Brett
Porter
Sent: Tuesday, September 08, 2009 7:57 PM
To: users@...
Subject: Re: making sure archiva goes exactly where you want it to


On 08/09/2009, at 10:33 PM, EJ Ciramella wrote:

> So that's kind of a pain, no?  Having to list things that should be
> coming from the codehaus repository in all the blacklists of every
> other repository?
>
> Is this truly the only way to do this?

I see you're point and personally I'd rather all these rules were in a
central location.

The best approach to the current situation is to use whitelists on every
repository (in which case blacklists for those not in that list are not
necessary). Only on repositories such as central where you generally
want everything except corporate artifacts do I use a blacklist
typically.

- Brett

[brettporter]

I'm not sure why you have codehaus listed, since they are all synced  
to central - or do you not proxy central? Or are you using the  
snapshots repo?

Anyway, in this scenario I would white list org/codehaus/** (and any  
other GIDs you ned from there) on Codehaus' proxy connector so it only  
uses that flaky link when a new version of something from there. Make  
sure you set releases to "once" and snapshots to "never" so that it  
doesn't go out looking for metadata unnecessarily (or vice versa if  
you are configuring the codehaus snapshots repository).

That will limit it to requests for codehaus artifacts - there's not  
much more you can do if those connections are not good other use the  
connector disable button when it is particularly bad.

- Brett

On 09/09/2009, at 11:42 PM, EJ Ciramella wrote:

> How many repositories do you have mapped?
>
> We have four internal ones:
>
> "bertha" - for our release and project branches
> "snapshot" - unused for now, but will house, you guessed it, snapshots
> "test" - for testing deployments
> "thirdparty" - (proxy) for storing all thirdparty artifacts we need to
> install
>
> And maybe 6 - 8 remote repositories that feed "thirdparty"
>
> Regularly, the codehaus ones get flaky and even tho we've set 2 second
> timeouts, each third party artifact has to time out.  That's pretty
> horrendous when a few projects have 20 DIRECT dependencies let alone
> hundreds of transitive ones.
>
> Does this paint a better picture?  I don't feel like the black/white
> lists are what we need, especially when we have 100% of the artifacts
> the build is looking for already in "thirdparty".  Is there a better  
> way
> to do this?  What we've been doing is when a remote repository gets
> wonky, we'll remove it and let the artifacts get served from our local
> stash, but that stinks if someone is actively adding new features to a
> module and requires new/different dependnencies to be looked up and
> downloaded.
>
> How do people normally accomplish this kinda thing?
>
> -----Original Message-----
> From: Brett Porter [mailto:brett@...] On Behalf Of Brett
> Porter
> Sent: Tuesday, September 08, 2009 7:57 PM
> To: users@...
> Subject: Re: making sure archiva goes exactly where you want it to
>
>
> On 08/09/2009, at 10:33 PM, EJ Ciramella wrote:
>
>> So that's kind of a pain, no?  Having to list things that should be
>> coming from the codehaus repository in all the blacklists of every
>> other repository?
>>
>> Is this truly the only way to do this?
>
> I see you're point and personally I'd rather all these rules were in a
> central location.
>
> The best approach to the current situation is to use whitelists on  
> every
> repository (in which case blacklists for those not in that list are  
> not
> necessary). Only on repositories such as central where you generally
> want everything except corporate artifacts do I use a blacklist
> typically.
>
> - Brett

[EJ Ciramella-3]

Thanks for the tips - I'll try to get that going soon.

Why do we have codehaus repositories?  Is everything synced (sonar, glassfish, etc)?

I thought we used the snapshots as well, but i'm disconnected from the VPN atm so I can't check.



-----Original Message-----
From: Brett Porter on behalf of Brett Porter
Sent: Thu 9/10/2009 9:53 PM
To: users@...
Subject: Re: making sure archiva goes exactly where you want it to
 
I'm not sure why you have codehaus listed, since they are all synced  
to central - or do you not proxy central? Or are you using the  
snapshots repo?

Anyway, in this scenario I would white list org/codehaus/** (and any  
other GIDs you ned from there) on Codehaus' proxy connector so it only  
uses that flaky link when a new version of something from there. Make  
sure you set releases to "once" and snapshots to "never" so that it  
doesn't go out looking for metadata unnecessarily (or vice versa if  
you are configuring the codehaus snapshots repository).

That will limit it to requests for codehaus artifacts - there's not  
much more you can do if those connections are not good other use the  
connector disable button when it is particularly bad.

- Brett

On 09/09/2009, at 11:42 PM, EJ Ciramella wrote:

> How many repositories do you have mapped?
>
> We have four internal ones:
>
> "bertha" - for our release and project branches
> "snapshot" - unused for now, but will house, you guessed it, snapshots
> "test" - for testing deployments
> "thirdparty" - (proxy) for storing all thirdparty artifacts we need to
> install
>
> And maybe 6 - 8 remote repositories that feed "thirdparty"
>
> Regularly, the codehaus ones get flaky and even tho we've set 2 second
> timeouts, each third party artifact has to time out.  That's pretty
> horrendous when a few projects have 20 DIRECT dependencies let alone
> hundreds of transitive ones.
>
> Does this paint a better picture?  I don't feel like the black/white
> lists are what we need, especially when we have 100% of the artifacts
> the build is looking for already in "thirdparty".  Is there a better  
> way
> to do this?  What we've been doing is when a remote repository gets
> wonky, we'll remove it and let the artifacts get served from our local
> stash, but that stinks if someone is actively adding new features to a
> module and requires new/different dependnencies to be looked up and
> downloaded.
>
> How do people normally accomplish this kinda thing?
>
> -----Original Message-----
> From: Brett Porter [mailto:brett@...] On Behalf Of Brett
> Porter
> Sent: Tuesday, September 08, 2009 7:57 PM
> To: users@...
> Subject: Re: making sure archiva goes exactly where you want it to
>
>
> On 08/09/2009, at 10:33 PM, EJ Ciramella wrote:
>
>> So that's kind of a pain, no?  Having to list things that should be
>> coming from the codehaus repository in all the blacklists of every
>> other repository?
>>
>> Is this truly the only way to do this?
>
> I see you're point and personally I'd rather all these rules were in a
> central location.
>
> The best approach to the current situation is to use whitelists on  
> every
> repository (in which case blacklists for those not in that list are  
> not
> necessary). Only on repositories such as central where you generally
> want everything except corporate artifacts do I use a blacklist
> typically.
>
> - Brett