file URLs a good idea for deployment?

View: New views
4 Messages — Rating Filter:   Alert me  

file URLs a good idea for deployment?

by Doug Douglass-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

We're transitioning from proximity (yes, very old!) to nexus. We've been running them both in parallel for several weeks now, so far so good.

Nexus, proximity and our CI server (CruiseControl) all run on the same host and as the same user.

We use a "ci-build" profile to override the scp:// deploymentUrl in our root pom and use local file:// URLs instead. This cut our CI build times in half.

Now, as we transition to nexus, we will replace scp:// with https:// deploymentUrl in our root pom, but what of the CI server?

Is it a good idea to use file:// deploymentUrl to deploy into nexus?

FYI, the CI artifacts are exclusively SNAPSHOT versions and we have unqiueVersion to false.

TIA,
Doug

Re: file URLs a good idea for deployment?

by Tamás Cservenák :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Doug,

as we always tell, you _may_ deploy "behind the back" to nexus storage
(like you described above, adding files using FS to nexus repo
storage), but is highly non-recommended.

Now, nexus will serve it up, but you have few caveats:

if the file gets into NFC (is requested before is deployed), nexus
will start to "refuse" it's serving. And with FS deployments, it will
be "unaware" of the fact that the file appeared, so you will need to
be careful with this.

You could have different tricks here, like using very short NFC TTL
(Not Found Cache TTL) and such.

Again, this might work, but could be unreliable and is not the recommended way?

Why do you dislike the http:// deployment? It is much-much snappier
that scm:// btw, is it only because of CI turnaround time?

~t~

On Thu, Jun 25, 2009 at 6:07 PM, Doug Douglass<douglass.doug@...> wrote:

> We're transitioning from proximity (yes, very old!) to nexus. We've been
> running them both in parallel for several weeks now, so far so good.
> Nexus, proximity and our CI server (CruiseControl) all run on the same host
> and as the same user.
> We use a "ci-build" profile to override the scp:// deploymentUrl in our root
> pom and use local file:// URLs instead. This cut our CI build times in half.
> Now, as we transition to nexus, we will replace scp:// with
> https:// deploymentUrl in our root pom, but what of the CI server?
>
> Is it a good idea to use file:// deploymentUrl to deploy into nexus?
> FYI, the CI artifacts are exclusively SNAPSHOT versions and we have
> unqiueVersion to false.
> TIA,
> Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: nexus-user-unsubscribe@...
For additional commands, e-mail: nexus-user-help@...


Re: file URLs a good idea for deployment?

by Doug Douglass-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks for the quick response Tamás and good points about our current "behind the back" tricks.

It's not that we dislike https deployments, in fact we haven't used this scheme yet.

We're a small team and we arrived at using file deployments for our CI builds since everything is running on the same server and as the same user, and the overhead of scp deployments was roughly doubling our build times.

Perhaps it's best for us to go ahead and use https deployment and see how this impacts our CI builds and only then determine if we need to use any tricks.

Thanks again,
Doug

2009/6/25 Tamás Cservenák <tamas@...>
Hi Doug,

as we always tell, you _may_ deploy "behind the back" to nexus storage
(like you described above, adding files using FS to nexus repo
storage), but is highly non-recommended.

Now, nexus will serve it up, but you have few caveats:

if the file gets into NFC (is requested before is deployed), nexus
will start to "refuse" it's serving. And with FS deployments, it will
be "unaware" of the fact that the file appeared, so you will need to
be careful with this.

You could have different tricks here, like using very short NFC TTL
(Not Found Cache TTL) and such.

Again, this might work, but could be unreliable and is not the recommended way?

Why do you dislike the http:// deployment? It is much-much snappier
that scm:// btw, is it only because of CI turnaround time?

~t~

On Thu, Jun 25, 2009 at 6:07 PM, Doug Douglass<douglass.doug@...> wrote:
> We're transitioning from proximity (yes, very old!) to nexus. We've been
> running them both in parallel for several weeks now, so far so good.
> Nexus, proximity and our CI server (CruiseControl) all run on the same host
> and as the same user.
> We use a "ci-build" profile to override the scp:// deploymentUrl in our root
> pom and use local file:// URLs instead. This cut our CI build times in half.
> Now, as we transition to nexus, we will replace scp:// with
> https:// deploymentUrl in our root pom, but what of the CI server?
>
> Is it a good idea to use file:// deploymentUrl to deploy into nexus?
> FYI, the CI artifacts are exclusively SNAPSHOT versions and we have
> unqiueVersion to false.
> TIA,
> Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: nexus-user-unsubscribe@...
For additional commands, e-mail: nexus-user-help@...



Re: file URLs a good idea for deployment?

by Tamás Cservenák :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Doug,

all I can say in "favor" of file:// deploy, is that you could _try_
(not sure about results, I am for weeks now on 1.4 "trunk" nexus :)
NFC TTL of 0. That would expire element in the moment it gets into
NFCache. But again, not sure will it work (will validation "eat it"?).

To test it:

1- set NFC TTL on the given hosted repo to 0 (if can at all).
2- use curl/wget to ask for a _known_ not present artifact (ie. wget
http://myhost/nexus/content/groups/public/foo/foo/1.0/foo-1.0.jar).
This should result in 404 of course. This will get artifact foo to
NFC.
3- _manually_ copy a dummy file to the place of the artifact above
(ie. touch ${nexus-work-dir}/storage/repoId/foo/foo/1.0/foo-1.0.jar).
This will create a 0 byte file. You can add some real content, but is
irrelevant for this test.
4- use again the same wget command from 2. It should retrieve
successfully whatever you put in there.

If these steps above prove as correct, that means NFC with value 0
will work for you. Simply taken, Nexus will serve the file if it is
present, will neglect NFC for that repo (actually will use NFC, but
all elems getting in there will already be expired).

This will end up with one potential problem: if your hosted repo is
accessed over some group (ie. public), and is _in_front_ of some large
repo, like central..... it may end up with lot of local FS lookups.
But if your team is small, the HW will be able to cope with this.

Again, this solution is definitely not recommended by us, Sonatype :)


~t~

On Thu, Jun 25, 2009 at 6:44 PM, Doug Douglass<douglass.doug@...> wrote:

> Thanks for the quick response Tamás and good points about our current
> "behind the back" tricks.
> It's not that we dislike https deployments, in fact we haven't used this
> scheme yet.
> We're a small team and we arrived at using file deployments for our CI
> builds since everything is running on the same server and as the same user,
> and the overhead of scp deployments was roughly doubling our build times.
> Perhaps it's best for us to go ahead and use https deployment and see how
> this impacts our CI builds and only then determine if we need to use any
> tricks.
> Thanks again,
> Doug
> 2009/6/25 Tamás Cservenák <tamas@...>
>>
>> Hi Doug,
>>
>> as we always tell, you _may_ deploy "behind the back" to nexus storage
>> (like you described above, adding files using FS to nexus repo
>> storage), but is highly non-recommended.
>>
>> Now, nexus will serve it up, but you have few caveats:
>>
>> if the file gets into NFC (is requested before is deployed), nexus
>> will start to "refuse" it's serving. And with FS deployments, it will
>> be "unaware" of the fact that the file appeared, so you will need to
>> be careful with this.
>>
>> You could have different tricks here, like using very short NFC TTL
>> (Not Found Cache TTL) and such.
>>
>> Again, this might work, but could be unreliable and is not the recommended
>> way?
>>
>> Why do you dislike the http:// deployment? It is much-much snappier
>> that scm:// btw, is it only because of CI turnaround time?
>>
>> ~t~
>>
>> On Thu, Jun 25, 2009 at 6:07 PM, Doug Douglass<douglass.doug@...>
>> wrote:
>> > We're transitioning from proximity (yes, very old!) to nexus. We've been
>> > running them both in parallel for several weeks now, so far so good.
>> > Nexus, proximity and our CI server (CruiseControl) all run on the same
>> > host
>> > and as the same user.
>> > We use a "ci-build" profile to override the scp:// deploymentUrl in our
>> > root
>> > pom and use local file:// URLs instead. This cut our CI build times in
>> > half.
>> > Now, as we transition to nexus, we will replace scp:// with
>> > https:// deploymentUrl in our root pom, but what of the CI server?
>> >
>> > Is it a good idea to use file:// deploymentUrl to deploy into nexus?
>> > FYI, the CI artifacts are exclusively SNAPSHOT versions and we have
>> > unqiueVersion to false.
>> > TIA,
>> > Doug
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: nexus-user-unsubscribe@...
>> For additional commands, e-mail: nexus-user-help@...
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: nexus-user-unsubscribe@...
For additional commands, e-mail: nexus-user-help@...