Random idea - jar preloading

[justinedelson]

I was upgrading a Hudson plugin to use a newer parent pom this evening when I had a random idea for a Nexus improvement that I figured I'd share... when I download a POM from Nexus, the odds are pretty good that I'm going to download the corresponding artifact (JAR, WAR, etc.) in relatively short order. So Nexus should automatically trigger a background download when it downloads a POM for which it doesn't have a corresponding artifact. Probably just the main artifact (picked up from the packaging type), but this could be configurable to include, for example, the source jar.

 

The reason this came up for me is that a Hudson plugin requires several dozen megabytes of artifacts and of course, I happened to attempt to do this upgrade while on an EVDO card. I managed to get through all of the POMs, but got stuck part of the way through the main 20MB dependency.

 

Then when I got home and triggered the remaining downloads, it seemed like Nexus's cache should already have the artifact in it.

 

It would also be interesting to think about Nexus fulfilling all dependencies. How deep you go here is the only question.

 

Anyway, food for thought.

 

Justin

[Brian E Fox]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Hi Justin,

We actually have an issue for this in the old .com jira. We do prefetch the hashes currently as a single pipelined request. Requesting the jars on a pom download usually would be a good idea but not every time a pom is read do you need the jars…particularly in ranges and with Mercury even less often (it reads the poms to build the dirty tree and then after resolution knows about the correct jars). More often than not though it would be the correct thing to do.

 

From: Edelson, Justin [mailto:Justin.Edelson@...]
Sent: Monday, January 12, 2009 7:28 PM
To: nexus-dev@...
Subject: [nexus-dev] Random idea - jar preloading

 

I was upgrading a Hudson plugin to use a newer parent pom this evening when I had a random idea for a Nexus improvement that I figured I'd share... when I download a POM from Nexus, the odds are pretty good that I'm going to download the corresponding artifact (JAR, WAR, etc.) in relatively short order. So Nexus should automatically trigger a background download when it downloads a POM for which it doesn't have a corresponding artifact. Probably just the main artifact (picked up from the packaging type), but this could be configurable to include, for example, the source jar.

 

The reason this came up for me is that a Hudson plugin requires several dozen megabytes of artifacts and of course, I happened to attempt to do this upgrade while on an EVDO card. I managed to get through all of the POMs, but got stuck part of the way through the main 20MB dependency.

 

Then when I got home and triggered the remaining downloads, it seemed like Nexus's cache should already have the artifact in it.

 

It would also be interesting to think about Nexus fulfilling all dependencies. How deep you go here is the only question.

 

Anyway, food for thought.

 

Justin