An API to search public repositories

View: New views
7 Messages — Rating Filter:   Alert me  

An API to search public repositories

by Evgeny Goldin :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

I was just wondering how a lookup between all public repositories happens? Is there an API each repository responds to? Or one (in our case, Artifactory) just attempts to download and see if it fails or not ?
--
Best regards,
Evgeny
http://evgeny-goldin.com/

Re: An API to search public repositories

by Evgeny Goldin :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


Does it have anything to do with Maven Wagon ?

Evgeny Goldin wrote:
Hi,

I was just wondering how a lookup between all public repositories happens? Is there an API each repository responds to? Or one (in our case, Artifactory) just attempts to download and see if it fails or not ?
--
Best regards,
Evgeny
http://evgeny-goldin.com/

Re: An API to search public repositories

by Yoav Landman :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Evgeny,

When Artifactory contacts a remote repository it first sends a HEAD request to see whether the artifact exists and if it has changed (changes are only checkedĀ  for snapshots - the maven metadata is queried). Even though there are more powerful search APIs provided by repo managers, including Artifactory itself, this is the fastest direct query form, which is standard and works with 'dumb' HTTP repositories as well (in fact, HTTP also offers conditional-GET in a single request, but repo1 does not support it, so we use a HEAD then GET). Other than that, each remore repository has an expiring cache for snapshots, a cache for failed retrievals (network errors etc.) and a cache for non-found artifacts (all configurable from the remote repo advanced configuration panel).
The actual resolution order between repositories is explained here: http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories (the last section called "Resolution Order").

HTH,

Yoav

On Thu, Jul 2, 2009 at 10:34 AM, Evgeny Goldin <evgenyg@...> wrote:

Hi,

I was just wondering how a lookup between all public repositories happens?
Is there an API each repository responds to? Or one (in our case,
Artifactory) just attempts to download and see if it fails or not ?

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24302503.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
Artifactory-users@...
https://lists.sourceforge.net/lists/listinfo/artifactory-users


------------------------------------------------------------------------------

_______________________________________________
Artifactory-users mailing list
Artifactory-users@...
https://lists.sourceforge.net/lists/listinfo/artifactory-users

Re: An API to search public repositories

by Evgeny Goldin :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Great, Yoav, thanks a lot!

Yoav Landman wrote:
Hi Evgeny,

When Artifactory contacts a remote repository it first sends a HEAD request
to see whether the artifact exists and if it has changed (changes are only
checked  for snapshots - the maven metadata is queried). Even though there
are more powerful search APIs provided by repo managers, including
Artifactory itself, this is the fastest direct query form, which is standard
and works with 'dumb' HTTP repositories as well (in fact, HTTP also offers
conditional-GET in a single request, but repo1 does not support it, so we
use a HEAD then GET). Other than that, each remore repository has an
expiring cache for snapshots, a cache for failed retrievals (network errors
etc.) and a cache for non-found artifacts (all configurable from the remote
repo advanced configuration panel).
The actual resolution order between repositories is explained here:
http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories (the
last section called "Resolution Order").

HTH,

Yoav

On Thu, Jul 2, 2009 at 10:34 AM, Evgeny Goldin <evgenyg@gmail.com> wrote:

>
> Hi,
>
> I was just wondering how a lookup between all public repositories happens?
> Is there an API each repository responds to? Or one (in our case,
> Artifactory) just attempts to download and see if it fails or not ?
>
> --
> View this message in context:
> http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24302503.html
> Sent from the Artifactory-Users mailing list archive at Nabble.com.
>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Artifactory-users mailing list
> Artifactory-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/artifactory-users
>

------------------------------------------------------------------------------

_______________________________________________
Artifactory-users mailing list
Artifactory-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/artifactory-users
--
Best regards,
Evgeny
http://evgeny-goldin.com/

Re: An API to search public repositories

by Evgeny Goldin :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories:

The search order is agnostic to the order in which sub-repositories are listed inside a virtual repository.

But it's not agnostic to the order remote repositories are listed, right ?
--
Best regards,
Evgeny
http://evgeny-goldin.com/

Re: An API to search public repositories

by Yoav Landman :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Right. Updated the doc to reflect that.

Thanks!

On Sun, Jul 5, 2009 at 10:58 AM, Evgeny Goldin <evgenyg@...> wrote:

http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories:

The search order is agnostic to the order in which sub-repositories are
listed inside a virtual repository.

But it's not agnostic to the order remote repositories are
http://img30.imageshack.us/img30/8305/32047237.png listed , right ?

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24340916.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
Artifactory-users@...
https://lists.sourceforge.net/lists/listinfo/artifactory-users


------------------------------------------------------------------------------

_______________________________________________
Artifactory-users mailing list
Artifactory-users@...
https://lists.sourceforge.net/lists/listinfo/artifactory-users

Re: An API to search public repositories

by Yoav Landman :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

P.S. This is not true just for remotes, but also applies to the internal order of locals an other virtual repos within a virtual repo - the wiki doc explains this in more detail.

On Sun, Jul 5, 2009 at 9:53 PM, Yoav Landman <ylandman@...> wrote:
Right. Updated the doc to reflect that.

Thanks!


On Sun, Jul 5, 2009 at 10:58 AM, Evgeny Goldin <evgenyg@...> wrote:

http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories:

The search order is agnostic to the order in which sub-repositories are
listed inside a virtual repository.

But it's not agnostic to the order remote repositories are
http://img30.imageshack.us/img30/8305/32047237.png listed , right ?

--
View this message in context: http://www.nabble.com/An-API-to-search-public-repositories-tp24302503p24340916.html
Sent from the Artifactory-Users mailing list archive at Nabble.com.


------------------------------------------------------------------------------
_______________________________________________
Artifactory-users mailing list
Artifactory-users@...
https://lists.sourceforge.net/lists/listinfo/artifactory-users



------------------------------------------------------------------------------

_______________________________________________
Artifactory-users mailing list
Artifactory-users@...
https://lists.sourceforge.net/lists/listinfo/artifactory-users