define and use multiple repository caches ?

[Helmut Denk]

hi gradle-users and developers,

for myCompanys ant+ivy-based buildsystem we use 2
repository caches. One for the SNAPSHOT-artefacts and
one for all the other artefacts. we do this because we
want to be able to run the command 'cleanSnapshotCache'
that wipes out all the cached SNAPSHOT-artefacts (including
metadata) but leaves all other cached artefacts untouched.

we learned that we need to wipe out SnapshotCache in some
situations because otherwise we get into trouble with the resolving
of our SNAPSHOT-artefacts. (-> multiple chained snapshot-repositories,
useOrigin, caching of ivy-metadata)

my question is: can i define and use multiple artefact-caches
with gradle too ? how ?

here the ivysettings.xml where you can see,
how we define and use the caches for native ivy:

(note: we know 3 BuildEnvironments: 'development', 'integration', 'release'
and use different ivysettings (resolver-chains) for each of them. what you
see here is the ivysettings.xml for buildEnv = 'development')  

<ivysettings>
	<settings
		defaultResolver="default-resolver"/>

	<caches useOrigin="${ivy-useOrigin}" lockStrategy="${ivy-lockStrategy}">
		<cache name="released"  basedir="${ivy-released-cache.dir}"/>
		<cache name="snapshot"  basedir="${ivy-snapshot-cache.dir}"/>
	</caches>
	
	<property name="ivy.local.default.root" value="${ivy-rep.dir}/local"/>
	<property name="ivy.local.default.ivy.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<property name="ivy.local.default.artifact.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<resolvers>
		<filesystem name="local" changingPattern=".*-SNAPSHOT" checkmodified="true" cache="snapshot">
			<ivy pattern="${ivy.local.default.root}/${ivy.local.default.ivy.pattern}" />
			<artifact pattern="${ivy.local.default.root}/${ivy.local.default.artifact.pattern}" />
		</filesystem>
	</resolvers>

	<property name="ivy.shared.default.root" value="${ivy-rep.dir}/shared"/>
	<property name="ivy.shared.default.ivy.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<property name="ivy.shared.default.artifact.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<resolvers>
		<filesystem name="shared" changingPattern=".*-SNAPSHOT" checkmodified="true" cache="snapshot">
			<ivy pattern="${ivy.shared.default.root}/${ivy.shared.default.ivy.pattern}" />
			<artifact pattern="${ivy.shared.default.root}/${ivy.shared.default.artifact.pattern}" />
		</filesystem>
	</resolvers>

	<property name="ivy.snapshot.default.root" value="${ivy-rep.url}/snapshot"/>
	<property name="ivy.snapshot.default.ivy.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<property name="ivy.snapshot.default.artifact.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<resolvers>
		<url name="snapshot" changingPattern=".*-SNAPSHOT" checkmodified="true" cache="snapshot">
			<ivy pattern="${ivy.snapshot.default.root}/${ivy.snapshot.default.ivy.pattern}" />
			<artifact pattern="${ivy.snapshot.default.root}/${ivy.snapshot.default.artifact.pattern}" />
		</url>
	</resolvers>

	<property name="ivy.released.default.root" value="${ivy-rep.url}/released"/>
	<property name="ivy.released.default.ivy.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<property name="ivy.released.default.artifact.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<resolvers>
		<url name="released" cache="released">
			<ivy pattern="${ivy.released.default.root}/${ivy.released.default.ivy.pattern}" />
			<artifact pattern="${ivy.released.default.root}/${ivy.released.default.artifact.pattern}" />
		</url>
	</resolvers>

	<property name="ivy.thirdparty.default.root" value="${ivy-rep.url}/thirdparty"/>
	<property name="ivy.thirdparty.default.ivy.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<property name="ivy.thirdparty.default.artifact.pattern" value="[type]s/[organisation]/[module]/[artifact]-[revision].[ext]"/>
	<resolvers>
		<url name="thirdparty" cache="released">
			<ivy pattern="${ivy.thirdparty.default.root}/${ivy.thirdparty.default.ivy.pattern}" />
			<artifact pattern="${ivy.thirdparty.default.root}/${ivy.thirdparty.default.artifact.pattern}" />
		</url>
	</resolvers>

	<resolvers>
		<chain name="default-resolver" dual="false" returnFirst="true">
			<resolver ref="local"/>
			<resolver ref="shared"/>
			<resolver ref="snapshot"/>
			<resolver ref="released"/>
			<resolver ref="thirdparty"/>
		</chain>
	</resolvers>
</ivysettings>

of course ... any way that makes the use of
multiple caches obsolete is preferred.

have a nice time

[hdockter]

On Jun 3, 2009, at 4:07 AM, Helmut Denk wrote:

>
> hi gradle-users and developers,
>
> for myCompanys ant+ivy-based buildsystem we use 2
> repository caches. One for the SNAPSHOT-artefacts and
> one for all the other artefacts. we do this because we
> want to be able to run the command 'cleanSnapshotCache'
> that wipes out all the cached SNAPSHOT-artefacts (including
> metadata) but leaves all other cached artefacts untouched.
>
> we learned that we need to wipe out SnapshotCache in some
> situations because otherwise we get into trouble with the resolving
> of our SNAPSHOT-artefacts. (-> multiple chained snapshot-repositories,
> useOrigin, caching of ivy-metadata)
>
> my question is: can i define and use multiple artefact-caches
> with gradle too

Yes

> ? how ?

Have a look at some internal Gradle code where we set the cache  
manager of a resolver:

http://svn.codehaus.org/gradle/gradle-core/branches/RB-0.6/src/main/groovy/org/gradle/api/internal/artifacts/ivyservice/DefaultResolverFactory.java

The interesting methods are createFlatDirResolver and  
createUseOriginCacheManager.

<snip>

>
> of course ... any way that makes the use of
> multiple caches obsolete is preferred.

Are those simply Ivy bugs? If so, are there Jira's for this?

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Helmut Denk]

hallo hans,

thanks for the hints. i had no time to step into this
subject so far ... but i will do so, because it's the final
step to get gradle 100% congruent with our ant+ivy
build-logik.

hdockter wrote:

Are those simply Ivy bugs? If so, are there Jira's for this?

hmm i don't know ... maybe bug maybe our miss-use or
miss-understanding of ivy.

we started with ivy in the 1.1.4  days ... i have to do
another round of investigations with ivy-2 -> tryout,
jira, ivy-mailinglist.

gruesse