Prevalent Repository

View: New views
9 Messages — Rating Filter:   Alert me  

Prevalent Repository

by Diego Miranda :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi to all!

We are using Prevayler in several projects. For those projects we've created a Prevalent
Repository. This repository allows the objects to be queried and indexed with Lucene.
We provide a query and indexing abstraction to simplify the search of the objects stored
in the repository.

Here, some examples:

- Creating a Repository and storing data:

PrevalenceRepository prevalence = PrevalenceRepository.newInstance();
Repository repository = prevalence.openRepository ("MyApp");

repository.storeOrUpdate (person);
repository.storeOrUpdate (collectionOfPersons);

- Creating index:

repository.createIndex (Person.class, "addresses.state");
repository.createIndex (Person.class, "lastName");

- Querying the objects:

QueryCriteria query = from(Person.class).where(
            and(eq("addresses.street", "Melo"),eq("lastName", "Smith"))
            );
Collection<Person> persons = (Collection<Person>)repository.find(query);


- Deleting objects:

repository.delete ( personOid )

- Working with the repository data

repository.executeDeterministic ( new DeterministicOperations (), argument1, argument2 );


I'd like to know (if possible) what do you think about this!

Best regards!
Diego Miranda
diego.miranda at gmail dot com

--
Ley de Hofstadter: Siempre toma más tiempo del que se preveía, aun cuando se tenga en cuenta la Ley de Hofstadter.

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Prevalent Repository

by Justin T. Sampson-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Diego,

Sounds cool.

Does each method of PrevalenceRepository send a Query or Transaction to an underlying Prevayler instance?

Are the Luncene indexes stored in the same Prevayler, or in Lucene's own filesystem storage? Do you care about transactional consistency between the indexes and the data? (You might not.)

Cheers,
Justin

--
Agile Focus - http://agilefocus.com/


On Mon, May 18, 2009 at 2:03 PM, Diego Miranda <diego.miranda@...> wrote:
Hi to all!

We are using Prevayler in several projects. For those projects we've created a Prevalent
Repository. This repository allows the objects to be queried and indexed with Lucene.
We provide a query and indexing abstraction to simplify the search of the objects stored
in the repository.

Here, some examples:

- Creating a Repository and storing data:

PrevalenceRepository prevalence = PrevalenceRepository.newInstance();
Repository repository = prevalence.openRepository ("MyApp");

repository.storeOrUpdate (person);
repository.storeOrUpdate (collectionOfPersons);

- Creating index:

repository.createIndex (Person.class, "addresses.state");
repository.createIndex (Person.class, "lastName");

- Querying the objects:

QueryCriteria query = from(Person.class).where(
            and(eq("addresses.street", "Melo"),eq("lastName", "Smith"))
            );
Collection<Person> persons = (Collection<Person>)repository.find(query);


- Deleting objects:

repository.delete ( personOid )

- Working with the repository data

repository.executeDeterministic ( new DeterministicOperations (), argument1, argument2 );


I'd like to know (if possible) what do you think about this!

Best regards!
Diego Miranda
diego.miranda at gmail dot com


------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Prevalent Repository

by Diego Miranda :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Justin!

| Does each method of PrevalenceRepository send a Query or Transaction to an underlying Prevayler instance?

Yes, each method sends a Query or Transaction. There is an unique entry point to the repository. This is a Repository interface and has the following methods:

    @Query public Object getImmutableByOId (Integer id);
    @Query public int size();
    @Query public int size(Class<?> clazz);

    @Query public Object getByOId (Integer id);
    @Query public Collection<?> getAll();
    @Query public <T>Collection<T> getAll (Class<? extends T> clazz);
    @Query public Collection<Integer> getOIds();
    @Query public Collection<?> getObjectsByOId (Collection<Integer> ids);
    @Query public <T>Collection<T> find ( QueryCriteria<? extends T> query );
   
    @TransactionWithQuery public Object storeOrUpdate (Object object) throws RepositoryException;
    @TransactionWithQuery public Object storeOrUpdate (Object object, boolean index) throws RepositoryException;
    @Transaction public void storeOrUpdate (Collection<?> object, boolean index) throws RepositoryException;
   
    @Transaction public void reindex() throws RepositoryException;
    @Transaction public void reindex(Class<?> clazz) throws RepositoryException;
    @Transaction public void createIndex (Class<?> clazz, String indexPath);
    @Transaction public void createIndex (Class<?> clazz, String indexPath, boolean allowNulls);
    @TransactionWithQuery public Object delete (Integer oid) throws RepositoryException;
    @TransactionWithQuery public Object executeDeterministic (Deterministic action, Object ...args) throws Exception;
   
    public void takeSnapshot();

|Are the Luncene indexes stored in the same Prevayler, or in Lucene's own filesystem storage? Do you care about |transactional consistency between the indexes and the data? (You might not.)

The Lucene indexes are stored in the same Prevayler. We keep the consistency between the indexes and the data, but it is not mandatory. For example you can store several objects in the repository and later execute a reindex. By default when you store an object, the object (if you created an index for the object type) is indexed. At this moment, the only validation we are doing at indexing time is the null checking. For example:

Person p = new Person();
p.setName ("John");

repository.createIndex (Person.class, "lastName", false); // Index on Person.lastName, not nulls allowed
repository.storeOrUpdate (p); // Throws an exception, p is not stored in the repository.

To store an object and avoid the exception:

repository.createIndex (Person.class, "lastName", false); // Index on Person.lastName, not nulls allowed
repository.storeOrUpdate (p, false); // Store the object
repository.reindex(Person.class); // Throws an exception

On the other hand, when you delete an object, if the object type has an index definition, the index for that object is deleted.

Regards
Diego.


On Mon, May 18, 2009 at 8:48 PM, Justin T. Sampson <justin@...> wrote:
Hi Diego,

Sounds cool.

Does each method of PrevalenceRepository send a Query or Transaction to an underlying Prevayler instance?

Are the Luncene indexes stored in the same Prevayler, or in Lucene's own filesystem storage? Do you care about transactional consistency between the indexes and the data? (You might not.)

Cheers,
Justin

--
Agile Focus - http://agilefocus.com/



On Mon, May 18, 2009 at 2:03 PM, Diego Miranda <diego.miranda@...> wrote:
Hi to all!

We are using Prevayler in several projects. For those projects we've created a Prevalent
Repository. This repository allows the objects to be queried and indexed with Lucene.
We provide a query and indexing abstraction to simplify the search of the objects stored
in the repository.

Here, some examples:

- Creating a Repository and storing data:

PrevalenceRepository prevalence = PrevalenceRepository.newInstance();
Repository repository = prevalence.openRepository ("MyApp");

repository.storeOrUpdate (person);
repository.storeOrUpdate (collectionOfPersons);

- Creating index:

repository.createIndex (Person.class, "addresses.state");
repository.createIndex (Person.class, "lastName");

- Querying the objects:

QueryCriteria query = from(Person.class).where(
            and(eq("addresses.street", "Melo"),eq("lastName", "Smith"))
            );
Collection<Person> persons = (Collection<Person>)repository.find(query);


- Deleting objects:

repository.delete ( personOid )

- Working with the repository data

repository.executeDeterministic ( new DeterministicOperations (), argument1, argument2 );


I'd like to know (if possible) what do you think about this!

Best regards!
Diego Miranda
diego.miranda at gmail dot com


------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org




--
Ley de Hofstadter: Siempre toma más tiempo del que se preveía, aun cuando se toma en cuenta la Ley de Hofstadter.

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Prevalent Repository

by William Pietri :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Diego Miranda wrote:
> Hi to all!
>
> We are using Prevayler in several projects. For those projects we've
> created a Prevalent
> Repository. This repository allows the objects to be queried and
> indexed with Lucene.

That's very neat, Diego!

One of the most frequent questions I get when I tell people about
Prevayler is, "How do I find my objects again?" Because they're used to
writing SQL queries, they're uncomfortable with the thought of managing
their objects themselves. This seems like it would be perfect for people
like that. Not only does it give them an easy-to-use interface, but the
use of a familiar indexing technology should be reassuring as well.

Do you plan to publish that code?

William

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Prevalent Repository

by Diego Miranda :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi William,

Yes, our idea is to publish the code. We are doing some refactoring now and adding some functionality like encrypting the snapshots and journals, deterministics triggers, and listeners over the objects.

Thanks and regards :)
Diego


On Tue, May 19, 2009 at 1:01 PM, William Pietri <william@...> wrote:
Diego Miranda wrote:
> Hi to all!
>
> We are using Prevayler in several projects. For those projects we've
> created a Prevalent
> Repository. This repository allows the objects to be queried and
> indexed with Lucene.

That's very neat, Diego!

One of the most frequent questions I get when I tell people about
Prevayler is, "How do I find my objects again?" Because they're used to
writing SQL queries, they're uncomfortable with the thought of managing
their objects themselves. This seems like it would be perfect for people
like that. Not only does it give them an easy-to-use interface, but the
use of a familiar indexing technology should be reassuring as well.

Do you plan to publish that code?

William

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org



--
Ley de Hofstadter: Siempre toma más tiempo del que se preveía, aun cuando se toma en cuenta la Ley de Hofstadter.

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Prevalent Repository

by Justin T. Sampson-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, May 19, 2009 at 12:19 PM, Diego Miranda <diego.miranda@...> wrote:
Yes, our idea is to publish the code. We are doing some refactoring now and adding some functionality like encrypting the snapshots and journals, deterministics triggers, and listeners over the objects.

I've added some simple Serializer decorators for compression and encryption on the master branch[1], to be included in the upcoming Prevayler 2.4 release. I remember the main challenge I ran into when I first tried to compress and encrypt journals was that compression and encryption mechanisms don't necessarily make it easy to read or write exactly to variable-length record boundaries (which is how transactions are written to the journal). So the approach I ended up with simply wraps the journal serializer in order to compress or encrypt each individual transaction, allowing Prevayler to continue to do its normal parsing of the journal file structure.

[1] http://github.com/jsampson/prevayler/tree/master/core/src/main/java/org/prevayler/foundation/serialization

What approach are you taking for encrypting journals?

Cheers,
Justin

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Prevalent Repository

by Diego Miranda :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Well, actually we have made a similar approach, but we extends from JavaSerializer. I think that using a wrapper (as you did) is better because you can reuse other Serializers.

But, what we would like to do is to encrypt only those transactions that require to be encrypted. To achive this we have created two classes SecureAnnotatedTransaction and SecureAnnotatedTransactionWithQuery. Then the Serializer makes serialization according the type of transaction.

Cheers.
Diego

On Tue, May 19, 2009 at 6:16 PM, Justin T. Sampson <justin@...> wrote:
On Tue, May 19, 2009 at 12:19 PM, Diego Miranda <diego.miranda@...> wrote:
Yes, our idea is to publish the code. We are doing some refactoring now and adding some functionality like encrypting the snapshots and journals, deterministics triggers, and listeners over the objects.

I've added some simple Serializer decorators for compression and encryption on the master branch[1], to be included in the upcoming Prevayler 2.4 release. I remember the main challenge I ran into when I first tried to compress and encrypt journals was that compression and encryption mechanisms don't necessarily make it easy to read or write exactly to variable-length record boundaries (which is how transactions are written to the journal). So the approach I ended up with simply wraps the journal serializer in order to compress or encrypt each individual transaction, allowing Prevayler to continue to do its normal parsing of the journal file structure.

[1] http://github.com/jsampson/prevayler/tree/master/core/src/main/java/org/prevayler/foundation/serialization

What approach are you taking for encrypting journals?

Cheers,
Justin

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org




--
Ley de Hofstadter: Siempre toma más tiempo del que se preveía, aun cuando se toma en cuenta la Ley de Hofstadter.

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables
unlimited royalty-free distribution of the report engine
for externally facing server and web deployment.
http://p.sf.net/sfu/businessobjects
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Prevalent Repository

by Justin T. Sampson-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wed, May 20, 2009 at 8:13 AM, Diego Miranda <diego.miranda@...> wrote:
Well, actually we have made a similar approach, but we extends from JavaSerializer. I think that using a wrapper (as you did) is better because you can reuse other Serializers.

But, what we would like to do is to encrypt only those transactions that require to be encrypted. To achive this we have created two classes SecureAnnotatedTransaction and SecureAnnotatedTransactionWithQuery. Then the Serializer makes serialization according the type of transaction.

Ah, cool. I did have the thought that the versions I wrote would end up just being examples, because folks would run into performance issues and need specialized handling for their apps. So it's interesting to hear what you ended up doing.

Cheers,
Justin

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://www.creativitycat.com 
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org

Re: Prevalent Repository

by Klaus Wuestefeld :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> Ah, cool. I did have the thought that the versions I wrote would end up just
> being examples, because folks would run into performance issues and need
> specialized handling for their apps. So it's interesting to hear what you
> ended up doing.

Performance issues??? As long as it is done in parallel its ok.

People will have so many idle cores standing around they will actually
be grateful if we give them some work. ;)

See you, Klaus.

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://www.creativitycat.com 
_______________________________________________
To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion
_______________________________________________
"Databases in Memoriam" -- http://www.prevayler.org