Disappearing Items from a Collection

[mumbly]

I'm running into an issue where I add an object to a collection which
is a member of another collection. When retrieving the top level item
on a subsequent call, the new item is sometimes in the collection and
sometimes is not. I can confirm that it is getting saved to the
database. Let me map out a general outline of the code in question.

@Entity
@Table(name = "base")
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
@DiscriminatorColumn(name = "type", discriminatorType =
DiscriminatorType.INTEGER)
public abstract class Base  {
 protected Collection<Sub> sub;

 @OneToMany(mappedBy = "base", cascade = CascadeType.REMOVE, fetch =
FetchType.LAZY)
 public Collection<Sub> getSubs() {
   return this.subs;
 }

 public void setSubs(Collection<Sub> subs) {
   this.subs = subs;
 }

 public void addSub(Sub sub) {
   this.subs.add(sub);
   sub.setPost(this);
 }
}

@Entity
@DiscriminatorValue("2")
public class Top extends Base {
}

@Entity
@Table(name = "sub")
public class Sub {
   private Top top;

 @ManyToOne(cascade = {}, fetch = FetchType.EAGER)
 @JoinColumn(name = "base_id", referencedColumnName = "id")
 public Top getTop() {
   return this.top;
 }

 public void setTop(Top top) {
   this.top = top;
 }

public class SubDAO {
public static void createSub(Integer topId, Sub sub) {
try {
  persistenceSvc.beginTx();
  em = persistenceSvc.getEntityManager();
  Base base = em.find(Base.class, id);
  base.addSub(sub);
  em.persist(vote);
  persistenceSvc.commitTx();
  em.refresh(base);
  em.refresh(sub);
  } finally {
  persistenceSvc.close();
  }
}

I'm obviously leaving out quite a bit, due to space, but I can expand
as necessary. I only provide the general outline to help folks follow
what happens.

So I create the new object (Sub), add it to the Collection in the
Base. Later I retrieve the Top (extends Base) and sometimes it has the
new item and sometimes it does not (if I do a restart of the app
server, it always has the item). The code for retrieval looks a little
something like:

try {
  persistenceSvc.beginTx();
  em = persistenceSvc.getEntityManager();
  Top top = em.find(Top.class, topId);
  em.refresh(question);
  System.out.println("size: " + top.getSubs().size());
  persistenceSvc.commitTx();
  return top;
 } finally {
  persistenceSvc.close();
 }

I do the size to ensure that I instantiate the Collection. If I run
this code repeatedly after having added a single Sub to a Top that
already had a Sub in the collection, I'll see something like:

size: 2
size: 2
size: 2
size: 1
size: 2
size: 1

I wanted to make sure that I wasn't seeing a caching issue, so I added
the following in my persistence.xml:

<property name="eclipselink.cache.shared.default" value="false"/>
<property name="eclipselink.cache.size.default" value="0"/>
<property name="eclipselink.cache.type.default" value="None"/>

When I turn the logging up to FINE, I can see it doing the database
call each time I fetch

[EL Fine] .... SELECT ... FROM sub WHERE base_id = ?
  [123]

I can see that there are two items matching that query by doing it
directly to the database. Yet I get alternate return sizes of 1 and 2.
Again, if I restart the server, I'll only get 1. Only the newly added
entry appears and disappears.

Some details of my configuration. Tomcat 6.0.x. Eclipselink 1.1.2,
1.1.3, 1.2.0 (tried the newer versions when they came out to see if
there was a bug that had been solved). MySQL 5.1. JDBC connection,
both through JNDI and an explicit connection. I'm using RESOURCE_LOCAL
for the transaction-type. I set the eclipselink.target-database, but
that is it in terms of properties (expect as above when testing
without caching).

I was able to finally get correct behavior by changing the fetching to
EAGER instead of LAZY. But I don't want to have to do that and I'd
like to understand this unexpected behavior so I can avoid it in the
future.

I know there is probably not enough info for a definitive conclusion,
but I'm looking for suggestions as to where to look, what might be the
cause and whether or not there are known bugs which might result in
this behavior. I'm happy to turn up logging to try and trace things or
even set breakpoints in the code. If you need more information, let me
know.

I don't have a simple, reproducible test case or I'd file a bug.

Any help or tips would be greatly appreciated. Thanks.

--Tim
_______________________________________________
eclipselink-users mailing list
eclipselink-users@...
https://dev.eclipse.org/mailman/listinfo/eclipselink-users

[James Sutherland]

Very odd.

Setting,
<property name="eclipselink.cache.shared.default" value="false"/>

Is enough to disable the cache, do not set,
<property name="eclipselink.cache.size.default" value="0"/>
<property name="eclipselink.cache.type.default" value="None"/>

But, given it still occurs with the cache disabled, it does not appear to be a cache issue.  It seems to be a database/transactional issue.  You could confirm this by executing a native SQL query through the same EntityManager before calling size, to see what result you get back.

What does your,  persistenceSvc.beginTx() and persistenceSvc.getEntityManager() methods do?
If you call em.clear(); before calling the find() and size, do you get the same result?

You may want to try a different database, to confirm if it is a database issue.  Are you using innodb with MySQL, can you try using it?

Tim McNerney wrote:

I'm running into an issue where I add an object to a collection which
is a member of another collection. When retrieving the top level item
on a subsequent call, the new item is sometimes in the collection and
sometimes is not. I can confirm that it is getting saved to the
database. Let me map out a general outline of the code in question.

@Entity
@Table(name = "base")
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
@DiscriminatorColumn(name = "type", discriminatorType =
DiscriminatorType.INTEGER)
public abstract class Base  {
 protected Collection<Sub> sub;

 @OneToMany(mappedBy = "base", cascade = CascadeType.REMOVE, fetch =
FetchType.LAZY)
 public Collection<Sub> getSubs() {
   return this.subs;
 }

 public void setSubs(Collection<Sub> subs) {
   this.subs = subs;
 }

 public void addSub(Sub sub) {
   this.subs.add(sub);
   sub.setPost(this);
 }
}

@Entity
@DiscriminatorValue("2")
public class Top extends Base {
}

@Entity
@Table(name = "sub")
public class Sub {
   private Top top;

 @ManyToOne(cascade = {}, fetch = FetchType.EAGER)
 @JoinColumn(name = "base_id", referencedColumnName = "id")
 public Top getTop() {
   return this.top;
 }

 public void setTop(Top top) {
   this.top = top;
 }

public class SubDAO {
public static void createSub(Integer topId, Sub sub) {
try {
  persistenceSvc.beginTx();
  em = persistenceSvc.getEntityManager();
  Base base = em.find(Base.class, id);
  base.addSub(sub);
  em.persist(vote);
  persistenceSvc.commitTx();
  em.refresh(base);
  em.refresh(sub);
  } finally {
  persistenceSvc.close();
  }
}

I'm obviously leaving out quite a bit, due to space, but I can expand
as necessary. I only provide the general outline to help folks follow
what happens.

So I create the new object (Sub), add it to the Collection in the
Base. Later I retrieve the Top (extends Base) and sometimes it has the
new item and sometimes it does not (if I do a restart of the app
server, it always has the item). The code for retrieval looks a little
something like:

try {
  persistenceSvc.beginTx();
  em = persistenceSvc.getEntityManager();
  Top top = em.find(Top.class, topId);
  em.refresh(question);
  System.out.println("size: " + top.getSubs().size());
  persistenceSvc.commitTx();
  return top;
 } finally {
  persistenceSvc.close();
 }

I do the size to ensure that I instantiate the Collection. If I run
this code repeatedly after having added a single Sub to a Top that
already had a Sub in the collection, I'll see something like:

size: 2
size: 2
size: 2
size: 1
size: 2
size: 1

I wanted to make sure that I wasn't seeing a caching issue, so I added
the following in my persistence.xml:

<property name="eclipselink.cache.shared.default" value="false"/>
<property name="eclipselink.cache.size.default" value="0"/>
<property name="eclipselink.cache.type.default" value="None"/>

When I turn the logging up to FINE, I can see it doing the database
call each time I fetch

[EL Fine] .... SELECT ... FROM sub WHERE base_id = ?
  [123]

I can see that there are two items matching that query by doing it
directly to the database. Yet I get alternate return sizes of 1 and 2.
Again, if I restart the server, I'll only get 1. Only the newly added
entry appears and disappears.

Some details of my configuration. Tomcat 6.0.x. Eclipselink 1.1.2,
1.1.3, 1.2.0 (tried the newer versions when they came out to see if
there was a bug that had been solved). MySQL 5.1. JDBC connection,
both through JNDI and an explicit connection. I'm using RESOURCE_LOCAL
for the transaction-type. I set the eclipselink.target-database, but
that is it in terms of properties (expect as above when testing
without caching).

I was able to finally get correct behavior by changing the fetching to
EAGER instead of LAZY. But I don't want to have to do that and I'd
like to understand this unexpected behavior so I can avoid it in the
future.

I know there is probably not enough info for a definitive conclusion,
but I'm looking for suggestions as to where to look, what might be the
cause and whether or not there are known bugs which might result in
this behavior. I'm happy to turn up logging to try and trace things or
even set breakpoints in the code. If you need more information, let me
know.

I don't have a simple, reproducible test case or I'd file a bug.

Any help or tips would be greatly appreciated. Thanks.

--Tim

[James Sutherland]

Very odd.

Setting,
<property name="eclipselink.cache.shared.default" value="false"/>

Is enough to disable the cache, do not set,
<property name="eclipselink.cache.size.default" value="0"/>
<property name="eclipselink.cache.type.default" value="None"/>

But, given it still occurs with the cache disabled, it does not appear to be a cache issue.  It seems to be a database/transactional issue.  You could confirm this by executing a native SQL query through the same EntityManager before calling size, to see what result you get back.

What does your,  persistenceSvc.beginTx() and persistenceSvc.getEntityManager() methods do?
If you call em.clear(); before calling the find() and size, do you get the same result?

You may want to try a different database, to confirm if it is a database issue.  Are you using innodb with MySQL, can you try using it?

Tim McNerney wrote:

I'm running into an issue where I add an object to a collection which
is a member of another collection. When retrieving the top level item
on a subsequent call, the new item is sometimes in the collection and
sometimes is not. I can confirm that it is getting saved to the
database. Let me map out a general outline of the code in question.

@Entity
@Table(name = "base")
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
@DiscriminatorColumn(name = "type", discriminatorType =
DiscriminatorType.INTEGER)
public abstract class Base  {
 protected Collection<Sub> sub;

 @OneToMany(mappedBy = "base", cascade = CascadeType.REMOVE, fetch =
FetchType.LAZY)
 public Collection<Sub> getSubs() {
   return this.subs;
 }

 public void setSubs(Collection<Sub> subs) {
   this.subs = subs;
 }

 public void addSub(Sub sub) {
   this.subs.add(sub);
   sub.setPost(this);
 }
}

@Entity
@DiscriminatorValue("2")
public class Top extends Base {
}

@Entity
@Table(name = "sub")
public class Sub {
   private Top top;

 @ManyToOne(cascade = {}, fetch = FetchType.EAGER)
 @JoinColumn(name = "base_id", referencedColumnName = "id")
 public Top getTop() {
   return this.top;
 }

 public void setTop(Top top) {
   this.top = top;
 }

public class SubDAO {
public static void createSub(Integer topId, Sub sub) {
try {
  persistenceSvc.beginTx();
  em = persistenceSvc.getEntityManager();
  Base base = em.find(Base.class, id);
  base.addSub(sub);
  em.persist(vote);
  persistenceSvc.commitTx();
  em.refresh(base);
  em.refresh(sub);
  } finally {
  persistenceSvc.close();
  }
}

I'm obviously leaving out quite a bit, due to space, but I can expand
as necessary. I only provide the general outline to help folks follow
what happens.

So I create the new object (Sub), add it to the Collection in the
Base. Later I retrieve the Top (extends Base) and sometimes it has the
new item and sometimes it does not (if I do a restart of the app
server, it always has the item). The code for retrieval looks a little
something like:

try {
  persistenceSvc.beginTx();
  em = persistenceSvc.getEntityManager();
  Top top = em.find(Top.class, topId);
  em.refresh(question);
  System.out.println("size: " + top.getSubs().size());
  persistenceSvc.commitTx();
  return top;
 } finally {
  persistenceSvc.close();
 }

I do the size to ensure that I instantiate the Collection. If I run
this code repeatedly after having added a single Sub to a Top that
already had a Sub in the collection, I'll see something like:

size: 2
size: 2
size: 2
size: 1
size: 2
size: 1

I wanted to make sure that I wasn't seeing a caching issue, so I added
the following in my persistence.xml:

<property name="eclipselink.cache.shared.default" value="false"/>
<property name="eclipselink.cache.size.default" value="0"/>
<property name="eclipselink.cache.type.default" value="None"/>

When I turn the logging up to FINE, I can see it doing the database
call each time I fetch

[EL Fine] .... SELECT ... FROM sub WHERE base_id = ?
  [123]

I can see that there are two items matching that query by doing it
directly to the database. Yet I get alternate return sizes of 1 and 2.
Again, if I restart the server, I'll only get 1. Only the newly added
entry appears and disappears.

Some details of my configuration. Tomcat 6.0.x. Eclipselink 1.1.2,
1.1.3, 1.2.0 (tried the newer versions when they came out to see if
there was a bug that had been solved). MySQL 5.1. JDBC connection,
both through JNDI and an explicit connection. I'm using RESOURCE_LOCAL
for the transaction-type. I set the eclipselink.target-database, but
that is it in terms of properties (expect as above when testing
without caching).

I was able to finally get correct behavior by changing the fetching to
EAGER instead of LAZY. But I don't want to have to do that and I'd
like to understand this unexpected behavior so I can avoid it in the
future.

I know there is probably not enough info for a definitive conclusion,
but I'm looking for suggestions as to where to look, what might be the
cause and whether or not there are known bugs which might result in
this behavior. I'm happy to turn up logging to try and trace things or
even set breakpoints in the code. If you need more information, let me
know.

I don't have a simple, reproducible test case or I'd file a bug.

Any help or tips would be greatly appreciated. Thanks.

--Tim