[jira] Created: (SOLR-1529) NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*

NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
------------------------------------------------------------------------------------

                 Key: SOLR-1529
                 URL: https://issues.apache.org/jira/browse/SOLR-1529
             Project: Solr
          Issue Type: Bug
    Affects Versions: 1.4
            Reporter: Asmodean


The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Asmodean updated SOLR-1529:
---------------------------

    Attachment: logger.patch

Simple patch that will avoid the NullPointerException, but doesn't fix the issue.

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar updated SOLR-1529:
----------------------------------------

    Fix Version/s: 1.5

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770987#action_12770987 ]

Hoss Man commented on SOLR-1529:
--------------------------------

I'm not at a machine where i can try to reproduce this, but i want to try and understand the severity ... looking at LogUPdateProcessorFactory via viewvc, I believe what this bug is reporting is that if a single request includes more then 8 (maxNumToLog) deleteByQuery directives, w/o including any deleteById directives, then the logger will generate an NPE.  (regardless of how many docs were deleted)

Which means something like this would probably be a way to recreate it in the example...

{noformat}
java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query></delete>"
{noformat}

... and adding a single <id> entry (even if bogus, or possibly blank) would work arround the problem....


{noformat}
java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><id /></delete>"

java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><id>NOT_A_REAL_ID</id></delete>"
{noformat}

...is that all correct?

Questions should influence whether we consider this a show stopper for 1.4...

 * Does the update succeed in spite of the NPE, or does the entire update fail?
 * Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)
 

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770987#action_12770987 ]

Hoss Man edited comment on SOLR-1529 at 10/28/09 4:56 PM:
----------------------------------------------------------

I'm not at a machine where i can try to reproduce this, but i want to try and understand the severity ... looking at LogUPdateProcessorFactory via viewvc, I believe what this bug is reporting is that if a single request includes more then 8 (maxNumToLog) deleteByQuery directives, w/o including any deleteById directives, then the logger will generate an NPE.  (regardless of how many docs were deleted)

Which means something like this would probably be a way to recreate it in the example...

{noformat}
java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
           <query>X</query><query>X</query><query>X</query><query>X</query>
           <query>X</query><query>X</query><query>X</query></delete>"
{noformat}

... and adding a single <id> entry (even if bogus, or possibly blank) would work arround the problem....


{noformat}
java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
             <query>X</query><query>X</query><query>X</query><query>X</query>
             <query>X</query><query>X</query><query>X</query><id /></delete>"

java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query>
             <query>X</query><query>X</query><query>X</query><query>X</query>
             <query>X</query><query>X</query><query>X</query><id>NOT_A_REAL_ID</id></delete>"
{noformat}

...is that all correct?

Questions should influence whether we consider this a show stopper for 1.4...

 * Does the update succeed in spite of the NPE, or does the entire update fail?
 * Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)
 




      was (Author: hossman):
    I'm not at a machine where i can try to reproduce this, but i want to try and understand the severity ... looking at LogUPdateProcessorFactory via viewvc, I believe what this bug is reporting is that if a single request includes more then 8 (maxNumToLog) deleteByQuery directives, w/o including any deleteById directives, then the logger will generate an NPE.  (regardless of how many docs were deleted)

Which means something like this would probably be a way to recreate it in the example...

{noformat}
java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query></delete>"
{noformat}

... and adding a single <id> entry (even if bogus, or possibly blank) would work arround the problem....


{noformat}
java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><id /></delete>"

java -Ddata=args -jar post.jar "<delete><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><query>X</query><id>NOT_A_REAL_ID</id></delete>"
{noformat}

...is that all correct?

Questions should influence whether we consider this a show stopper for 1.4...

 * Does the update succeed in spite of the NPE, or does the entire update fail?
 * Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)
 



 

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771004#action_12771004 ]

Yonik Seeley commented on SOLR-1529:
------------------------------------

bq. Does the update succeed in spite of the NPE, or does the entire update fail?

The deletes should still succeed - the logger chains before it does stuff.

bq.  Is this bug new in 1.4, or will it also occur in 1.3 (skimming viewvc for the 1.3 tag it looks like this bug has been around for a while)

It does indeed look like it's been around since the beginning (7/07)

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771228#action_12771228 ]

Yonik Seeley commented on SOLR-1529:
------------------------------------

In case it wasn't clear - I don't consider this a 1.4 showstopper.

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771372#action_12771372 ]

Asmodean commented on SOLR-1529:
--------------------------------

Just to clarify from our point of view:

What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors or exceptions, we simply get an exception and have to rollback. It might for example look like the following sample code:

PSEUDO-CODE

try {
  while (hasMoreDeletesToProcessFromOurApplication) {
    if (requestBatchMaxSizeReached) {
      break;
    }

    request.deleteByQuery(...)
  }

  request.process(solrServer)
  solServer.commit()
} catch (Exception e) {
  solrServer.rollback();
  throw e;
}

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771372#action_12771372 ]

Asmodean edited comment on SOLR-1529 at 10/29/09 10:14 AM:
-----------------------------------------------------------

Just to clarify from our point of view:

What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors or exceptions, we simply get an exception and have to rollback. It might for example look like the following sample code:

PSEUDO-CODE

{code}

try {
  while (hasMoreDeletesToProcessFromOurApplication) {
    if (requestBatchMaxSizeReached) {
      break;
    }

    request.deleteByQuery(...)
  }

  request.process(solrServer)
  solServer.commit()
} catch (Exception e) {
  solrServer.rollback();
  throw e;
}

{code}

      was (Author: asmodean):
    Just to clarify from our point of view:

What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors or exceptions, we simply get an exception and have to rollback. It might for example look like the following sample code:

PSEUDO-CODE

try {
  while (hasMoreDeletesToProcessFromOurApplication) {
    if (requestBatchMaxSizeReached) {
      break;
    }

    request.deleteByQuery(...)
  }

  request.process(solrServer)
  solServer.commit()
} catch (Exception e) {
  solrServer.rollback();
  throw e;
}
 

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771372#action_12771372 ]

Asmodean edited comment on SOLR-1529 at 10/29/09 10:15 AM:
-----------------------------------------------------------

Just to clarify from our point of view:

What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors, we simply get an exception and have to rollback). It might for example look like the following sample code:

PSEUDO-CODE

{code}

try {
  while (hasMoreDeletesToProcessFromOurApplication) {
    if (requestBatchMaxSizeReached) {
      break;
    }

    request.deleteByQuery(...)
  }

  request.process(solrServer)
  solServer.commit()
} catch (Exception e) {
  solrServer.rollback();
  throw e;
}

{code}

      was (Author: asmodean):
    Just to clarify from our point of view:

What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors or exceptions, we simply get an exception and have to rollback. It might for example look like the following sample code:

PSEUDO-CODE

{code}

try {
  while (hasMoreDeletesToProcessFromOurApplication) {
    if (requestBatchMaxSizeReached) {
      break;
    }

    request.deleteByQuery(...)
  }

  request.process(solrServer)
  solServer.commit()
} catch (Exception e) {
  solrServer.rollback();
  throw e;
}

{code}
 

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771372#action_12771372 ]

Asmodean edited comment on SOLR-1529 at 10/29/09 10:17 AM:
-----------------------------------------------------------

Just to clarify from our point of view:

What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors, we simply get an exception and have to rollback). It might for example look like the following sample code:

PSEUDO-CODE

{code}

try {
  while (hasMoreDeletesToProcessFromOurApplication) {
    if (requestBatchMaxSizeReached) {
      break;
    }

    request.deleteByQuery(...)
  }

  request.process(solrServer)
  solServer.commit()
} catch (Exception e) {
  // SolrServerException or IOException

  solrServer.rollback();
  throw e;
}

{code}

      was (Author: asmodean):
    Just to clarify from our point of view:

What our client using the Solr web application see when it processes a request (... containing more than 8 *byQuery and no *byId) is an exception (we cant differentiate between types of errors, we simply get an exception and have to rollback). It might for example look like the following sample code:

PSEUDO-CODE

{code}

try {
  while (hasMoreDeletesToProcessFromOurApplication) {
    if (requestBatchMaxSizeReached) {
      break;
    }

    request.deleteByQuery(...)
  }

  request.process(solrServer)
  solServer.commit()
} catch (Exception e) {
  solrServer.rollback();
  throw e;
}

{code}
 

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yonik Seeley reassigned SOLR-1529:
----------------------------------

    Assignee: Yonik Seeley

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>            Assignee: Yonik Seeley
>             Fix For: 1.5
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/SOLR-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yonik Seeley resolved SOLR-1529.
--------------------------------

       Resolution: Fixed
    Fix Version/s:     (was: 1.5)
                   1.4

Thanks, I just committed your patch (though I had to apply it by hand).

> NullPointerException in LogUpdateProcessorFactory.java when deleting by query *only*
> ------------------------------------------------------------------------------------
>
>                 Key: SOLR-1529
>                 URL: https://issues.apache.org/jira/browse/SOLR-1529
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 1.4
>            Reporter: Asmodean
>            Assignee: Yonik Seeley
>             Fix For: 1.4
>
>         Attachments: logger.patch
>
>
> The problem occurs when a RequestUpdate has deletions that are all *byQuery (as opposed to *byId). The variable 'deletes' is in this case never initialized and will cause a NullPointerException in some cases (where the number of deletions are high enough).

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.