Question about the message "Indexing failed. Rolled back all changes."

[Bertie Shen]

Hi,

  When I use
http://localhost:8180/solr/admin/dataimport.jsp?handler=/dataimport to debug
the indexing config file, I always see the status message on the right part
<str name="">Indexing failed. Rolled back all changes.</str>, even the
indexing process looks to be successful. I am not sure whether you guys have
seen the same phenomenon or not.  BTW, I usually check the checkbox Clean
and sometimes check Commit box, and then click Debug Now button.

  Thanks.

[Shalin Shekhar Mangar]

On Sat, Nov 7, 2009 at 1:10 PM, Bertie Shen <bertie.shen@...> wrote:

>
>  When I use
> http://localhost:8180/solr/admin/dataimport.jsp?handler=/dataimport to
> debug
> the indexing config file, I always see the status message on the right part
> <str name="">Indexing failed. Rolled back all changes.</str>, even the
> indexing process looks to be successful. I am not sure whether you guys
> have
> seen the same phenomenon or not.  BTW, I usually check the checkbox Clean
> and sometimes check Commit box, and then click Debug Now button.
>
>

Do you see any exceptions in the logs?

--
Regards,
Shalin Shekhar Mangar.

[Bertie Shen]

No. I did not check the logs.

But  even after I successfully index data using
http://host:port/solr-example/dataimport?command=full-import&commit=true&clean=true,
do solr search which returns meaningful results, and then visit
http://host:port/solr-example/dataimport?command=status, I can see the
following result

<response>
-
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">1</int>
</lst>
-
<lst name="initArgs">
-
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</lst>
<str name="command">status</str>
<str name="status">idle</str>
<str name="importResponse"/>
-
<lst name="statusMessages">
<str name="Time Elapsed">0:2:11.426</str>
<str name="Total Requests made to DataSource">584</str>
<str name="Total Rows Fetched">1538</str>
<str name="Total Documents Skipped">0</str>
<str name="Full Dump Started">2009-11-09 23:54:41</str>
*<str name="">Indexing failed. Rolled back all changes.</str>*
<str name="Committed">2009-11-09 23:54:42</str>
<str name="Optimized">2009-11-09 23:54:42</str>
<str name="Rolledback">2009-11-09 23:54:42</str>
</lst>
-
<str name="WARNING">
This response format is experimental.  It is likely to change in the future.
</str>
</response>

On Mon, Nov 9, 2009 at 7:39 AM, Shalin Shekhar Mangar <
shalinmangar@...> wrote:

> On Sat, Nov 7, 2009 at 1:10 PM, Bertie Shen <bertie.shen@...> wrote:
>
> >
> >  When I use
> > http://localhost:8180/solr/admin/dataimport.jsp?handler=/dataimport to
> > debug
> > the indexing config file, I always see the status message on the right
> part
> > <str name="">Indexing failed. Rolled back all changes.</str>, even the
> > indexing process looks to be successful. I am not sure whether you guys
> > have
> > seen the same phenomenon or not.  BTW, I usually check the checkbox Clean
> > and sometimes check Commit box, and then click Debug Now button.
> >
> >
> Do you see any exceptions in the logs?
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

[Avlesh Singh]

>
> But  even after I successfully index data using http://host:port/solr-example/dataimport?command=full-import&commit=true&clean=true,
> do solr search which returns meaningful results
>
I am not sure what "meaningful" means. The full-import command starts an
asynchronous process to start re-indexing. The response that you get in
return to the above mentioned URL, (always) indicates that a full-import has
been started. It does NOT know about anything that might go wrong with the
process itself.

and then visit http://host:port/solr-example/dataimport?command=status, I
> can see thefollowing result ...
>
The status URL is the one which tells you what is going on with the process.
The message - "Indexing failed. Rolled back all changes" can come because of
multiple reasons - missing database drivers, incorrect sql queries, runtime
errors in custom transformers etc.

Start the full-import once more. Keep a watch on the Solr server log. If you
can figure out what's going wrong, great; otherwise, copy-paste the
exception stack-trace from the log file for specific answers.

Cheers
Avlesh

On Tue, Nov 10, 2009 at 1:32 PM, Bertie Shen <bertie.shen@...> wrote:

> No. I did not check the logs.
>
> But  even after I successfully index data using
> http://host:port
> /solr-example/dataimport?command=full-import&commit=true&clean=true,
> do solr search which returns meaningful results, and then visit
> http://host:port/solr-example/dataimport?command=status, I can see the
> following result
>
> <response>
> -
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">1</int>
> </lst>
> -
> <lst name="initArgs">
> -
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst>
> </lst>
> <str name="command">status</str>
> <str name="status">idle</str>
> <str name="importResponse"/>
> -
> <lst name="statusMessages">
> <str name="Time Elapsed">0:2:11.426</str>
> <str name="Total Requests made to DataSource">584</str>
> <str name="Total Rows Fetched">1538</str>
> <str name="Total Documents Skipped">0</str>
> <str name="Full Dump Started">2009-11-09 23:54:41</str>
> *<str name="">Indexing failed. Rolled back all changes.</str>*
> <str name="Committed">2009-11-09 23:54:42</str>
> <str name="Optimized">2009-11-09 23:54:42</str>
> <str name="Rolledback">2009-11-09 23:54:42</str>
> </lst>
> -
> <str name="WARNING">
> This response format is experimental.  It is likely to change in the
> future.
> </str>
> </response>
>
> On Mon, Nov 9, 2009 at 7:39 AM, Shalin Shekhar Mangar <
> shalinmangar@...> wrote:
>
> > On Sat, Nov 7, 2009 at 1:10 PM, Bertie Shen <bertie.shen@...>
> wrote:
> >
> > >
> > >  When I use
> > > http://localhost:8180/solr/admin/dataimport.jsp?handler=/dataimport to
> > > debug
> > > the indexing config file, I always see the status message on the right
> > part
> > > <str name="">Indexing failed. Rolled back all changes.</str>, even the
> > > indexing process looks to be successful. I am not sure whether you guys
> > > have
> > > seen the same phenomenon or not.  BTW, I usually check the checkbox
> Clean
> > > and sometimes check Commit box, and then click Debug Now button.
> > >
> > >
> > Do you see any exceptions in the logs?
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>

[yountod]

I'm getting the same thing.  The process runs, seemingly successfully, and I can even go to other SOLR pages pointing to the same server and pull queries against the index with these just-added entires.  But the response to the original import says "failed" and "rollback" both through the XML response and also in the logs.

Why is the process reporting failure and saying it did not commit/rolled back, when it actually succeeded in importing and indexing?  If it rolled back, as the logs say, I would expect to not be able to pull those rows out with new queries against the index.

Avlesh Singh wrote:

>
> But  even after I successfully index data using http://host:port/solr-example/dataimport?command=full-import&commit=true&clean=true,
> do solr search which returns meaningful results
>
I am not sure what "meaningful" means. The full-import command starts an
asynchronous process to start re-indexing. The response that you get in
return to the above mentioned URL, (always) indicates that a full-import has
been started. It does NOT know about anything that might go wrong with the
process itself.

and then visit http://host:port/solr-example/dataimport?command=status, I
> can see thefollowing result ...
>
The status URL is the one which tells you what is going on with the process.
The message - "Indexing failed. Rolled back all changes" can come because of
multiple reasons - missing database drivers, incorrect sql queries, runtime
errors in custom transformers etc.

Start the full-import once more. Keep a watch on the Solr server log. If you
can figure out what's going wrong, great; otherwise, copy-paste the
exception stack-trace from the log file for specific answers.

Cheers
Avlesh

On Tue, Nov 10, 2009 at 1:32 PM, Bertie Shen <[e-mail]> wrote:

> No. I did not check the logs.
>
> But  even after I successfully index data using
> http://host:port
> /solr-example/dataimport?command=full-import&commit=true&clean=true,
> do solr search which returns meaningful results, and then visit
> http://host:port/solr-example/dataimport?command=status, I can see the
> following result
>
> <response>
> -
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">1</int>
> </lst>
> -
> <lst name="initArgs">
> -
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst>
> </lst>
> <str name="command">status</str>
> <str name="status">idle</str>
> <str name="importResponse"/>
> -
> <lst name="statusMessages">
> <str name="Time Elapsed">0:2:11.426</str>
> <str name="Total Requests made to DataSource">584</str>
> <str name="Total Rows Fetched">1538</str>
> <str name="Total Documents Skipped">0</str>
> <str name="Full Dump Started">2009-11-09 23:54:41</str>
> *<str name="">Indexing failed. Rolled back all changes.</str>*
> <str name="Committed">2009-11-09 23:54:42</str>
> <str name="Optimized">2009-11-09 23:54:42</str>
> <str name="Rolledback">2009-11-09 23:54:42</str>
> </lst>
> -
> <str name="WARNING">
> This response format is experimental.  It is likely to change in the
> future.
> </str>
> </response>
>
> On Mon, Nov 9, 2009 at 7:39 AM, Shalin Shekhar Mangar <
> [e-mail]> wrote:
>
> > On Sat, Nov 7, 2009 at 1:10 PM, Bertie Shen <[e-mail]>
> wrote:
> >
> > >
> > >  When I use
> > > http://localhost:8180/solr/admin/dataimport.jsp?handler=/dataimport to
> > > debug
> > > the indexing config file, I always see the status message on the right
> > part
> > > <str name="">Indexing failed. Rolled back all changes.</str>, even the
> > > indexing process looks to be successful. I am not sure whether you guys
> > > have
> > > seen the same phenomenon or not.  BTW, I usually check the checkbox
> Clean
> > > and sometimes check Commit box, and then click Debug Now button.
> > >
> > >
> > Do you see any exceptions in the logs?
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>

[yountod]

The process initially completes with:

  <str name="Full Dump Started">2009-11-13 09:40:46</str>
  <str name="">Indexing completed. Added/Updated: 200000 documents. Deleted 0 documents.</str>


...but then it fails with:

  <str name="Full Dump Started">2009-11-13 09:40:46</str>
  <str name="">Indexing failed. Rolled back all changes.</str>
  <str name="Committed">2009-11-13 09:41:10</str>
  <str name="Optimized">2009-11-13 09:41:10</str>
  <str name="Rolledback">2009-11-13 09:41:10</str>


====================================================================
I think it may have something to do with this, which I found by using the DataImport.jsp:
====================================================================
(Thread.java:636) Caused by: java.sql.SQLException: Illegal value for setFetchSize(). at com.mysql.jdbc.Statement.setFetchSize(Statement.java:1864) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:242) ... 28 more</str>

[Bertie Shen]

Hey,

  I figured out why we always we have see <str name="">Indexing failed.
Rolled back all changes.</str>.  It is because we need a
dataimport.properties file at conf/, into which indexing will write a last
indexing time. Without that file, SolrWriter.java will put throw an
exception and Solr will have this misleading  <str name="">Indexing failed.
Rolled back all changes.</str>. output, although indexing is actually
successfully completed.

  I think we need to improve this functionality, at least documentation.

  There are one more thing that we need to pay attention to, i.e. we need to
make dataimport.properties writable by other users, otherwise,
last_index_time will not be written and the error message may still be
there.

On Fri, Nov 13, 2009 at 9:35 AM, yountod <tyoung@...> wrote:

>
> The process initially completes with:
>
>  <str name="Full Dump Started">2009-11-13 09:40:46</str>
>  <str name="">Indexing completed. Added/Updated: 200000 documents. Deleted
> 0 documents.</str>
>
>
> ...but then it fails with:
>
>  <str name="Full Dump Started">2009-11-13 09:40:46</str>
>   <str name="">Indexing failed. Rolled back all changes.</str>
>   <str name="Committed">2009-11-13 09:41:10</str>
>  <str name="Optimized">2009-11-13 09:41:10</str>
>  <str name="Rolledback">2009-11-13 09:41:10</str>
>
>
> ====================================================================
> I think it may have something to do with this, which I found by using the
> DataImport.jsp:
> ====================================================================
> (Thread.java:636) Caused by: java.sql.SQLException: Illegal value for
> setFetchSize(). at
> com.mysql.jdbc.Statement.setFetchSize(Statement.java:1864) at
>
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:242)
> ... 28 more</str>
>
>
>
> --
> View this message in context:
> http://old.nabble.com/Question-about-the-message-%22Indexing-failed.-Rolled-back-all--changes.%22-tp26242714p26340360.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>

[Lance Norskog-2]

This is definitely a bug. Please open a JIRA issue for this.

On Sat, Nov 21, 2009 at 10:53 AM, Bertie Shen <bertie.shen@...> wrote:

> Hey,
>
>  I figured out why we always we have see <str name="">Indexing failed.
> Rolled back all changes.</str>.  It is because we need a
> dataimport.properties file at conf/, into which indexing will write a last
> indexing time. Without that file, SolrWriter.java will put throw an
> exception and Solr will have this misleading  <str name="">Indexing failed.
> Rolled back all changes.</str>. output, although indexing is actually
> successfully completed.
>
>  I think we need to improve this functionality, at least documentation.
>
>  There are one more thing that we need to pay attention to, i.e. we need to
> make dataimport.properties writable by other users, otherwise,
> last_index_time will not be written and the error message may still be
> there.
>
> On Fri, Nov 13, 2009 at 9:35 AM, yountod <tyoung@...> wrote:
>
>>
>> The process initially completes with:
>>
>>  <str name="Full Dump Started">2009-11-13 09:40:46</str>
>>  <str name="">Indexing completed. Added/Updated: 200000 documents. Deleted
>> 0 documents.</str>
>>
>>
>> ...but then it fails with:
>>
>>  <str name="Full Dump Started">2009-11-13 09:40:46</str>
>>   <str name="">Indexing failed. Rolled back all changes.</str>
>>   <str name="Committed">2009-11-13 09:41:10</str>
>>  <str name="Optimized">2009-11-13 09:41:10</str>
>>  <str name="Rolledback">2009-11-13 09:41:10</str>
>>
>>
>> ====================================================================
>> I think it may have something to do with this, which I found by using the
>> DataImport.jsp:
>> ====================================================================
>> (Thread.java:636) Caused by: java.sql.SQLException: Illegal value for
>> setFetchSize(). at
>> com.mysql.jdbc.Statement.setFetchSize(Statement.java:1864) at
>>
>> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:242)
>> ... 28 more</str>
>>
>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Question-about-the-message-%22Indexing-failed.-Rolled-back-all--changes.%22-tp26242714p26340360.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>

--
Lance Norskog
goksron@...