Solr-ruby crashing "(eval):1: Illegal octal digit"

[Thiago Jackiw-2]

For some reason solr-ruby is crashing when returning the response
below. It works fine if I try the same query directly without using
solr-ruby. Any ideas?

Query:
?wt=ruby&fl=zip_code&q=accounting

Response:
{'responseHeader'=>{'status'=>0,'QTime'=>1,'params'=>{'wt'=>'ruby','rows'=>'1','start'=>'5','q'=>'accounting','fl'=>'zip_code','qt'=>'standard'}},'response'=>{'numFound'=>15005,'start'=>5,'docs'=>[{'zip_code'=>[08817]}]}}

Error message:
solr/response/ruby.rb:30:in `initialize': invalid ruby code:
(eval):1:in `initialize': compile error (Solr::Exception) (eval):1:
Illegal octal digit

Thanks,

Thiago

[Yonik Seeley]

On 10/17/07, Thiago Jackiw <tjackiw@...> wrote:

> For some reason solr-ruby is crashing when returning the response
> below. It works fine if I try the same query directly without using
> solr-ruby. Any ideas?
>
> Query:
> ?wt=ruby&fl=zip_code&q=accounting
>
> Response:
> {'responseHeader'=>{'status'=>0,'QTime'=>1,'params'=>{'wt'=>'ruby','rows'=>'1','start'=>'5','q'=>'accounting','fl'=>'zip_code','qt'=>'standard'}},'response'=>{'numFound'=>15005,'start'=>5,'docs'=>[{'zip_code'=>[08817]}]}}
>
> Error message:
> solr/response/ruby.rb:30:in `initialize': invalid ruby code:
> (eval):1:in `initialize': compile error (Solr::Exception) (eval):1:
> Illegal octal digit

It's complaining about the 08817 (an octal constant, not a decimal one...)
Solr returned this???  What's the fieldType of zip_code?

-Yonik

[Erik Hatcher]

Looks like your zip_code field should be made a string data type  
instead of numeric.   How do you have it defined in your schema.xml?

That is quite possibly a bug in the response writer, but that could  
be tricky for the Java code to write out out legal numerics.

I've been considering writing a custom solr-ruby specific Solr  
response writer so dates and ordered hashes come out in a friendly  
Ruby format - this might be another case where this could be handy -  
however this is illegal Ruby code it seems.  Making it a string type  
would likely do the trick though.

        Erik


On Oct 17, 2007, at 3:58 AM, Thiago Jackiw wrote:

> For some reason solr-ruby is crashing when returning the response
> below. It works fine if I try the same query directly without using
> solr-ruby. Any ideas?
>
> Query:
> ?wt=ruby&fl=zip_code&q=accounting
>
> Response:
> {'responseHeader'=>{'status'=>0,'QTime'=>1,'params'=>
> {'wt'=>'ruby','rows'=>'1','start'=>'5','q'=>'accounting','fl'=>'zip_co
> de','qt'=>'standard'}},'response'=>
> {'numFound'=>15005,'start'=>5,'docs'=>[{'zip_code'=>[08817]}]}}
>
> Error message:
> solr/response/ruby.rb:30:in `initialize': invalid ruby code:
> (eval):1:in `initialize': compile error (Solr::Exception) (eval):1:
> Illegal octal digit
>
> Thanks,
>
> Thiago

[Yonik Seeley]

On 10/17/07, Yonik Seeley <yonik@...> wrote:

> On 10/17/07, Thiago Jackiw <tjackiw@...> wrote:
> > For some reason solr-ruby is crashing when returning the response
> > below. It works fine if I try the same query directly without using
> > solr-ruby. Any ideas?
> >
> > Query:
> > ?wt=ruby&fl=zip_code&q=accounting
> >
> > Response:
> > {'responseHeader'=>{'status'=>0,'QTime'=>1,'params'=>{'wt'=>'ruby','rows'=>'1','start'=>'5','q'=>'accounting','fl'=>'zip_code','qt'=>'standard'}},'response'=>{'numFound'=>15005,'start'=>5,'docs'=>[{'zip_code'=>[08817]}]}}
> >
> > Error message:
> > solr/response/ruby.rb:30:in `initialize': invalid ruby code:
> > (eval):1:in `initialize': compile error (Solr::Exception) (eval):1:
> > Illegal octal digit
>
> It's complaining about the 08817 (an octal constant, not a decimal one...)
> Solr returned this???  What's the fieldType of zip_code?

Ah, I bet zip_code could be of type "integer" which just stores the
text representation of a number, and you probably indexed "08817"
which solr faithfully spit back out.  As Erik says, you probably want
a string type for this to preserve leading zeros and allow dashes.
You should be able to change the type of the field in the schema and
not have to reindex (since they are "compatible").

-Yonik

[Thiago Jackiw-2]

Yeah unfortunately I can't change that field, it has to be kept as
integer. And also I don't think it would make any difference because
when you eval the string "08817" or the number 08817 they both get the
same error.

Thiago


On 10/17/07, Yonik Seeley <yonik@...> wrote:

> On 10/17/07, Yonik Seeley <yonik@...> wrote:
> > On 10/17/07, Thiago Jackiw <tjackiw@...> wrote:
> > > For some reason solr-ruby is crashing when returning the response
> > > below. It works fine if I try the same query directly without using
> > > solr-ruby. Any ideas?
> > >
> > > Query:
> > > ?wt=ruby&fl=zip_code&q=accounting
> > >
> > > Response:
> > > {'responseHeader'=>{'status'=>0,'QTime'=>1,'params'=>{'wt'=>'ruby','rows'=>'1','start'=>'5','q'=>'accounting','fl'=>'zip_code','qt'=>'standard'}},'response'=>{'numFound'=>15005,'start'=>5,'docs'=>[{'zip_code'=>[08817]}]}}
> > >
> > > Error message:
> > > solr/response/ruby.rb:30:in `initialize': invalid ruby code:
> > > (eval):1:in `initialize': compile error (Solr::Exception) (eval):1:
> > > Illegal octal digit
> >
> > It's complaining about the 08817 (an octal constant, not a decimal one...)
> > Solr returned this???  What's the fieldType of zip_code?
>
> Ah, I bet zip_code could be of type "integer" which just stores the
> text representation of a number, and you probably indexed "08817"
> which solr faithfully spit back out.  As Erik says, you probably want
> a string type for this to preserve leading zeros and allow dashes.
> You should be able to change the type of the field in the schema and
> not have to reindex (since they are "compatible").
>
> -Yonik
>

[Yonik Seeley]

On 10/17/07, Thiago Jackiw <tjackiw@...> wrote:
> Yeah unfortunately I can't change that field, it has to be kept as
> integer.

Why is that?

> And also I don't think it would make any difference because
> when you eval the string "08817" or the number 08817 they both get the
> same error.

You probably forgot to include the quotes when you tested this.

irb(main):005:0* eval('"08817"')
=> "08817"
irb(main):006:0> eval('08817')
SyntaxError: (eval):1:in `irb_binding': compile error
(eval):1: Illegal octal digit
08817
  ^
        from (irb):6
        from (irb):6
irb(main):007:0>

-Yonik

[Erik Hatcher]

On Oct 17, 2007, at 1:36 PM, Thiago Jackiw wrote:
> Yeah unfortunately I can't change that field, it has to be kept as
> integer. And also I don't think it would make any difference because
> when you eval the string "08817" or the number 08817 they both get the
> same error.

I still don't understand why you need to keep that as an integer.  
You're obviously sending it in as a String (or it wouldn't have a  
leading 0 on it).  And when you get it back as an integer it'll be  
8817, not 08817 so to redisplay it you'd surely want to pad it  
again.  Could you elaborate on why you want to store this as an  
integer?  Keep in mind that you could have a zip code field for  
display purposes and one for sorting or ranges using a copyField.

Any objections to changing Solr's Ruby response writer to prefixing  
integers with "0d"?   That'll do the trick:

   irb(main):002:0> eval('0d08')
   => 8

That actually makes good sense to me.  I've started doing this  
locally and once I've created the test case infrastructure for this  
I'll commit unless there are issues I've not considered.

        Erik

[Yonik Seeley]

On 10/19/07, Erik Hatcher <erik@...> wrote:
> Any objections to changing Solr's Ruby response writer to prefixing
> integers with "0d"?

I think it would look nicer to simply strip leading zeros... they
aren't valid in JSON, and Python treats it as an octal constant (will
be illegal in python 3, like ruby, I think).

Since "integer" is meant to be a legacy type that may have existing
zero padded numbers, I guess this should be done in the response
writer (in the base text writer or JSON writer).

-Yonik