How to retrieve the coefficients of an SVM model

[chung-3]

Hi List,
I'm using weka to build an SVM model for regression. I load an arff file
into an Instances object and then I use
weka.classifiers.functions.LibSVM to build the model. Is there a way to
retrieve the coefficients of the trained model. I refer to the
coefficients a_i in:

f(x) = SUM_i { a_i * k(x,x_i) }

where k(.,.) is the kernel of the SVM model. Here is my code:
(Thank you in advance)

public class Main {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) throws Exception {

        try {
            Instances data = new Instances(
                    new BufferedReader(
                    new
FileReader(System.getProperty("user.home")+"/Desktop/dataSet-8")));
            data.deleteAttributeAt(0);
            data.setClassIndex(data.numAttributes()-1);

            weka.classifiers.functions.LibSVM regression = new
weka.classifiers.functions.LibSVM();
            String[] options = {"-S","3"};
            regression.setOptions(options);
            regression.buildClassifier(data);
           

        } catch (IOException ex) {
            Logger.getLogger(Main.class.getName()).log(Level.SEVERE,
null, ex);
        }

    }
}



Sopasakis Pantelis
Dipl. Chemical Engineer,
MSc. Applied Mathematics
National Technical University of Athens
Automatic Control Laboratory
email: chvng@...
tel(office): [phone]



_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Peter Reutemann-3]

> I'm using weka to build an SVM model for regression. I load an arff file
> into an Instances object and then I use
> weka.classifiers.functions.LibSVM to build the model. Is there a way to
> retrieve the coefficients of the trained model.

LibSVM is merely a reflection wrapper to access the libsvm library and
doesn't offer the output of the coefficients.
Not sure whether the libsvm library allows the retrieval of the
coefficients at all.

[...]

Cheers, Peter
--
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
http://www.cs.waikato.ac.nz/~fracpete/           Ph. [phone]

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[chung-3]

Hi Peter,
 I'm working with LibSVM at the time in order to train some SVM models
but I also need some tools from weka. So I have to preprocess by data
(as Instances objects) through weka, write them in a file that LibSVM
can read from (this is called DSD format) and finally retrieve the
coefficients back from a file. LibSVM returns the coefficients of the
trained model but just stores them in a file from which I'll have to
parse them to use them within my program.
 Well, it seems I have to get in the source code of LibSVM to see if I
can do something...

Cheers,
Pantelis


On Wed, 2009-11-04 at 07:59 +1300, Peter Reutemann wrote:

> > I'm using weka to build an SVM model for regression. I load an arff file
> > into an Instances object and then I use
> > weka.classifiers.functions.LibSVM to build the model. Is there a way to
> > retrieve the coefficients of the trained model.
>
> LibSVM is merely a reflection wrapper to access the libsvm library and
> doesn't offer the output of the coefficients.
> Not sure whether the libsvm library allows the retrieval of the
> coefficients at all.
>
> [...]
>
> Cheers, Peter

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Peter Reutemann-3]

Please no top-posting, see mailing list etiquette why
(http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html).

>  I'm working with LibSVM at the time in order to train some SVM models
> but I also need some tools from weka. So I have to preprocess by data
> (as Instances objects) through weka, write them in a file that LibSVM
> can read from (this is called DSD format)

Why not just use the Weka converter "weka.core.converters.LibSVMSaver"?

> and finally retrieve the
> coefficients back from a file. LibSVM returns the coefficients of the
> trained model but just stores them in a file from which I'll have to
> parse them to use them within my program.

I just remembered, that the LibSVM wrapper in the developer version
(maybe also the stable-3.6 version) allows you to store the generated
model in a file via the "-model <file>" parameter. Maybe that helps...

Cheers, Peter
--
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
http://www.cs.waikato.ac.nz/~fracpete/           Ph. [phone]

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[chung-3]

On Wed, 2009-11-04 at 13:36 +1300, Peter Reutemann wrote:

> Please no top-posting, see mailing list etiquette why
> (http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html).
>
> >  I'm working with LibSVM at the time in order to train some SVM models
> > but I also need some tools from weka. So I have to preprocess by data
> > (as Instances objects) through weka, write them in a file that LibSVM
> > can read from (this is called DSD format)
>
> Why not just use the Weka converter "weka.core.converters.LibSVMSaver"?
>
> > and finally retrieve the
> > coefficients back from a file. LibSVM returns the coefficients of the
> > trained model but just stores them in a file from which I'll have to
> > parse them to use them within my program.
>
> I just remembered, that the LibSVM wrapper in the developer version
> (maybe also the stable-3.6 version) allows you to store the generated
> model in a file via the "-model <file>" parameter. Maybe that helps...
>
> Cheers, Peter

LibSVMSaver is used to save an Instances object in DSD format (i.e. the
format that libSVM accepts), but you can't save the model itself in DSD
format. The most convenient option would be the PMML support. Many open
source projects use PMML as input and output files for their models.

Best Regards,
Pantelis



_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Azizi Abdullah]

dear all,

I am working on an ensemble algorithm to combine classifiers to improve
the classification accuracy. I found that many Weka's classes such as
Boosting, Bagging and etc. which seem quite relevant to me. Besides that,
I think I've seen people around writing java code to make use of these
classes. Does anybody knows or can point me locations that can run the
algorithms from java code? Is the same way as in
http://weka.wikispaces.com/Programmatic+Use can be used to run weka's
ensemble algorithms?

kind regards
azizia.

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Peter Reutemann-3]

> I am working on an ensemble algorithm to combine classifiers to improve
> the classification accuracy. I found that many Weka's classes such as
> Boosting, Bagging and etc. which seem quite relevant to me. Besides that,
> I think I've seen people around writing java code to make use of these
> classes. Does anybody knows or can point me locations that can run the
> algorithms from java code? Is the same way as in
> http://weka.wikispaces.com/Programmatic+Use can be used to run weka's
> ensemble algorithms?

All classifiers are derived from the same superclass,
weka.classifiers.Classifier (or if you work off subversion on the
developer version, then this is an interface now), hence they all work
the same. Depending on what kind of ensemble you're trying to
implement, you'll have to pick an appropriate superclass from the
weka.classifiers package (e.g., randomizable, with several base
classifier -> RandomizableMultipleClassifiersCombiner). See FAQ "How
do I write a new classifier or filter?" for more information (link to
FAQs available from the Weka homepage).

You might also want to take a look at the article "Use WEKA in your
Java code", which is a bit more elaborate than then "Programmatic Use"
one:
  http://weka.wikispaces.com/Use+WEKA+in+your+Java+code

Cheers, Peter
--
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
http://www.cs.waikato.ac.nz/~fracpete/           Ph. [phone]

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Azizi Abdullah]

Dear all,

Thanks for the information. But, I have a simple question. Is the Weka
ensemble or boosting algorithm support the following things automatically:
(1) Multi-class with multiple labels
(2) feature vectors to represent documents

kind regards
azizia.

>
> All classifiers are derived from the same superclass,
> weka.classifiers.Classifier (or if you work off subversion on the
> developer version, then this is an interface now), hence they all work
> the same. Depending on what kind of ensemble you're trying to
> implement, you'll have to pick an appropriate superclass from the
> weka.classifiers package (e.g., randomizable, with several base
> classifier -> RandomizableMultipleClassifiersCombiner). See FAQ "How
> do I write a new classifier or filter?" for more information (link to
> FAQs available from the Weka homepage).
>
> You might also want to take a look at the article "Use WEKA in your
> Java code", which is a bit more elaborate than then "Programmatic Use"
> one:
>   http://weka.wikispaces.com/Use+WEKA+in+your+Java+code
>
> Cheers, Peter
> --
> Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
> http://www.cs.waikato.ac.nz/~fracpete/           Ph. [phone]
>
> _______________________________________________
> Wekalist mailing list
> Send posts to: Wekalist@...
> List info and subscription status:
> https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette:
> http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Peter Reutemann-3]

Please no top-posting, see mailing list etiquette why
(http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html).

> Thanks for the information. But, I have a simple question. Is the Weka
> ensemble or boosting algorithm support the following things automatically:
> (1) Multi-class with multiple labels

Just to clarify: Weka allows you only to have a single class
attribute. See also FAQ "Does WEKA support multi-label
classification?".

If you develop an ensemble classifier, then it depends a lot on the
base classifier(s) what data can be processed (= their capabilities).
The MultipleClassifiersCombiner superclass, for instance, returns as
capabilities only the capabilities that *all* of the base classifiers
share (see "getCapabilities()" method).

> (2) feature vectors to represent documents

See FAQ "How do I perform text classification?".

Link to the FAQs available from the Weka homepage.

Cheers, Peter
--
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
http://www.cs.waikato.ac.nz/~fracpete/           Ph. [phone]

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html