Need clarification on Prediction errors

[aramki]

Some parts of this message have been removed. Learn more about Nabble's security policy.

June/30/2009

Could you clarify the following on prediction errors.

Which error i should take for comparing the algorithms.

Last time you clarified that correlation coefficient cannot a parameter for selecting the best predicting algorithm.

Now if you compare the MLP vs RF MLP has low Absolute errors where as RF has lower squared errors.

Pl clarify which algorithm i should consider for my numeric prediction.

Description	MLP -4H	RF
Correlation coefficient	0.247	0.535
Mean absolute error	3.692	3.742
Root mean squared error	4.915	4.6
Relative absolute error	57.59 %	58.37%
Root relative squared error	72.67 %	68.10%

Thanks in advance.

with warm regards

A.Ramakrishnan

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Peter Reutemann-3]

> Could you clarify the following on prediction errors.
>
> Which error i should take for comparing the algorithms.
>
> Last time you clarified that correlation coefficient cannot a parameter for
> selecting the best predicting algorithm.
>
> Now if you compare the MLP vs RF MLP has low Absolute errors where as RF has
> lower squared errors.
>
> Pl clarify which algorithm i should consider for my numeric prediction.
>
> Description
>
> MLP -4H
>
> RF
>
> Correlation coefficient
>
> 0.247
>
> 0.535
>
> Mean absolute error
>
> 3.692
>
> 3.742
>
> Root mean squared error
>
> 4.915
>
> 4.6
>
> Relative absolute error
>
> 57.59 %
>
> 58.37%
>
> Root relative squared error
>
> 72.67 %
>
> 68.10%

To be honest, both models don't seem to perform very well. Correlation
coefficient (CC) should be close to 1 (somewhere above 0.95) and root
relative squared error (RRSE) should be close to 0% (definitely below
10%). But then I don't know what data your dealing.
If I had to choose from those two models, I'd go for the RF one, as
the CC is higher and RRSE is lower. But that's just my 2c.

Cheers, Peter
--
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ
http://www.cs.waikato.ac.nz/~fracpete/           Ph. [phone]

_______________________________________________
Wekalist mailing list
Send posts to: Wekalist@...
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html