Optimization Question

[ian.mcdonald]

I am interested in doing a portfolio optimization, but I need to impose
non-linear, functional constraints (i.e Max Drawdown, or VaR, etc). I've
looked into several of the optimization routines (nlminb, constrOptim and
optim with SANN/L-BFGS-B) but it appears that all of them are geared to
linear constraints. Other than using a penalty function method, is anyone
aware of any other approaches available in  R?

Thanks
Ian McDonald
Malbec Partners

        [[alternative HTML version deleted]]

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

Sorry, this is not an answer, but I would be interested in specifics
and what theory you use, i.e. what multivariate distribution and maybe
a reference as to calculation of MDD, VaR, ES, ... ?

On Tue, Dec 30, 2008 at 4:48 PM,  <ian.mcdonald@...> wrote:

> I am interested in doing a portfolio optimization, but I need to impose
> non-linear, functional constraints (i.e Max Drawdown, or VaR, etc). I've
> looked into several of the optimization routines (nlminb, constrOptim and
> optim with SANN/L-BFGS-B) but it appears that all of them are geared to
> linear constraints. Other than using a penalty function method, is anyone
> aware of any other approaches available in  R?
>
> Thanks
> Ian McDonald
> Malbec Partners
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> R-SIG-Finance@... mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

[Patrick Burns-2]

I suspect that including Maximum drawdown in an
optimization is unlikely to be useful.  However, I'm
anxious to be proven wrong.

It's not clear to me what specifically you are wanting
to do, but Manfred Gilli has done a fair amount on
VaR in optimization. (But I don't think you'll get any
R code off him.)

Patrick Burns
patrick@...
[phone]
http://www.burns-stat.com
(home of S Poetry and "A Guide for the Unwilling S User")

ian.mcdonald@... wrote:

> I am interested in doing a portfolio optimization, but I need to impose
> non-linear, functional constraints (i.e Max Drawdown, or VaR, etc). I've
> looked into several of the optimization routines (nlminb, constrOptim and
> optim with SANN/L-BFGS-B) but it appears that all of them are geared to
> linear constraints. Other than using a penalty function method, is anyone
> aware of any other approaches available in  R?
>
> Thanks
> Ian McDonald
> Malbec Partners
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-SIG-Finance@... mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>
>
>

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

[ian.mcdonald]

I wanted to be able to generally include a non-linear constraint on an optimization. I was using maxdrawdown, var, or any risk measure as examples.

Real world example might be when one needs to optimize a portfolio with weight constraints (max size to one or more securities).  We might choose a utility function such as info ratio, calmar ratio, etc, but the resulting solution, while wonderful from utility perpective, has poor real world pnl and isn't simply scalable due to weight constraints. One can modify the utility function, or constrain the risk (or profit) to some minimum.  

So generally investigating optimization with potentially non-linear constraints. I think a penalty function into utility is best but wondering if there were other approaches.

Ian McDonald


----- Original Message -----
From: Patrick Burns [patrick@...]
Sent: 12/30/2008 09:01 PM GMT
To: Ian McDonald
Cc: r-sig-finance@...
Subject: Re: [R-SIG-Finance] Optimization Question



I suspect that including Maximum drawdown in an
optimization is unlikely to be useful.  However, I'm
anxious to be proven wrong.

It's not clear to me what specifically you are wanting
to do, but Manfred Gilli has done a fair amount on
VaR in optimization. (But I don't think you'll get any
R code off him.)

Patrick Burns
patrick@...
[phone]
http://www.burns-stat.com
(home of S Poetry and "A Guide for the Unwilling S User")

ian.mcdonald@... wrote:

> I am interested in doing a portfolio optimization, but I need to impose
> non-linear, functional constraints (i.e Max Drawdown, or VaR, etc). I've
> looked into several of the optimization routines (nlminb, constrOptim and
> optim with SANN/L-BFGS-B) but it appears that all of them are geared to
> linear constraints. Other than using a penalty function method, is anyone
> aware of any other approaches available in  R?
>
> Thanks
> Ian McDonald
> Malbec Partners
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-SIG-Finance@... mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>
>
>

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

Well, for BM/GBM, you could calculate the expected value of MDD over
the investment horizon according to Magdon-Ismail et al. 2003 (?) or
their article in Risk and use that. That shouldn't be too hard, but
depending on how the parameters are determined, it's unlikely to be
very useful for risk management. Which is why I would be interested in
which specific distributional assumptions the original question was
aiming at. Indeed I'd be grateful for references on sampling
distributions of MDD for processes other than BM, or for that matter
even what form the sampling distribution of MDD of BM takes. Or do you
plan to use some Monte Carlo method to calculate MDD or any of the
other risk measures in the post?

On Tue, Dec 30, 2008 at 10:01 PM, Patrick Burns <patrick@...> wrote:

> I suspect that including Maximum drawdown in an
> optimization is unlikely to be useful.  However, I'm
> anxious to be proven wrong.
>
> It's not clear to me what specifically you are wanting
> to do, but Manfred Gilli has done a fair amount on
> VaR in optimization. (But I don't think you'll get any
> R code off him.)
>
> Patrick Burns
> patrick@...
> [phone]
> http://www.burns-stat.com
> (home of S Poetry and "A Guide for the Unwilling S User")
>
> ian.mcdonald@... wrote:
>>
>> I am interested in doing a portfolio optimization, but I need to impose
>> non-linear, functional constraints (i.e Max Drawdown, or VaR, etc). I've
>> looked into several of the optimization routines (nlminb, constrOptim and
>> optim with SANN/L-BFGS-B) but it appears that all of them are geared to
>> linear constraints. Other than using a penalty function method, is anyone
>> aware of any other approaches available in  R?
>>
>> Thanks
>> Ian McDonald
>> Malbec Partners
>>
>>        [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-SIG-Finance@... mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only.
>> -- If you want to post, subscribe first.
>>
>>
>>
>
> _______________________________________________
> R-SIG-Finance@... mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only.
> -- If you want to post, subscribe first.
>

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

[Guy Yollin-2]

Ian,

An excellent technique for general purpose non-linear optimization is the differential evolution algorithm:

http://www.icsi.berkeley.edu/~storn/code.html

It has been implemented in the DEoptim package available on CRAN; the algorithm is very flexible and you could readily implement an maximum drawdown or VaR optimization strategy using DE.

Back in February 2008, David Basterfield gave a webcast on the differential evolution algorithm with a number of non-linear portfolio optimization examples; it is still available on the Insightful website under events/finance:

        www.insightful.com

Note, a number of papers have shown the superiority of conditional-value-at-risk (CVaR) to VaR as a risk metric.  The CVaR portfolio optimization can be framed as a linear programming problem and you could then use a linear programming package (e.g. lpSolve or linprog) to find mean-CVaR optimum portfolios.  The books Portfolio Construction and Risk Budgeting by Bernd Scherer and Introduction to Modern Portfolio Optimization by Bernd Scherer and Douglas Martin provides some details on this and other non-standard portfolio optimization techniques; although both these books include S+NuOPT examples, the key points are transferrable to R.

Best,

-- Guy




-----Original Message-----
From: r-sig-finance-bounces@... [mailto:r-sig-finance-bounces@...] On Behalf Of ian.mcdonald@...
Sent: Tuesday, December 30, 2008 7:49 AM
To: r-sig-finance@...
Subject: [R-SIG-Finance] Optimization Question

I am interested in doing a portfolio optimization, but I need to impose
non-linear, functional constraints (i.e Max Drawdown, or VaR, etc). I've
looked into several of the optimization routines (nlminb, constrOptim and
optim with SANN/L-BFGS-B) but it appears that all of them are geared to
linear constraints. Other than using a penalty function method, is anyone
aware of any other approaches available in  R?

Thanks
Ian McDonald
Malbec Partners

        [[alternative HTML version deleted]]

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

[Patrick Burns-2]

I would think that the two possibilities are:

as you say, adding a penalty to the objective,

looking through the optimization task view to
see if there are any appropriate functions that
handle non-linear constraints.

Pat

ian.mcdonald@... wrote:

> I wanted to be able to generally include a non-linear constraint on an
> optimization. I was using maxdrawdown, var, or any risk measure as
> examples.
>
> Real world example might be when one needs to optimize a portfolio
> with weight constraints (max size to one or more securities).  We
> might choose a utility function such as info ratio, calmar ratio, etc,
> but the resulting solution, while wonderful from utility perpective,
> has poor real world pnl and isn't simply scalable due to weight
> constraints. One can modify the utility function, or constrain the
> risk (or profit) to some minimum.
> So generally investigating optimization with potentially non-linear
> constraints. I think a penalty function into utility is best but
> wondering if there were other approaches.
>
> Ian McDonald
>
>
> ----- Original Message -----
> From: Patrick Burns [patrick@...]
> Sent: 12/30/2008 09:01 PM GMT
> To: Ian McDonald
> Cc: r-sig-finance@...
> Subject: Re: [R-SIG-Finance] Optimization Question
>
>
>
> I suspect that including Maximum drawdown in an
> optimization is unlikely to be useful.  However, I'm
> anxious to be proven wrong.
>
> It's not clear to me what specifically you are wanting
> to do, but Manfred Gilli has done a fair amount on
> VaR in optimization. (But I don't think you'll get any
> R code off him.)
>
> Patrick Burns
> patrick@...
> [phone]
> http://www.burns-stat.com
> (home of S Poetry and "A Guide for the Unwilling S User")
>
> ian.mcdonald@... wrote:
>> I am interested in doing a portfolio optimization, but I need to
>> impose non-linear, functional constraints (i.e Max Drawdown, or VaR,
>> etc). I've looked into several of the optimization routines (nlminb,
>> constrOptim and optim with SANN/L-BFGS-B) but it appears that all of
>> them are geared to linear constraints. Other than using a penalty
>> function method, is anyone aware of any other approaches available
>> in  R?
>>
>> Thanks
>> Ian McDonald
>> Malbec Partners
>>
>>     [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-SIG-Finance@... mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only.
>> -- If you want to post, subscribe first.
>>
>>
>>  
>
>

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

[Brian G. Peterson]

Ian McDonald asked:

> I am interested in doing a portfolio optimization, but I need to impose
> non-linear, functional constraints (i.e Max Drawdown, or VaR, etc). I've
> looked into several of the optimization routines (nlminb, constrOptim and
> optim with SANN/L-BFGS-B) but it appears that all of them are geared to
> linear constraints. Other than using a penalty function method, is anyone
> aware of any other approaches available in  R?
>
> Thanks
> Ian McDonald
> Malbec Partners

Guy Yollin responded:

> Ian,
>
> An excellent technique for general purpose non-linear optimization is the
> differential evolution algorithm:
>
> http://www.icsi.berkeley.edu/~storn/code.html
>
> It has been implemented in the DEoptim package available on CRAN; the
> algorithm is very flexible and you could readily implement an maximum
> drawdown or VaR optimization strategy using DE.
>
> Back in February 2008, David Basterfield gave a webcast on the
> differential evolution algorithm with a number of non-linear portfolio
> optimization examples; it is still available on the Insightful website
> under events/finance:
>
>         http://www.insightful.com/
>
> Note, a number of papers have shown the superiority of
> conditional-value-at-risk (CVaR) to VaR as a risk metric.  The CVaR
> portfolio optimization can be framed as a linear programming problem and
> you could then use a linear programming package (e.g. lpSolve or linprog)
> to find mean-CVaR optimum portfolios.  The books Portfolio Construction
> and Risk Budgeting by Bernd Scherer and Introduction to Modern Portfolio
> Optimization by Bernd Scherer and Douglas Martin provides some details on
> this and other non-standard portfolio optimization techniques; although
> both these books include S+NuOPT examples, the key points are
> transferrable to R.
>
> Best,
>
> -- Guy

Guy raises some excellent points.  I think that one key here is that there
are typically other constraints on a large portfolio optimization problem,
not just the return/risk ratio.  In addition to weight constraints over
the portfolio generally, the investor may have additional restrictions on
style allocations, lockup periods on particular investments, etc.  As the
number of constraints rises, the availability and suitability of a
solution decreases.

To extend Guy's comments about CVaR or ES, it is superior for optimization
because it is a coherent risk measure under most circumstances,  some of
our work extended ES to include skewness and kurtosis using the Cornish
Fisher and Edgeworth expansions.  Those functions are available in R in
PerformanceAnalytics.

We've also done some work with non-linear optimization under constraints
using Rdonlp2 (non-free for commercial use) in R.  An exploratory example
of that work was published here:

http://library.witpress.com/pages/PaperInfo.asp?PaperID=18906

Another interesting point is how you arrive at predictions of the values
in question. VaR and its cousins are typically a one-step-ahead
prediction.  If you want to optimize over longer time frames than one
period, you need to predict parameters (mean, variance, skewness,
kurtosis) further into the future with some confidence.  This generally
becomes a multi-layered problem as you seek to find the best predictors of
your input parameters before finally applying it all to the portfolio
optimization algorithm.

I've generally found most published discussion of risk budgeting to be
dissatisfying because they skip over many of these nuances,  Hopefully
future work by this community (myself included!) can remedy some of those
oversights.

Regards,

    - Brian

--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

[ian.mcdonald]

Thank you, will investigate.

--Ian




"Brian G. Peterson" <brian@...>
Sent by: r-sig-finance-bounces@...
12/31/2008 07:27 AM

To
"r-sig-finance" <r-sig-finance@...>
cc

Subject
Re: [R-SIG-Finance] Optimization Question





Ian McDonald asked:
> I am interested in doing a portfolio optimization, but I need to impose
> non-linear, functional constraints (i.e Max Drawdown, or VaR, etc). I've
> looked into several of the optimization routines (nlminb, constrOptim
and
> optim with SANN/L-BFGS-B) but it appears that all of them are geared to
> linear constraints. Other than using a penalty function method, is
anyone
> aware of any other approaches available in  R?
>
> Thanks
> Ian McDonald
> Malbec Partners


Guy Yollin responded:
> Ian,
>
> An excellent technique for general purpose non-linear optimization is
the

> differential evolution algorithm:
>
> http://www.icsi.berkeley.edu/~storn/code.html
>
> It has been implemented in the DEoptim package available on CRAN; the
> algorithm is very flexible and you could readily implement an maximum
> drawdown or VaR optimization strategy using DE.
>
> Back in February 2008, David Basterfield gave a webcast on the
> differential evolution algorithm with a number of non-linear portfolio
> optimization examples; it is still available on the Insightful website
> under events/finance:
>
>         http://www.insightful.com/
>
> Note, a number of papers have shown the superiority of
> conditional-value-at-risk (CVaR) to VaR as a risk metric.  The CVaR
> portfolio optimization can be framed as a linear programming problem and
> you could then use a linear programming package (e.g. lpSolve or

linprog)
> to find mean-CVaR optimum portfolios.  The books Portfolio Construction
> and Risk Budgeting by Bernd Scherer and Introduction to Modern Portfolio
> Optimization by Bernd Scherer and Douglas Martin provides some details
on
> this and other non-standard portfolio optimization techniques; although
> both these books include S+NuOPT examples, the key points are
> transferrable to R.
>
> Best,
>
> -- Guy

Guy raises some excellent points.  I think that one key here is that there
are typically other constraints on a large portfolio optimization problem,
not just the return/risk ratio.  In addition to weight constraints over
the portfolio generally, the investor may have additional restrictions on
style allocations, lockup periods on particular investments, etc.  As the
number of constraints rises, the availability and suitability of a
solution decreases.

To extend Guy's comments about CVaR or ES, it is superior for optimization
because it is a coherent risk measure under most circumstances,  some of
our work extended ES to include skewness and kurtosis using the Cornish
Fisher and Edgeworth expansions.  Those functions are available in R in
PerformanceAnalytics.

We've also done some work with non-linear optimization under constraints
using Rdonlp2 (non-free for commercial use) in R.  An exploratory example
of that work was published here:

http://library.witpress.com/pages/PaperInfo.asp?PaperID=18906

Another interesting point is how you arrive at predictions of the values
in question. VaR and its cousins are typically a one-step-ahead
prediction.  If you want to optimize over longer time frames than one
period, you need to predict parameters (mean, variance, skewness,
kurtosis) further into the future with some confidence.  This generally
becomes a multi-layered problem as you seek to find the best predictors of
your input parameters before finally applying it all to the portfolio
optimization algorithm.

I've generally found most published discussion of risk budgeting to be
dissatisfying because they skip over many of these nuances,  Hopefully
future work by this community (myself included!) can remedy some of those
oversights.

Regards,

    - Brian

--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.


        [[alternative HTML version deleted]]

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.

[Greg James-2]

Folks,

While I'm not involved with aspects of this problem, let me just say that
reading everyone's contributions to this discussion has been very
educational for me. Thank you to all for being so generous with your
knowledge.

Greg James

_______________________________________________
R-SIG-Finance@... mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.