A Walk-Forward Attempt on FAA

So in the first post about FAA, I was requested to make a walk-forward test of FAA. While the results here aren’t good, I’d like to share the general process anyway.

Here’s the additional code I wrote, assuming the first post‘s code is still in your environment (the demo will have the function in the namespace as well):

weightMom <- seq(0, 1, by=.5)
weightVol <- c(0, .5, 1)
weightCor <- c(0, .5, 1)
monthLookback=c(3, 4, 6, 10)
permutations <- expand.grid(weightMom, weightVol, weightCor, monthLookback)
colnames(permutations) <- c("wMom", "wVol", "wCor", "monthLookback")

require(doMC)
registerDoMC(detectCores())
t1 <- Sys.time()
out <- foreach(i = 1:nrow(permutations), .combine = cbind) %dopar% {
  FAAreturns(prices=adPrices, 
             monthLookback = permutations$monthLookback[i], 
             weightMom = permutations$wMom[i], 
             weightCor = permutations$wCor[i], 
             weightVol=permutations$wVol[i])
}
t2 <- Sys.time()
print(t2-t1)

out <- out["1998-10::"] #start at 1999 due to NAs with data

FAAwalkForward <- function(portfolios, applySubset = apply.quarterly, applyFUN = Return.cumulative) {
  metrics <- applySubset(portfolios, applyFUN)
  weights <- list()
  for(i in 1:nrow(metrics)) {
    row <- metrics[i,]
    winners <- row==max(row)
    weight <- winners/rowSums(winners) #equal weight all top performers
    weights[[i]] <- weight
  }
  weights <- do.call(rbind, weights)
  returns <- Return.rebalancing(portfolios, weights)
  return(returns)
}

WFQtrRets <- FAAwalkForward(portfolios = out, applySubset = apply.quarterly, applyFUN = Return.cumulative)
WFYrRets <- FAAwalkForward(portfolios = out, applySubset = apply.yearly, applyFUN = Return.cumulative)
WFMoRets <- FAAwalkForward(portfolios = out, applySubset = apply.monthly, applyFUN = Return.cumulative)

WF <- cbind(WFQtrRets, WFYrRets, WFMoRets)
colnames(WF) <- c("quarterly", "annually", "monthly")
WF <- WF["1999::"]

original <- FAAreturns(adPrices)
original <- original["1999::"]
WF <- cbind(WF, original)
colnames(WF)[4] <- "original"
charts.PerformanceSummary(WF)


Return.annualized(WF)
maxDrawdown(WF)
SharpeRatio.annualized(WF)

So what I did was take about a hundred permutations, and compute them all in parallel using the doMC package (Windows uses a different parallel architecture, but this post explains a more OS-agnostic method). Next, I wrote a small function that would compute a metric for all of the permutations for some period (monthly, quarterly, annually), and equal-weight all of the maximum configurations, which were in some cases, more than one. This would be the strategy’s holdings over the next period, and repeat this iteration to the end. I started in late 1998, as I used a 10-month lookback period for one of the monthly lookback settings, while the data starts in mid 1997.

Here are the results:

> Return.annualized(WF)
                  quarterly  annually   monthly  original
Annualized Return 0.1303674 0.1164749 0.1204314 0.1516936
> maxDrawdown(WF)
               quarterly  annually   monthly  original
Worst Drawdown 0.1666639 0.1790334 0.1649651 0.1315625
> SharpeRatio.annualized(WF)
                                quarterly annually  monthly original
Annualized Sharpe Ratio (Rf=0%)  1.257753 1.025863 1.194066 1.519912

In short, using a walk-forward with FAA seriously harmed the performance.

After seeing these disappointing results, I mulled over as to the why, and here’s my intuition:

A) FAA uses a ranking algorithm, which loses the nuance of slightly changing this weight or that, leading to many identical configurations for a given time period. That some of these or all of these configurations are only the best for that period is a very real possibility.

B) Given that there are so few securities, it’s quite possible that the walk-forward process is simply chasing performance–that is, switching into a configuration right after it had its one moment in the sun. Considering that the original strategy is fairly solid to begin with, it certainly seems to be better to pick a decent configuration and stick with it.

C) One last thought that stuck with me is that the original FAA strategy meets all the qualifications *for* a walk-forward strategy already. It is a monthly-rebalanced algorithm that seeks to maximize an objective function (rank of the weighted sum of ranks of momentum, volatility, and correlation) in a robust fashion–it’s simply that instead of strategy configurations, the inputs were seven mutual funds. To me, in fact, the more I think about it, the more FAA looks like an extremely solid walk-forward framework, simply presented as a strategy in and of itself.

In any case, while the results were far from spectacular, I’m hoping that the code has given others some ideas about how to conduct some generalized returns-based walk-forward testing on their own strategies.

On one last note, if any readers have ideas that they’d like me to investigate, I’m always open to input and new ideas.

Thanks for reading.