Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
DISCLAIMER: Any losses incurred based on the content of this post are the responsibility of the trader, not the author. The author takes no responsibility for the conduct of others nor offers any guarantees.
Introduction: Efficient Market Hypothesis
Burton Malkiel, in the finance classic A Random Walk Down Wall Street, made the accessible, popular case for the efficient market hypothesis (EMH). One can sum up the EMH as, “the price is always right.” No trader can know more about the market; the market price for an asset, such as a stock, is always correct. This means that trading, which relies on forecasting the future movements of prices, is as profitable as forecasting whether a coin will land heads-up; in short, traders are wasting their time. The best one can do is buy a large portfolio of assets representing the composition of the market and earn the market return rate (about 8.5% a year). Don’t try to pick winners and losers; just pick a low-expense, “dumb” fund, and you’ll do better than any highly-paid mutual fund manager (who isn’t smart enough to be profitable).
The EMH comes in various forms and “stengths”. The strong form of the EMH claims that prices cannot be beaten by anyone. Period. All information is contained in the market price; no one has any information that could “correct” the market price. The semi-strong form walks this back by claiming that prices react to new information instantaneously, so no one can consistently beat the market (this walks back the strong form, where new information is irrelevant). The weak form claims that only those traders with insider knowledge can beat the market; public information is of no help.
If true, the EMH condemns much of the finance sector and traders to practicing an elaborate exercise in futility, so naturally its controversial. Obviously the weak version is most plausible (if not because the other two versions imply the weak version), and my reading of Burton Malkiel’s statements is that he’s a believer in either the weak or semi-strong versions, but any version dooms investors to the “boring” market rate (since insider trading is illegal). Many don’t like that idea.
There are good reasons to believe any form of the EMH is wrong. First, the EMH, if true, implies stock movement behavior exhibits certain characteristics that can be statistically tested. In particular, “extreme” price movements do not happen. Stocks price movements would resemble a Wiener process (good news if your a mathematician or financial engineer, since these process have very nice properties), and price changes would follow a bell-curve shape (the Normal distribution) where 99.7% of most price movements happen nearby the average; “extreme” events are almost never seen. But the famed mathematician Benoit Mandelbrot (the one famous for developing the theory of fractals) showed that extreme price movements, rather than being relatively uncommon, happen very frequently; we see price movements that should not be seen within a million years or longer every couple decades or so. There are other assaults too from behavioral economics and other fields, along with anecdotal evidence such as the wealth of Warren Buffett.
A Random Walk Down Wall Street has been through (at least) eleven editions, so one should not be surprised that Malkiel has heard and addressed some of these issues. For example, instead of arguing against behavioral economics, he co-opts some findings from the theory, such as confirmation bias or the brain’s propensity for pattern-seeking, that supports his thesis that trying to beat the market is a futile effort (ignoring some of the theory’s majori criticisms of the EMH)1..
As for some of the attacks against the markets’ alleged efficiency, Malkiel has a good defense; even if markets are not really, truly efficient (efficiency meaning “the price is always right”), for the rank-and-file investor, this doesn’t matter. They’re efficient enough for an investor to be unable to game them profitably. Fees and small mistakes will eat away any gains to be had; worse, they may reverse them from an opportunity-cost perspective, if not even a purely monetary one.
Malkiel has a point: index funds can win just by being low cost, thanks to being “dumb” by just matching the composition of the market. Every transaction has a cost. Small gains may fail to make up for the costs in research, and the opportunity to make those gains may disappear as soon as they’re spotted. But most damaging of all is the fees.
Fees are Important
Fees and transaction costs are the friction sapping the energy from a trading system until it grinds to a red-hot halt. They’re the house’s cut at a poker table that make the game a losing one for all involved except the house. They’re often mentioned as an afterthought in books; a strategy is presented without accounting for fees, and the author says, “We should account for fees,” and nothing more. This could lead to a neophyte trader believing that fees are more a nuisance than something to be seriously accounted for when backtesting. Nothing could be further from the truth.
Every effort should be taken to minimize fees. Opt for the cheapest brokers (don’t pay for the advice; the premium paid for what passes for expertise is likely not worth it, even if the advice were good, and it often isn’t). Given the choice between a trading system making lots of trades and a trading system making few, opt for few. Remember that a single percentage point difference in returns means a big difference in profitability (thanks to compounding). Heck, people could become rich on Wall Street if only they can dodge the fees!
In my blog post on backtesting with Python, I introduced a problem to readers: adjust my backtesting system to account for transaction costs. Naturally, I never give a problem without solving it myself first (how else do I know it can be done?), and I retried it on the data sets I looked at in the post, adding in a 2% commission.
The result? The strategy that was barely profitable turned into a losing one (let alone losing compared to SPY).
The point was important enough that I felt sorry that I did not post this result before. I still will not post my Python solution; why spoil the problem? But I can still make that point in R (I never made equivalent R problems).
Cleaning Up the Trading System
Readers who read both the Python and R posts on backtesting will notice that they are not exactly equivalent. The Python trading system trades based on signal, while the R system trades based on regime (the R system will reinforce a position so it’s always 10% of the portfolio, even after the initial crossing). They Python system trades in batches of 100, while the R system has no such restriction. Here, I redo the R trading system, when trading from a batch of tech stocks, and bring it back into line with they Python trading system.
I load in quantstrat and repeat many of the steps from earlier articles.
if (!require("quantstrat")) {
install.packages("quantstrat", repos="http://R-Forge.R-project.org")
library(quantstrat)
}
if (!require("IKTrading")) {
if (!require("devtools")) {
install.packages("devtools")
}
library(devtools)
install_github("IKTrading", username = "IlyaKipnis")
library(IKTrading)
}
start <- as.Date("2010-01-01")
end <- as.Date("2016-10-01")
rm(list = ls(.blotter), envir = .blotter) # Clear blotter environment
currency("USD") # Currency being used
Sys.setenv(TZ = "MDT") # Allows quantstrat to use timestamps
initDate <- "2009-12-31" # A date prior to first close price; needed (why?)
# Get new symbols
symbols <- c("AAPL", "MSFT", "GOOG", "FB", "TWTR", "NFLX", "AMZN", "YHOO",
"SNY", "NTDOY", "IBM", "HPQ")
getSymbols(Symbols = symbols, src = "yahoo", from = start, to = end)
# Quickly define adjusted versions of each of these
`%s%` <- function(x, y) {paste(x, y)}
`%s0%` <- function(x, y) {paste0(x, y)}
for (s in symbols) {
eval(parse(text = s %s0% "_adj <- adjustOHLC(" %s0% s %s0% ")"))
}
symbols_adj <- paste(symbols, "adj", sep = "_")
stock(symbols_adj, currency = "USD", multiplier = 1)
strategy_st <- portfolio_st <- account_st <- "SMAC-20-50"
rm.strat(portfolio_st)
rm.strat(strategy_st)
initPortf(portfolio_st, symbols = symbols_adj,
initDate = initDate, currency = "USD")
initAcct(account_st, portfolios = portfolio_st,
initDate = initDate, currency = "USD",
initEq = 1000000)
initOrders(portfolio_st, store = TRUE)
strategy(strategy_st, store = TRUE)
As mentioned before, the strategy from my earlier article traded based on the current regime. This feature was implemented by using the signal function sigComparrison(), which tracks whether one indicator is greater than another (or vice versa). To trade based on a single crossing, use the sigCrossover() function (which is similar to sigComparison() but only generates a signal at the first point of crossing).
For some reason, the quantstrat developers wrote sigCrossover() so it returns either TRUE or NA rather than TRUE or FALSE. This leads to bad behavior for my system, so I’ve written an alternative, sigCrossover2(), that exhibits the latter behavior.
sigCrossover2 <- function(label, data = mktdata, columns,
relationship = c("gt", "lt", "eq", "gte", "lte"),
offset1 = 0, offset2 = 0) {
# A wrapper for sigCrossover, exhibiting the same behavior except returning
# an object containing TRUE/FALSE instead of TRUE/NA
res <- sigCrossover(label = label, data = data, columns = columns,
relationship = relationship, offset1 = offset1,
offset2 = offset2)
res[is.na(res)] <- FALSE
return(res)
}
Use this in my strategy instead of sigCrossover().
add.indicator(strategy = strategy_st, name = "SMA",
arguments = list(x = quote(Cl(mktdata)),
n = 20),
label = "fastMA")
## [1] "SMAC-20-50"
add.indicator(strategy = strategy_st, name = "SMA",
arguments = list(x = quote(Cl(mktdata)),
n = 50),
label = "slowMA")
## [1] "SMAC-20-50"
# Next comes trading signals
add.signal(strategy = strategy_st, name = "sigCrossover2", # Remember me?
arguments = list(columns = c("fastMA", "slowMA"),
relationship = "gt"),
label = "bull")
## [1] "SMAC-20-50"
add.signal(strategy = strategy_st, name = "sigCrossover2",
arguments = list(columns = c("fastMA", "slowMA"),
relationship = "lt"),
label = "bear")
## [1] "SMAC-20-50"
Now the strategy will trade based on a single crossover instead of balancing the portfolio so that 10% of equity is devoted to a signle position. I prefer this; the earlier strategy engaged in trades that would not be as profitable as those made at the point of crossover, driving up the number of transactions and thus the resulting fees.
Now, for trading in batches of 100. The original rule for opening a position used the function osMaxDollar() from the package IKTrading to size a position based on a dollar amount. The line floor(getEndEq(account_st_2, Date = timestamp) * .1)), passed to osMaxDollar()‘s maxSize and tradeSize parameters, instructed the system to place trades so they did not exceed 10% of the equity in the account. I modify the osMaxDollar() function to accept a batchSize parameter so that we can place trades in batches of 100.
# Based on Ilya Kipnis's osMaxDollar(); lots of recycled code
osMaxDollarBatch <- function(data, timestamp, orderqty, ordertype, orderside,
portfolio, symbol, prefer = "Open", tradeSize,
maxSize, batchSize = 100, integerQty = TRUE,
...) {
# An order sizing function that limits position size based on dollar value of
# the position, optionally controlling for number of batches to purchase
#
# Args:
# data: ??? (held over from Mr. Kipnis's original osMaxDollar function)
# timestamp: The current date being evaluated (some object, like a string,
# from which time can be inferred)
# orderqty: ??? (held over from Mr. Kipnis's original osMaxDollar function)
# ordertype: ??? (held over from Mr. Kipnis's original osMaxDollar
# function)
# orderside: ??? (held over from Mr. Kipnis's original osMaxDollar
# function)
# portfolio: A string representing the portfolio being treated; will be
# passed to getPosQty
# symbol: A string representing the symbol being traded
# prefer: A string that indicates whether the Open or Closing price is
# used for determining the price of the asset in backtesting (set
# to "Close" to use the closing price)
# tradeSize: Numeric, indicating the dollar value to transact (using
# negative numbers for selling short)
# maxSize: Numeric, indicating the dollar limit to the position (use
# negative numbers for the short side)
# batchSize: The number of stocks purchased per batch (only applies if
# integerQty is TRUE); default value is 100, but setting to 1
# effectively nullifies the batchSize
# integerQty: A boolean indicating whether or not to truncate to the
# nearest integer of contracts/shares/etc.
# ...: ??? (held over from Mr. Kipnis's original osMaxDollar function)
#
# Returns:
# A numeric quantity representing the number of shares to purchase
pos <- getPosQty(portfolio, symbol, timestamp)
if (prefer == "Close") {
price <- as.numeric(Cl(mktdata[timestamp, ]))
} else {
price <- as.numeric(Op(mktdata[timestamp, ]))
}
posVal <- pos * price
if (orderside == "short") {
dollarsToTransact 0) {
dollarsToTransact = 0
}
} else {
dollarsToTransact <- min(tradeSize, maxSize - posVal)
if (dollarsToTransact < 0) {
dollarsToTransact = 0
}
}
qty <- dollarsToTransact/price
if (integerQty) {
# Controlling for batch size only makes sense if we were working with
# integer quantities anyway; if we didn't care about being integers,
# why bother?
qty <- trunc(qty / batchSize) * batchSize
}
return(qty)
}
Now we add trading rules, replacing osMaxDollar() with our new osMaxDollarBatch():
Now the updated analytics:
# Now for analytics
updatePortf(portfolio_st)
## [1] "SMAC-20-50"
dateRange <- time(getPortfolio(portfolio_st)$summary)[-1]
updateAcct(account_st, dateRange)
## [1] "SMAC-20-50"
updateEndEq(account_st)
## [1] "SMAC-20-50"
tStats <- tradeStats(Portfolios = portfolio_st, use="trades",
inclZeroDays = FALSE)
tStats[, 4:ncol(tStats)] <- round(tStats[, 4:ncol(tStats)], 2)
print(data.frame(t(tStats[, -c(1,2)])))
## AAPL_adj AMZN_adj FB_adj GOOG_adj HPQ_adj
## Num.Txns 39.00 33.00 23.00 35.00 29.00
## Num.Trades 20.00 17.00 12.00 18.00 15.00
## Net.Trading.PL 96871.62 99943.99 116383.00 27476.81 51431.03
## Avg.Trade.PL 4843.58 5879.06 9698.58 1526.49 3428.74
## Med.Trade.PL 803.16 2755.00 4681.50 -830.50 -947.54
## Largest.Winner 40299.33 33740.00 74888.01 27149.18 54379.08
## Largest.Loser -7543.93 -14152.00 -17185.01 -9588.94 -15344.27
## Gross.Profits 134300.99 140947.00 141620.01 62944.47 121893.03
## Gross.Losses -37429.37 -41003.01 -25237.01 -35467.66 -70462.01
## Std.Dev.Trade.PL 12463.46 13985.62 22585.57 8247.78 20283.11
## Percent.Positive 55.00 64.71 75.00 38.89 40.00
## Percent.Negative 45.00 35.29 25.00 61.11 60.00
## Profit.Factor 3.59 3.44 5.61 1.77 1.73
## Avg.Win.Trade 12209.18 12813.36 15735.56 8992.07 20315.51
## Med.Win.Trade 7272.58 7277.00 8810.00 7254.76 10230.34
## Avg.Losing.Trade -4158.82 -6833.84 -8412.34 -3224.33 -7829.11
## Med.Losing.Trade -2837.23 -5474.50 -4752.00 -3537.02 -8954.58
## Avg.Daily.PL 4216.84 4519.19 10124.27 1386.81 2571.42
## Med.Daily.PL 304.94 2241.50 4346.99 -1121.00 -2841.81
## Std.Dev.Daily.PL 12476.98 13232.69 23637.40 8479.65 20764.83
## Ann.Sharpe 5.37 5.42 6.80 2.60 1.97
## Max.Drawdown -24834.98 -44379.01 -42976.02 -26932.62 -53384.75
## Profit.To.Max.Draw 3.90 2.25 2.71 1.02 0.96
## Avg.WinLoss.Ratio 2.94 1.87 1.87 2.79 2.59
## Med.WinLoss.Ratio 2.56 1.33 1.85 2.05 1.14
## Max.Equity 102428.61 99943.99 120250.00 36354.82 52580.84
## Min.Equity -11759.00 -3269.00 -14696.00 -21199.34 -53384.75
## End.Equity 96871.62 99943.99 116383.00 27476.81 51431.03
## IBM_adj MSFT_adj NFLX_adj NTDOY_adj SNY_adj
## Num.Txns 40.00 39.00 33.00 37.00 44.00
## Num.Trades 20.00 20.00 17.00 19.00 22.00
## Net.Trading.PL 10761.37 28739.24 232193.00 19555.00 -25046.79
## Avg.Trade.PL 538.07 1436.96 13658.41 1029.21 -1138.49
## Med.Trade.PL -651.11 -2351.12 2500.00 -1440.00 349.03
## Largest.Winner 20407.71 18337.28 151840.00 43452.01 14393.86
## Largest.Loser -7512.68 -11360.92 -29741.57 -8685.00 -16256.19
## Gross.Profits 45828.56 88916.25 295344.86 79742.00 61804.49
## Gross.Losses -35067.19 -60177.00 -63151.85 -60186.99 -86851.28
## Std.Dev.Trade.PL 6103.11 8723.81 39673.15 12193.29 8412.38
## Percent.Positive 40.00 45.00 64.71 42.11 54.55
## Percent.Negative 60.00 55.00 35.29 57.89 45.45
## Profit.Factor 1.31 1.48 4.68 1.32 0.71
## Avg.Win.Trade 5728.57 9879.58 26849.53 9967.75 5150.37
## Med.Win.Trade 3157.60 9507.66 8307.00 2560.50 3140.38
## Avg.Losing.Trade -2922.27 -5470.64 -10525.31 -5471.54 -8685.13
## Med.Losing.Trade -2153.73 -4955.57 -6835.07 -5832.00 -8354.55
## Avg.Daily.PL 538.07 1135.14 14355.81 226.94 -1138.49
## Med.Daily.PL -651.11 -2858.40 2847.00 -1512.00 349.03
## Std.Dev.Daily.PL 6103.11 8854.93 40866.48 12019.71 8412.38
## Ann.Sharpe 1.40 2.04 5.58 0.30 -2.15
## Max.Drawdown -37072.32 -30916.29 -54616.13 -50112.01 -36718.77
## Profit.To.Max.Draw 0.29 0.93 4.25 0.39 -0.68
## Avg.WinLoss.Ratio 1.96 1.81 2.55 1.82 0.59
## Med.WinLoss.Ratio 1.47 1.92 1.22 0.44 0.38
## Max.Equity 30247.68 42996.90 264941.01 40014.00 11471.08
## Min.Equity -6824.64 -17726.78 -3255.72 -32476.99 -32673.80
## End.Equity 10761.37 28739.24 232193.00 19555.00 -25046.79
## TWTR_adj YHOO_adj
## Num.Txns 9.00 43.00
## Num.Trades 5.00 22.00
## Net.Trading.PL 27051.99 106619.99
## Avg.Trade.PL 5410.40 4846.36
## Med.Trade.PL -27.01 -1296.00
## Largest.Winner 1800.00 54746.00
## Largest.Loser -10362.00 -7700.00
## Gross.Profits 43340.99 155671.99
## Gross.Losses -16289.00 -49051.99
## Std.Dev.Trade.PL 20764.84 15313.45
## Percent.Positive 40.00 31.82
## Percent.Negative 60.00 68.18
## Profit.Factor 2.66 3.17
## Avg.Win.Trade 21670.50 22238.86
## Med.Win.Trade 21670.50 14091.99
## Avg.Losing.Trade -5429.67 -3270.13
## Med.Losing.Trade -5900.00 -3105.01
## Avg.Daily.PL -3622.25 4406.10
## Med.Daily.PL -2963.50 -1404.00
## Std.Dev.Daily.PL 5565.94 15548.28
## Ann.Sharpe -10.33 4.50
## Max.Drawdown -60115.00 -33162.01
## Profit.To.Max.Draw 0.45 3.22
## Avg.WinLoss.Ratio 3.99 6.80
## Med.WinLoss.Ratio 3.67 4.54
## Max.Equity 42758.99 110806.00
## Min.Equity -17356.00 -10732.99
## End.Equity 27051.99 106619.99
final_acct <- getAccount(account_st)
plot(final_acct$summary$End.Eq["2010/2016"], main = "Portfolio Equity")
getSymbols("SPY", from = start, to = end)
## [1] "SPY"
# A crude estimate of end portfolio value from buying and holding SPY
plot(final_acct$summary$End.Eq["2010/2016"] / 1000000,
main = "Portfolio Equity", ylim = c(0.8, 2.5))
lines(SPY$SPY.Adjusted / SPY$SPY.Adjusted[[1]], col = "blue")
Modelling Fees
We’ve now replicated the Python analysis in R (or as close as we will get using quantstrat). Let’s now model our portfolio when a 2% commission is applied to every trade (that is, you pay the broker 2% of the total value of the trade, whether you buy or sell). quantstrat allows you to easily model such a fee. To do so, create a function that computes the fee given the number of shares being sold and the share price. Then pass this to the argument TxnFees when you create the rule (this takes either a function, a function name, or a negative number representing the fee). Be sure to add this to all rules (though it need not be the same fee structure for each rule).
fee <- function(TxnQty, TxnPrice, Symbol) {
# A function for computing a transaction fee that is 2% of total value of
# transaction
#
# Args:
# TxnQty: Numeric for number of shares being traded
# TxnPrice: Numeric for price per share
# Symbol: The symbol being traded (not used here, but will be passed)
#
# Returns:
# The fee to be applied
return(-0.02 * abs(TxnQty * TxnPrice))
}
rm(list = ls(.blotter), envir = .blotter) # Clear blotter environment
stock(symbols_adj, currency = "USD", multiplier = 1)
rm.strat(portfolio_st)
rm.strat(strategy_st)
initPortf(portfolio_st, symbols = symbols_adj,
initDate = initDate, currency = "USD")
initAcct(account_st, portfolios = portfolio_st,
initDate = initDate, currency = "USD",
initEq = 1000000)
initOrders(portfolio_st, store = TRUE)
strategy(strategy_st, store = TRUE)
add.indicator(strategy = strategy_st, name = "SMA",
arguments = list(x = quote(Cl(mktdata)),
n = 20),
label = "fastMA")
add.indicator(strategy = strategy_st, name = "SMA",
arguments = list(x = quote(Cl(mktdata)),
n = 50),
label = "slowMA")
add.signal(strategy = strategy_st, name = "sigCrossover2", # Remember me?
arguments = list(columns = c("fastMA", "slowMA"),
relationship = "gt"),
label = "bull")
add.signal(strategy = strategy_st, name = "sigCrossover2",
arguments = list(columns = c("fastMA", "slowMA"),
relationship = "lt"),
label = "bear")
add.rule(strategy = strategy_st, name = "ruleSignal",
arguments = list(sigcol = "bull",
sigval = TRUE,
ordertype = "market",
orderside = "long",
replace = FALSE,
TxnFees = "fee", # Apply the fee with the trade
# If you wanted a flat fee, replace
# this with a negative number
# representing the fee
prefer = "Open",
osFUN = osMaxDollarBatch,
maxSize = quote(floor(getEndEq(account_st,
Date = timestamp) * .1)),
tradeSize = quote(floor(getEndEq(account_st,
Date = timestamp) * .1)),
batchSize = 100),
type = "enter", path.dep = TRUE, label = "buy")
add.rule(strategy = strategy_st, name = "ruleSignal",
arguments = list(sigcol = "bear",
sigval = TRUE,
orderqty = "all",
ordertype = "market",
orderside = "long",
replace = FALSE,
TxnFees = "fee", # Apply the fee with the trade
prefer = "Open"),
type = "exit", path.dep = TRUE, label = "sell")
# Having set up the strategy, we now backtest
applyStrategy(strategy_st, portfolios = portfolio_st)
Let’s now look at our portfolio.
# Now for analytics
updatePortf(portfolio_st)
## [1] "SMAC-20-50"
dateRange <- time(getPortfolio(portfolio_st)$summary)[-1]
updateAcct(account_st, dateRange)
## [1] "SMAC-20-50"
updateEndEq(account_st)
## [1] "SMAC-20-50"
tStats <- tradeStats(Portfolios = portfolio_st, use="trades",
inclZeroDays = FALSE)
tStats[, 4:ncol(tStats)] <- round(tStats[, 4:ncol(tStats)], 2)
print(data.frame(t(tStats[, -c(1,2)])))
## AAPL_adj AMZN_adj FB_adj GOOG_adj HPQ_adj
## Num.Txns 39.00 33.00 23.00 35.00 29.00
## Num.Trades 20.00 17.00 12.00 18.00 15.00
## Net.Trading.PL 20624.18 43224.01 68896.98 -27891.65 -6635.25
## Avg.Trade.PL 4843.58 5879.06 9698.58 1526.49 3428.74
## Med.Trade.PL 803.16 2755.00 4681.50 -830.50 -947.54
## Largest.Winner 37593.97 31266.76 71410.75 24869.77 51318.19
## Largest.Loser -9426.91 -15583.36 -18874.11 -11201.90 -17051.87
## Gross.Profits 134300.99 140947.00 141620.01 62944.47 121893.03
## Gross.Losses -37429.37 -41003.01 -25237.01 -35467.66 -70462.01
## Std.Dev.Trade.PL 12463.46 13985.62 22585.57 8247.78 20283.11
## Percent.Positive 55.00 64.71 75.00 38.89 40.00
## Percent.Negative 45.00 35.29 25.00 61.11 60.00
## Profit.Factor 3.59 3.44 5.61 1.77 1.73
## Avg.Win.Trade 12209.18 12813.36 15735.56 8992.07 20315.51
## Med.Win.Trade 7272.58 7277.00 8810.00 7254.76 10230.34
## Avg.Losing.Trade -4158.82 -6833.84 -8412.34 -3224.33 -7829.11
## Med.Losing.Trade -2837.23 -5474.50 -4752.00 -3537.02 -8954.58
## Avg.Daily.PL 2218.84 2736.55 7953.30 -212.11 542.98
## Med.Daily.PL -1549.09 541.93 2355.65 -2396.58 -4741.84
## Std.Dev.Daily.PL 12205.15 12945.81 23168.30 8308.18 20348.24
## Ann.Sharpe 2.89 3.36 5.45 -0.41 0.42
## Max.Drawdown -56123.63 -60668.36 -51083.34 -46522.72 -81892.46
## Profit.To.Max.Draw 0.37 0.71 1.35 -0.60 -0.08
## Avg.WinLoss.Ratio 2.94 1.87 1.87 2.79 2.59
## Med.WinLoss.Ratio 2.56 1.33 1.85 2.05 1.14
## Max.Equity 59846.94 43224.01 80524.92 4063.82 8591.39
## Min.Equity -15308.91 -18513.81 -24333.09 -42458.90 -81892.46
## End.Equity 20624.18 43224.01 68896.98 -27891.65 -6635.25
## IBM_adj MSFT_adj NFLX_adj NTDOY_adj SNY_adj
## Num.Txns 40.00 39.00 33.00 37.00 44.00
## Num.Trades 20.00 20.00 17.00 19.00 22.00
## Net.Trading.PL -62316.03 -48328.40 163636.67 -53742.98 -111417.94
## Avg.Trade.PL 538.07 1436.96 13658.41 1029.21 -1138.49
## Med.Trade.PL -651.11 -2351.12 2500.00 -1440.00 349.03
## Largest.Winner 18145.45 15954.84 146848.00 40548.11 12141.77
## Largest.Loser -9256.52 -13175.01 -31173.09 -10517.40 -17910.22
## Gross.Profits 45828.56 88916.25 295344.86 79742.00 61804.49
## Gross.Losses -35067.19 -60177.00 -63151.85 -60186.99 -86851.28
## Std.Dev.Trade.PL 6103.11 8723.81 39673.15 12193.29 8412.38
## Percent.Positive 40.00 45.00 64.71 42.11 54.55
## Percent.Negative 60.00 55.00 35.29 57.89 45.45
## Profit.Factor 1.31 1.48 4.68 1.32 0.71
## Avg.Win.Trade 5728.57 9879.58 26849.53 9967.75 5150.37
## Med.Win.Trade 3157.60 9507.66 8307.00 2560.50 3140.38
## Avg.Losing.Trade -2922.27 -5470.64 -10525.31 -5471.54 -8685.13
## Med.Losing.Trade -2153.73 -4955.57 -6835.07 -5832.00 -8354.55
## Avg.Daily.PL -1294.25 -853.51 12129.90 -1757.68 -3090.09
## Med.Daily.PL -2449.41 -4777.80 783.80 -3466.08 -1631.09
## Std.Dev.Daily.PL 5974.29 8677.26 40049.20 11764.24 8256.61
## Ann.Sharpe -3.44 -1.56 4.81 -2.37 -5.94
## Max.Drawdown -75697.82 -60653.33 -63238.29 -82945.57 -111417.94
## Profit.To.Max.Draw -0.82 -0.80 2.59 -0.65 -1.00
## Avg.WinLoss.Ratio 1.96 1.81 2.55 1.82 0.59
## Med.WinLoss.Ratio 1.47 1.92 1.22 0.44 0.38
## Max.Equity 8016.15 3615.01 218329.97 1177.74 0.00
## Min.Equity -67681.68 -57038.32 -5157.36 -81767.83 -111417.94
## End.Equity -62316.03 -48328.40 163636.67 -53742.98 -111417.94
## TWTR_adj YHOO_adj
## Num.Txns 9.00 43.00
## Num.Trades 5.00 22.00
## Net.Trading.PL 9470.41 19783.95
## Avg.Trade.PL 5410.40 4846.36
## Med.Trade.PL -27.01 -1296.00
## Largest.Winner 0.00 51665.84
## Largest.Loser -12099.12 -9466.80
## Gross.Profits 43340.99 155671.99
## Gross.Losses -16289.00 -49051.99
## Std.Dev.Trade.PL 20764.84 15313.45
## Percent.Positive 40.00 31.82
## Percent.Negative 60.00 68.18
## Profit.Factor 2.66 3.17
## Avg.Win.Trade 21670.50 22238.86
## Med.Win.Trade 21670.50 14091.99
## Avg.Losing.Trade -5429.67 -3270.13
## Med.Losing.Trade -5900.00 -3105.01
## Avg.Daily.PL -5536.07 2341.16
## Med.Daily.PL -4925.13 -3332.34
## Std.Dev.Daily.PL 5438.76 15231.86
## Ann.Sharpe -16.16 2.44
## Max.Drawdown -71707.02 -68750.59
## Profit.To.Max.Draw 0.13 0.29
## Avg.WinLoss.Ratio 3.99 6.80
## Med.WinLoss.Ratio 3.67 4.54
## Max.Equity 36769.43 42502.03
## Min.Equity -34937.58 -55447.71
## End.Equity 9470.41 19783.95
final_acct <- getAccount(account_st)
plot(final_acct$summary$End.Eq["2010/2016"], main = "Portfolio Equity")
Before the fees were accounted for, we made a modest profit, though one worse than if we bought and held SPY. But with this commission structure, we barely break even!
Not surprisingly, our strategy looks even worse when compared to buy-and-hold-SPY.
plot(final_acct$summary$End.Eq["2010/2016"] / 1000000,
main = "Portfolio Equity", ylim = c(0.8, 2.5))
lines(SPY$SPY.Adjusted / SPY$SPY.Adjusted[[1]], col = "blue")
The 2% commission is very punishing. With this commission, we would need to profit by about 4% in order for a trade to really, truly be profitable. This is harder than it sounds (and doesn’t even allow for losses), and the losses add up.
Of course if you’re using an online trading platform you may opt for a broker that charges a flat rate, say, $10 per trade. This means that with each trade your profit would need to be at least $10 in order to beat the fee. Also, the larger the trade, the smaller the fee is relative to the trade, making the trade more likely to be profitable.
Let’s suppose a $1,000,000 portfolio pays a $100 fee per trade. I model this below.
rm(list = ls(.blotter), envir = .blotter) # Clear blotter environment
stock(symbols_adj, currency = "USD", multiplier = 1)
rm.strat(portfolio_st)
rm.strat(strategy_st)
initPortf(portfolio_st, symbols = symbols_adj,
initDate = initDate, currency = "USD")
initAcct(account_st, portfolios = portfolio_st,
initDate = initDate, currency = "USD",
initEq = 1000000)
initOrders(portfolio_st, store = TRUE)
strategy(strategy_st, store = TRUE)
add.indicator(strategy = strategy_st, name = "SMA",
arguments = list(x = quote(Cl(mktdata)),
n = 20),
label = "fastMA")
add.indicator(strategy = strategy_st, name = "SMA",
arguments = list(x = quote(Cl(mktdata)),
n = 50),
label = "slowMA")
add.signal(strategy = strategy_st, name = "sigCrossover2", # Remember me?
arguments = list(columns = c("fastMA", "slowMA"),
relationship = "gt"),
label = "bull")
add.signal(strategy = strategy_st, name = "sigCrossover2",
arguments = list(columns = c("fastMA", "slowMA"),
relationship = "lt"),
label = "bear")
add.rule(strategy = strategy_st, name = "ruleSignal",
arguments = list(sigcol = "bull",
sigval = TRUE,
ordertype = "market",
orderside = "long",
replace = FALSE,
TxnFees = -100, # Apply the fee with the trade
prefer = "Open",
osFUN = osMaxDollarBatch,
maxSize = quote(floor(getEndEq(account_st,
Date = timestamp) * .1)),
tradeSize = quote(floor(getEndEq(account_st,
Date = timestamp) * .1)),
batchSize = 100),
type = "enter", path.dep = TRUE, label = "buy")
add.rule(strategy = strategy_st, name = "ruleSignal",
arguments = list(sigcol = "bear",
sigval = TRUE,
orderqty = "all",
ordertype = "market",
orderside = "long",
replace = FALSE,
TxnFees = -100,
prefer = "Open"),
type = "exit", path.dep = TRUE, label = "sell")
# Having set up the strategy, we now backtest
applyStrategy(strategy_st, portfolios = portfolio_st)
Now for comparison with SPY.
updatePortf(portfolio_st)
## [1] "SMAC-20-50"
dateRange <- time(getPortfolio(portfolio_st)$summary)[-1]
updateAcct(account_st, dateRange)
## [1] "SMAC-20-50"
updateEndEq(account_st)
## [1] "SMAC-20-50"
final_acct <- getAccount(account_st)
plot(final_acct$summary$End.Eq["2010/2016"] / 1000000,
main = "Portfolio Equity", ylim = c(0.8, 2.5))
lines(SPY$SPY.Adjusted / SPY$SPY.Adjusted[[1]], col = "blue")
Not quite as punishing as the 2% rate, though admittedly slightly less than when no fee was applied at all. And while you may say, “No one would ever go with a broker charging $100 per trade when they can get $10 or $5 per trade,” remember most people don’t have $1,000,000 accounts (and I’m sure that if you applied a $5 fee to a more modest account, you’d discover that $5 per trade actually makes a big difference: consider that a challenge).
Conclusion
Fees are only the beginning of potential, unmodelled avenues through which a backtest can lead to optimistic results. There’s slippage, where orders are placed at unwanted prices that lead to further losses than initially planned, and becomes a serious problem in periods of high volatility. Another potential problem that cannot be modelled by backtesting at all is affecting the market with an order. Prices in backtesting are treated as givens, but in the real world, an order affects the price of the asset bought and could lead to others reacting in unexpected ways. There’s also the propensity for backtesting to overfit data; the strategy appears to be profitable when it merely managed to learn the ups and downs of the historic period.
In a practice that relies on slim profits, where the market may only be just barely inefficient, there’s little room for error. Do what you can to accurately model what a trade would actually look like (especially when quantstrat makes doing so easy). Little price differences can lead to big differences in the end.
# Session details sessionInfo() ## R version 3.3.3 (2017-03-06) ## Platform: i686-pc-linux-gnu (32-bit) ## Running under: Ubuntu 16.04.2 LTS ## ## locale: ## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C ## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 ## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 ## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C ## [9] LC_ADDRESS=C LC_TELEPHONE=C ## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C ## ## attached base packages: ## [1] methods stats graphics grDevices utils datasets base ## ## other attached packages: ## [1] IKTrading_1.0 roxygen2_6.0.1 ## [3] digest_0.6.12 Rcpp_0.12.10 ## [5] quantstrat_0.9.1739 foreach_1.4.4 ## [7] blotter_0.9.1741 PerformanceAnalytics_1.4.4000 ## [9] FinancialInstrument_1.2.0 quantmod_0.4-7 ## [11] TTR_0.23-1 xts_0.9-7 ## [13] zoo_1.7-14 RWordPress_0.2-3 ## [15] optparse_1.3.2 knitr_1.15.1 ## ## loaded via a namespace (and not attached): ## [1] xml2_1.1.1 magrittr_1.5 getopt_1.20.0 lattice_0.20-35 ## [5] R6_2.2.0 highr_0.6 stringr_1.2.0 tools_3.3.3 ## [9] grid_3.3.3 commonmark_1.2 iterators_1.0.8 bitops_1.0-6 ## [13] codetools_0.2-15 RCurl_1.95-4.8 evaluate_0.10 stringi_1.1.3 ## [17] XMLRPC_0.3-0 XML_3.98-1.5
1 The EMH claims that information about past prices will not help predict future price movements, so technical analysis is pure alchemy. However, we need to remember that price data and future prices are being produced by people who do look at past prices. Suppose a technical indicator indicates a bullish movement in prices; for example, a slow moving average crossed over a fast moving average. Forget whether there’s any inherent reason this should lead to a bullish price movement; there’s a good chance there isn’t any, and all the narratives explaining the rationale of the indicator are rubbish (as I’m inclined to believe). If a trader sees the signal, and believes that enough traders: (a) see the signal; (b) interpret the signal in the same way; and © will act on the signal as it recommends, then that trader should do the same. Enough people believe the asset is undervalued to make it act as if it were undervalued, and the trader should go long. So technical signals could become self-fulfilling prophecies, like J.M. Keynes’ beauty contest.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.