auto-optimization framework.afl

一个更精度的平滑涵数, 可用于股票交易系统.用于Amibroker 平台

//------------------------------------------------------------------------------
//
//  Formula Name:    Auto-Optimization Framework
//  Author/Uploader: Dave Merrill 
//  E-mail:          
//  Date/Time Added: 2003-10-22 11:58:47
//  Origin:          
//  Keywords:        optimize,auto-optimize,backtest
//  Level:           advanced
//  Flags:           system,exploration,function
//  Formula URL:     http://www.amibroker.com/library/formula.php?id=304
//  Details URL:     http://www.amibroker.com/library/detail.php?id=304
//
//------------------------------------------------------------------------------
//
//  this is a backtest framework that continously and automatically optimizes
//  settings for a trading rule you provide. it chooses settings that measure
//  best over a selectable lookback period. this re-optimization can be done
//  every bar, at a user-definable interval, or once at the start of trading.
//  it requires AFL editing to use; instructions are provided.
//
//------------------------------------------------------------------------------

//////////////////////////////////////////////////////////////////////////
// INTRO
//////////////////////////////////////////////////////////////////////////
/*

------------------------------------------------------------------------
Auto-Optimization Framework
v 1.0 (hah) 10/20/03 Dave Merrill (email: dmerrill at usa dot net)
thanks to Tomasz Janeczko for AmiBroker, and everyone in the AmiBroker groups at Yahoo
comments, suggestions, bug reports etc are most welcome
------------------------------------------------------------------------


WHAT DOES THIS DO?

you've probably done walk-forward optimizations, where you optimize a trading system over some time period, then trade with those settings for a while, optimize again at some later date, trade on, etc.. in fact, this is like real life, unless you assume your initial settings will be optimal forever and you'll never change them.

this code is a backtest framework that does this continously and automatically. you provide a trading rule, and it constantly optimizes the parameters for it, always using the settings that measure best over a selectable lookback period. this re-optimization can be done every bar, at a user-definable interval, or once at the start of trading.

one interesting fact is that so far, almost no trading rules I've tested are very profitable when managed this way, with any framework settings I've tried. the ones that are are far from universally successful over various stocks and time frames. I find this very interesting and quite puzzling, as is well documented on the AB group. maybe there are bugs that cause this; please let me know about any you find. maybe the equity evaluation function or other framework behaviors could be improved; please let me know if you come up with great changes. maybe it's just not a very effective method; if you can explain to me why this is so, please do!

and lastly, if you find any hugely profitable uses for this thing, or other interesting insights from poking around with it, PLEASE let me in on 'em! (:-)


IMPORTANT USAGE NOTE!!!!

parts of this code belong to the framework itself, and shouldn't be modified in normal use. other parts you need to edit, to select the specific trading rule you want to test, and establish settings for how the framework will operate. 

all sections are clearly labelled as editable or not. it's in your best interest not to change the ones that say NOT EDITABLE, at least to start with.


DON'T BE IMTIMIDATED

though this is pretty long, it's actually not that complicated, either to use or in its internals. it's broken down into sections, most of which you don't need to even look at. if I could hide the no-touchy bits away I would (:-). also, a lot of it is explanation, not code, and a lot of the code is a collection of sample trading rules.


INSTRUCTIONS

1) add trading rule(s) you want to test to the CUSTOM TRADING RULES section. see comments there for how your rules will be called, and what they need to do. some examples are provided. you can have as many rules here as you want, but only one can be in use at a time. step 3 below controls which one is active.

2) if those rules require other custom functions, add them to the CUSTOM UTILITY FUNCTIONS section.

3) edit the framework function BuySellRule(), at the start of the FRAMEWORK CONFIGURATION section, to call the trading rule you want to test now.

4) edit the SetParameterRanges() call, next in the FRAMEWORK CONFIGURATION section, to set the ranges over which you want to test each of the two optimization parameters. some examples are provided; comment out all but the one you want to use.

5) if desired, edit the rest of the FRAMEWORK CONFIGURATION section, to control various aspects of how the framework itself behaves.

6) set the AA range, stock universes you want to test, and other AA settings, then run a Backtest or Portfolio Backtest.

7) if you want to see the parameters selected and other decisions being made under the hood, you can Explore instead of backtest. if desired, edit the EXPLORATION section at the end to change the columns and bars displayed; some possible extensions and modifications are provided. note that exploring doesn't actually trade, it's only a readout for your interest.


GETTING STARTED

this comes set up to run the included trading rule 'BuySellCCI'. backtest it on, say, the NASDAQ Composite Index, to get an idea of how it performs (nothing great). explore it to see the parameter values the framework selected (the best_p1 column). try changing the parameter ranges tested, as described in the instructions above.

BuySellCCI uses a single parameter, requiring only 40 tests to optimize. this makes it quicker to run than most two-parameter systems, which typically need about 40 x 40 tests or more, depending on the number of values you want to test. performance can definitely be an issue as the number of tests goes up, and even single parameter systems can be pretty slow with a large number of stocks.

in the FRAMEWORK CONFIGURATION section, try changing equity_lookback_bars (how far back performance is evaluated during optimization) and equity_lookback_frequency (how often optimization is done). if you want, you can actually optimize these; possible optimization settings are provided. be cautious in interpreting those results though. I'd do it more for a quick sense of how things behave than to necessarily choose the highest performing settings. at the very least, test a variety of stocks and time frames with any settings you arrive at, to make sure they're not a fluke.

try some of the other included trading rules, maybe starting with the MA/EMA/DEMA/TEMA cross rules. hook them up and select parameter ranges to test as described in the instructions above.

try adding your own trading rules, and experiment with optimization ranges and equity feedback parameters.

try modifying the performance scoring methods, and adding your own.

*/


//////////////////////////////////////////////////////////////////////////
// FRAMEWORK INIT - NOT EDITABLE
//////////////////////////////////////////////////////////////////////////

buy = sell = short = cover = 0;
equity_lookback_bars = equity_lookback_frequency = equity_lookback_smoothing = equity_drawdown_discount_pct = 0;
e = best_e = best_perf_score = best_p1 = best_p2 = 0;
p1_start = p2_start = 2;
p1_end = p2_end = 20;
p1_step = p2_step = 1;
bar_index = BarIndex();


//////////////////////////////////////////////////////////////////////////
// FRAMEWORK FUNCTIONS - NOT EDITABLE
//////////////////////////////////////////////////////////////////////////

function SetParameterRanges(p1_start_p, p1_end_p, p1_step_p, p2_start_p, p2_end_p, p2_step_p) {
	if(p1_step_p != 0) {
		p1_start = p1_start_p;
		p1_end = p1_end_p;
		p1_step = p1_step_p;
	} else {
		p1_start = p1_end = p1_step = 1;
	}
	if(p2_step_p != 0) {
		p2_start = p2_start_p;
		p2_end = p2_end_p;
		p2_step = p2_step_p;
	} else {
		p2_start = p2_end = p2_step = 1;
	}
}
function PerformanceScoreMDDDiscount(e, perf_score) {
	e_max = Highest(e); // peak equity so far
	mdd = Highest(e_max - e);	// max drawdown so far
	mdd_fraction = Highest(mdd / e_max);	// fraction max drawdown is of peak equity
	perf_score = perf_score - (perf_score * mdd_fraction * (equity_drawdown_discount_pct/100));	// reduce score by mdd fraction scaled by drawdown discount
	return perf_score;
}


//////////////////////////////////////////////////////////////////////////
// CUSTOM UTILITY FUNCTIONS - EDITABLE
//////////////////////////////////////////////////////////////////////////

function EMAx(array, period) {	// use in place of AB EMA function when period needs to be an array
	return AMA(array, 2 / (period + 1));
}
function StochTransform(array, period) {
	return 100*(array-LLV(array,period))/(HHV(array,period)-LLV(array,period));
}
function MeanDev(array, mean, range) {
	result = 0;
	for(i = LastValue(Highest(range)); i < BarCount; i++) {
		result[i] = 0;	// the mean is not 'moving' over the range (outside the loop)
		tm = mean[i]; 
		for(j = 0; j < range[i]; j++) {
			result[i] = result[i] + abs(array[i - j] - tm);
		}
		result[i] = result[i] / range[i]; 
	}
	return result;
}
function CCIx(period) {	// use in place of AB CCI function when period needs to be an array
	// CCI = (TypicalPrice - MA(TypicalPrice, 20)) / (.015 x MeanDeviation)
	SMATP = MA(Avg, period);
	MD = MeanDev(Avg, SMATP, period);
	result = (Avg - SMATP) / (0.015 * MD);
	return result;
}
function TradeDivergences(array1, array2) {
	dir1 = IIf(array1 > Ref(array1, -1), 1, IIf(array1 < Ref(array1, -1), -1, 0));
	dir2 = IIf(array2 > Ref(array2, -1), 1, IIf(array2 < Ref(array2, -1), -1, 0));
	buy = cover = (dir1 == 1) and (dir2 == -1);
	sell = short = (dir1 == -1) and (dir2 == 1);
}
function TradeReversalsArith(array, threshold) {
	array = last_hi = last_lo = IIf(IsNull(array), -1, array);
	last_signal[0] = 0;
	buy = sell = 0;
	for(i = 1; i < BarCount; i++) {
		buy[i] = array[i] >= last_lo[i-1] + threshold[i] and last_signal[i-1] != 1 and array[i-1] != -1;
		sell[i] = array[i] <= last_hi[i-1] - threshold[i] and last_signal[i-1] != -1 and array[i-1] != -1;
		last_signal[i] = IIf(buy[i], 1, IIf(sell[i], -1, last_signal[i-1]));
		always_track = array[i-1] == -1 or buy[i] or sell[i];
		last_lo[i] = IIf(always_track or array[i] < last_lo[i-1], array[i], last_lo[i-1]);
		last_hi[i] = IIf(always_track or array[i] > last_hi[i-1], array[i], last_hi[i-1]);
	}
	short = sell;
	cover = buy;
}


/////////////////////////////////////////////////////////////////////////
// CUSTOM TRADING RULES - EDITABLE
/////////////////////////////////////////////////////////////////////////

/*

this is where you put the various trading rules you may want to test. only one rule can be in use at a time. select which one actually gets used by editing the framework function BuySellRule(), at the start of the FRAMEWORK CONFIGURATION section.

all rules should set buy, sell, short and cover, in response to the parameters passed to it and whatever logic you want.

your rule will be passed 3 parameters:
	dynamic (true/false)	: true if parameters are arrays, not just single values; see ABOUT THE 'DYNAMIC' PARAMETER, below
	p1 (number/array)		: first optimization parameter
	p2 (number/array)		: second optimization parameter

p1 and p2 are the values being optimized on. trading rules can use them for any optimizable purpose, MA periods or crossover thresholds for example.

during optimization, the rule in use will be called multiple times, once with each combination of p1 and p2 being tested. at this time, p1 and p2 will be simple numeric values, not arrays. once optimal settings have been determined for every bar, the rule will be called once more, to generate actual trading signals. this time, p1 and p2 will be arrays, each bar containing the optimal p1 and p2 value for that bar.

ABOUT THE 'DYNAMIC' PARAMETER

not all AB functions can take arrays as inputs. for example, MA, TEMA and DEMA can use an array as their period, but EMA can't. it's usually possible to code an equivalent that can in AFL, but it may be slower, or in a DLL, which I didn't want to depend on for this. this speed difference can be big, since the trading rule is called many times during optimization. 

if desired, your rule can include both slower array-capable code for use in the final signal-generation phase, and faster, non-array-capable code for use during the many optimization tries. if you do this, use the 'dynamic' parameter to know which to call. see BuySellCCI, below, for an example.

*/

function BuySellEMACrossPrice(dynamic, p1, p2) {	// p1 = EMA period
	m = EMAx(c, p1);
	buy = cover = Cross(c, m);
	sell = short = Cross(m, c);
}
function BuySellMACross(dynamic, p1, p2) {	// p1 = MA1 period, p2 = MA2 period increase over MA1 period
	m1 = MA(c, p1);
	m2 = MA(c, p1 + p2);
	buy = cover = Cross(m1, m2);
	sell = short = Cross(m2, m1);
}
function BuySellEMACross(dynamic, p1, p2) {	// p1 = MA1 period, p2 = MA2 period increase over MA1 period
	m1 = EMAx(c, p1);
	m2 = EMAx(c, p2);
	buy = cover = Cross(m1, m2);
	sell = short = Cross(m2, m1);
}
function BuySellDEMACross(dynamic, p1, p2) {	// p1 = MA1 period, p2 = MA2 period increase over MA1 period
	m1 = DEMA(c, p1);
	m2 = DEMA(c, p1 + p2);
	buy = cover = Cross(m1, m2);
	sell = short = Cross(m2, m1);
}
function BuySellTEMACross(dynamic, p1, p2) {	// p1 = MA1 period, p2 = MA2 period increase over MA1 period
	m1 = TEMA(c, p1);
	m2 = TEMA(c, p1 + p2);
	buy = cover = Cross(m1, m2);
	sell = short = Cross(m2, m1);
	//short = cover = 0;	// at least sometimes, short is perf_scoreable, but only barely, so RAR drops
}
function BuySellCCI(dynamic, p1, p2) {	// p1 = CCI period; fixed threshold of zero
	If(dynamic) {
		cc = CCIx(p1);
	} else {
		cc = CCI(p1);
	}
	buy = cover = Cross(cc, 0);
	sell = short = Cross(0, cc);
}
function BuySellCCI2(dynamic, p1, p2) {	// p1 = CCI period; moveable symetrical thresholds, mirrored above and below zero for long and short
	If(dynamic) {
		cc = CCIx(p1);
	} else {
		cc = CCI(p1);
	}
	buy = cover = Cross(cc, p2);
	sell = short = Cross(-p2, cc);
}
function BuySellCCI3(dynamic, p1, p2) {	// p1 = CCI period; moveable absolute threshold, same numeric level for long and short
	If(dynamic) {
		cc = CCIx(p1);
	} else {
		cc = CCI(p1);
	}
	buy = cover = Cross(cc, p2);
	sell = short = Cross(p2, cc);
}
function BuySellStochCCI(dynamic, p1, p2) {	// p1 = CCI period
	cc = CCIx(period);