New Training Video!
Click Here to see a training video of this step!
By default, a new OneClick Search will only look at the Average Annual Return. However, to create a robust trading system, many more fields need to be evaluated than simply the Average Annual Return. Not only does one need to consider things like holding periods and drawdowns, but there are many other important performance values that help prevent curve-fitting.
While it is the responsibility of the user to customize each OneClick Search to their own specific needs, the descriptions below offer an indication of the many things that can be done. Technical analysts often differ in the methods they use and the performance values they observe to analyze their trading systems. But one of the most helpful aspects of the OneClick process is that one must clearly define the qualities their trading systems must have before it will be accepted for trading. Interestingly, this is information one should have regardless of their trading software, since even a manual search for trading systems requires the same degree of consideration. But by defining it mathematically within the OneClick Setups, a search can be created that follows your requirements precisely.
Creating Strategy Filters
Scoring the Strategies
Parameter Shift Tests
Restarting a OneClick Search
Creating Strategy Filters
Strategy filters allow you to reject strategies that have performance values considered unacceptable. For example, you might want to reject strategies that have only 5 trades in the back test, since this wouldn't provide enough historical data for statistical significance. Or you might want to reject strategies with drawdowns greater than 25%, since these might be considered too great a risk. There is no limit to the number of filters you can create, and any performance value in the Detailed Analysis can be used as a filter.
To create a filter, first update your OneClick Setup through the Setups > OneClick Setups menu, and then navigate to the Strategy Performance tab.
As you can see, there is one strategy filter already in place. A field named Average Annual Return is listed in the Evaluation Fields on the left. On the right, the Best and Worst Rated Values are configured as 50 and 5, and a box is checked below the Worst Rated Value to Reject Strategies Outside This Value. In short, any strategies that do not have an Average Annual Return value of at least 5% will be rejected by this strategy filter.
The above is one example, but we'll be adding two additional filters below. First, we'll add a new filter requiring that there be at least 20 trades in the back test. And second, we'll add a new filter for Kurtosis, a performance number that identifies how evenly the returns are distributed across the different trades.
To begin adding a new Evaluation Field, click the Add button below the Evaluation Fields listing.
First, enter a name such as Number of Trades. Next, move the cursor into the Definition entry and click the Fields button.
Within the Field Finder window, the listing contains macros that reference every available performance value. The macro is on the left in the listing, and its associated description is on the right. At the top, you'll also see that you can change the listing to display performance values from any of the Detailed Analysis reports, or from the Combination Results Listing.
For now, leave the Report Source as Fixed Trade Equity Report, and then locate and highlight the $F_NumTrades - Number of Trades item. Then click OK.
Your OneClick Field Setup should look like the above. Click OK.
After Number of Trades is added to the Evaluation Fields listing, you should then set the Best Rated Value as 100 and the Worst Rated Value as 20. Finally, check the box named Reject Strategies Outside This Value. This tells the OneClick Search to reject any strategies that don't have at least 20 trades in the back test.
Next, we'll add another Evaluation Field for Kurtosis. Again click the Add button below the Evaluation Fields.
You can click the Fields button to locate the Kurtosis macro, or you can simply type it in as in the example above. When your OneClick Field Setup looks identical to the above, click OK to save the setting.
When Kurtosis has been added as an Evaluation Field, leave the Best Rated Value at 0 and change the Worst Rated Value to 5. Also check the Reject Strategies Outside This Value box. The use of this strategy filter will reject any strategies where the Kurtosis is greater than 5, helping ensure we see only systems where returns are evenly distributed across the trades.
As mentioned earlier, there is no limit to the number of filters you can add. You might want to add additional strategy filters for drawdowns, standard deviations, worst year or worst month, Monte Carlo returns, or any other performance value that's important to you. But keep in mind that the more filters you add, the difficult it will be to find systems that meet your criteria. If your filters are too strict, your OneClick Search might not find any trading systems at all.
Scoring the Strategies
What happens if there are dozens of strategies that pass the strategy filters entered in the above section? How does the OneClick Search determine which strategy is best? The answer is a flexible scoring mechanism that analyzes the Evaluation Fields you've defined, and weighs them according to your own definitions. The result is a unique score for each strategy that allows it to be ranked against the others.
In our example setup, we'll be changing the settings of our 3 Evaluation Fields so that the score from the Average Annual Return and Kurtosis each make up 50% of the score, and the Number of Trades is not included in the scoring at all.
The first step will be to remove Number of Trades from the scoring altogether. Highlight Number of Trades in the Evaluation Fields listing.
As shown in the above example, move the Importance slider all the way to the left. As you'll see in the Impact setting, the Number of Trades now has a 0% impact on the score. Put simply, it is no longer included in the score.
Next, select Average Annual Return from the Evaluation Fields listing.
As you can see, Average Annual Return is already listed as Very Important, and it's current Impact is 80% of the score. We'll need to adjust the Kurtosis so that it has the same weight as Average Annual Return.
Select Kurtosis in the Evaluation Fields listing.
When Kurtosis is selected as the Evaluation Field, move the Importance slider all the way to the right: Very Important. You should then see that the Impact is displayed as 50%, just like the Average Annual Return.
We've now revised our scoring mechanism so that the OneClick Search looks at both Average Annual Return and Kurtosis. It should be noted, however, that this particular scoring mechanism identifies only which strategies should be optimized. As you may recall, the Optimization phase takes a number of the best scoring strategies up to that point and runs the optimization routines against them. It is this scoring mechanism that identifies which of the strategies are to be optimized.
To identify which strategies are kept as the final best performers, the Multi-System Performance tab is used. More information on Multi-Systems is provided below.
StrataSearch uses the phrase Alternate Data Analysis, but it is the ability to test your strategy against alternate time periods or sectors. The OneClick Search allows you to test up to 10 alternate time periods or sectors, and automatically accept or reject the strategies based on their performance against the alternate data.
In this example, we'll be setting up two alternate data evaluations. In the first, we'll test our same sector against an alternate time period, and in the second we'll test our strategy against an entirely different sector.
To define your alternate data, first navigate to the Alternate Data tab of the OneClick Setups.
Your own settings may be slightly different depending on price and sector data you have available. But in the example above, the first Alternate Data has been configured to evaluate 1/1/2000 to 6/30/2003 for the Nasdaq 100 Index Components. The second Alternate Data has been configured to evaluate 1/2/2004 to 6/9/2008, but with the S&P 500 Index sector.
Now that we've defined our Alternate Data, we can add strategy filters to reject strategies that do not perform well against our Alternate Data.
Navigate back to the Strategy Performance tab and click Add beneath the Evaluation Fields to add a new filter.
Using the Fields button to locate the desired macro, the definition of $F_A1MaxDrawDPctU was selected. The description for this macro is Alt 1 Data - Maximum Unrealized Drawdown Pct. Click OK to save the entry.
As shown above, the Importance slider was moved all the way to the left: Not Important. This means the Alternate Time Period test will not be included in the scoring. However, the Best Rated Value is set to 0 and the Worst Rated Value is set to 20 with the Reject Strategies Outside This Value box being checked.
This Alternate Data test is now configured to evaluate the Nasdaq 100 Index Components between 1/1/2000 and 6/30/2003 and reject any strategies that had a drawdown in equity greater than 20%.
Next, we'll add a strategy filter for the Alternate Sector test. Again click the Add button beneath the Evaluation Fields.
Using the Fields button, the macro $F_A2AvgAnnReturn was selected. The description for this macro is Alt 2 Data - Average Annual Return. Click OK to save the new setting.
Again, the Importance slider was moved all the way to the left: Not Important. The Best Rated Value was set at 100, and the Worst Rated Value was set at 5 with the Reject Strategies Outside This Value box being checked.
This Alternate Data evaluation will now run the strategy from 1/2/2004 to 6/9/2008 against the S&P 500 Index (an entirely different sector from the search itself), and reject any strategies that do not have an Average Annual Return of at least 5%.
Parameter Shift Tests
In addition to Out-of-Sample Tests, another helpful test is to identify if your strategies are still effective when parameters are shifted one or more steps in either direction. For example, if your strategy uses RSI(14), will RSI(13) and RSI(15) still give you profitable results? Many analysts believe that if your strategy still works effectively when parameters are shifted in either direction, then your strategy has a much higher likelihood of being effective moving forward.
The OneClick Search allows you to test shifted parameters automatically, rejecting strategies that do not perform well when using neighboring parameter sets.
In this example, we'll be creating a new strategy filter that is somewhat different than the ones we've created thus far. This one will contain a calculation including two performance values. We'll be rejecting strategies where the annual returns of shifted parameter sets are not at least 75% of the annual returns of the system itself.
To begin, click the Add button beneath the Evaluation Fields.
Using the Fields button to locate the desired macros, the above Definition was created: $F_1SAvgAnnReturn / $F_AvgAnnReturn * 100. In short, this calculation identifies the ratio of the shifted annual return to the underlying annual return, displayed as a percentage. Click OK to save the setting.
Again, since we won't include this field in the scoring, we'll move the Importance slider all the way to the left: Not Important. We'll also set the Best Rated Value to 100 and the Worst Rated Value to 75 while checking the Reject Strategies Outside This Value box.
We've now added a complete Parameter Shift Test, rejecting any strategies where the annual returns of parameter shifted strategies are not at least 75% of the annual return of the underlying strategy itself.
The Parameter Shift Test that we've implemented was based on shifting all parameters 1 step in either direction. StrataSearch allows you to create parameter shift tests for 1, 2, 3, 4, and 5 steps in either direction.
Creating a single strategy is a great start. But what if the most powerful strategies provide so few trading opportunities that your cash is only in the market 10% of the time? Wouldn't it be nice if several strategies could be traded together? You'd get use of the most powerful strategies, keep your money in the market a higher percentage of the time, and get the diversity of spreading your cash across multiple strategies. StrataSearch allows you to combine your strategies together into Multi-Systems. Even further, the OneClick Search can do this automatically, including the strategies that create the best combined returns.
To create a multi-system as your final result, return to the General tab of the OneClick Setup.
In the Number of Strategies entry, enter the number of strategies you'd like to include in your multi-system. The above example has been set to 3.
In the example we've been working on thus far, the AutoSearch is configured to use the same base trading rules each time, with each additional strategy using a variety of supporting entry and exit trading rules. In general, however, a multi-system will be more effective when entirely unique strategies are used. This creates a greater diversity of strategies, and helps identify entirely unique trading opportunities.
Whether multiple strategies are created by the OneClick Search or not, the score of the final results always come from the Multi-System Performance tab. As each new strategy is created and run, its score will be calculated according to the definitions on the Multi-System Performance tab, and if all strategy filters are passed, the score created from this tab identifies which strategies will be included in the final result.
Navigate to the Multi-System Performance tab.
By default, our Multi-System scoring was set up to evaluate the Average Annual Return. As you can see, the settings look very similar to those on the Strategy Performance tab. The one difference is that this tab does not contain the Reject Strategies Outside This Value boxes, since the Multi-System Performance processing contains only scoring but no filtering.
In this step, we'll add one additional Evaluation Field that considers equity drawdowns. First click the Add button beneath the Evaluation Fields.
You can use the Fields button to locate the macro, but the one selected is $F_MaxDrawDPctU. The description for this macro is Max Drawdown Percent - Unrealized. Click OK to save the new Evaluation Field.
As shown in the above example, the Importance slider has been move all the way to the right: Very Important. Also the Best Rated Value has been set to 0 while the Worst Rated Value has been set to 100.
Our Multi-System scoring has now been configured to look at both Average Annual Return and Equity Drawdowns. After passing the strategy filters, the OneClick Search will now select the strategies that have the highest score according to this setting.
Restarting a OneClick Search
After changing the filtering and scoring, we may want to restart our search from scratch. Strategies that previously passed the filters are not be required to pass the new settings. Likewise, the new scoring algorithm may select an entirely different batch of strategies. To prevent our strategies coming from a mixture of the old and new settings, we'll clear our results and start the search over from the beginning.
There are two places where a OneClick Search can be reset. The first can be found on the General tab of the OneClick Setups.
Click the Clear Results button to reset the OneClick Search. The OneClick Search cannot be running in the Processor when this option is selected.
The second place where a OneClick Search can be reset is from the Setups > OneClick Setups menu.
By highlighting the OneClick Setup you wish to clear, and then clicking the Clear Results button, you can reset that particular OneClick Search to scratch.
When the results of your setup have been cleared, you can then restart the OneClick Search just as you first ran it. Open the Run > Run Combinations menu and select the OneClick Setup of your choice.
Many of the these features are for advanced users and are not recommended while you're just beginning to learn how the OneClick Search works. However, you can briefly look these through, and come back to them later when you're ready for additional features.
- Your OneClick Search can be configured for Long Positions, Short Positions, or either Long or Short Positions. This is set on the General tab of the OneClick Setups.
- The Results tab allows you to Keep All Individual Combination Results. In short, this allows your OneClick Search to keep results for all the individual strategies that passed the strategy filters.
- The Results tab allows you to Archive Multi-Systems As Score Progresses. As enhancements are made to your final result, you can archive the prior systems so that you can see how the systems improved over time.
- The Results tab allows you to send an e-mail to the address of your choice whenever an enhancement to the multi-system has been found.
- The OneClick Search contains a feature called Dynamic Trading Strategies that allows you to run a complete Walk-Forward Analysis of any trading strategy or group of trading rules. See Walk-Forward Analysis for more information.
- The Curve-Fit Protection tab provides a quick glimpse of the many curve-fit protection mechanisms that can be used. It's up to the user to identify which of these they'd like to use.
The OneClick Search is a very powerful extension of the AutoSearch, providing a great deal of flexibility and many automated search opportunities. But as mentioned earlier, it is up to the user to direct the OneClick Search in whatever way they wish. A OneClick Search will only find what you tell it to find. You therefore must give it proper instructions to locate what you're looking for.
In the following steps, we've seen how to create strategies manually and through automation. We've also seen how to examine the back test results of our strategies. In the next section, we'll see how StrataSearch can help you trade your system when you're ready.
New Training Video!
Click Here to see a training video of this step!
Next Step: Trading Your System
See Also: OneClick, OneClick Setups, Alternate Data Analysis, Parameter Shift Analysis, OneClick Scoring