Building trading algorithms from scratch. Part II.

Today, we continue our algorithmic journey. Last time we stopped at copying algorithm logic from TradingView into NinjaTrader and faced some difficulties with inner methods of two different platforms. This time we start with a ready-to-use NinjaTrader script. Since the code is getting messier, I'll just share the links to the files and not quote it inside the article. So here is our starting point, and below you can see the result of the backtest on the same time period as we used in TradingView (January 2017 - May 2020).

Do you like the result? I don't. The equity curve's shape resembles the one in TradinView, but the magnitude of negative trades is much bigger. What are the possible reasons? Besides, the inner methods, that often affect backtesting, this time, I used gold futures instead of XAUUSD, which is a FOREX instrument.  FOREX instruments have floating spread. Floating spread means that sometimes it may become wider. This floating spread often distorts data and backtest. In other words, backtesting ignores a potentially significant source of the additional cost. I had many algorithms that proved to be profitable in backtesting but were destroyed by floating spreads. 

Difficulties with spread are not the only issue at hand. FOREX market doesn't have one verified source of data because it's not an exchange-based market. Each Foreign Exchange broker has its' own liquidity pool, so the quotes from different providers can vary. 

All of the problems can be solved by using Gold futures data. Gold futures are an exchange-based instrument, so it has a single source of quotes (which, in my case, is CME). Besides, because of high liquidity, the spread is almost always at its minimum. So, whenever possible, use futures data - it is more reliable (There is still a case when different brokers may have slightly different quotes, and we talk about it in one of our courses, which are free by the way.)

Ok, a little bit of marketing and forward we go. Let's answer the question - do we want to continue with such a strategy? The profit factor is 1.25, and all of it comes from one large trade. Relying on strategy with one or two huge outliers is always a bad idea. Always. However, I will move forward with the strategy because I want to show my whole process, and if we stop after every bad idea, we can be at step one for a month or so. 

What are the next steps, that I want to do? First, I run through the trades and try to find out if there are any inconsistencies with the initial logic that I wanted to implement. This time I was quite lucky. Let's take a look at the chart below. 

I’m referring to the second long trade. It opened instantly after the price crossed the moving average. This behavior is consistent with the logic we programmed, but not with the logic I wanted to capture. In particular, I want the strategy to react only to the reverses that happen only after price crosses the moving average. Otherwise, you're trying to play with the reverse twice. This is greedy and, I believe, not optimal. If my explanation isn’t clear, think about it this way. Once we get the reverse below the moving average, we enter long. Once the price goes above the moving average, that means we already seized the reverse. So when the price goes below the moving average once again, we shouldn't consider anything that happened before the cross over. In other words, the second long position that you see on the chart shouldn’t be there, because the price just crossed below the moving average.

What can we do with this? Well, let’s add one more condition to opening long and short positions. In particular, if I want to long, then the price shouldn’t have crossed above the moving average in the last 10 bars. In other words, I don’t want any crossings in between the low we analyze in the strategy and the entry.

The new code can be found here. Before we check our new result, it's important to note the data division into out and in samples. Since I test my initial ideas on TradingView, and TradingView has limited data for backtesting I don't do any data division. I didn't divide data when I copied the logic into NinjaTrader as well because I wanted to be sure, that my TradingView and NinjaTrader logics are the same and equity curves look alike. Now, however, when we shifted to NinjaTrader completely, it's time to think about in and out samples. I usually test algorithms on 10 years of data if I use fast timeframes, such as hourly and 4-hourly. 10 years of data I divide into 2 chunks: in-sample is the first 8 years of data I use to work with the algorithm and out-of-sample is the last 2 years I use for final testing. 

Ok, now let's check two backtest - our initial strategy and the modified one. Below you see the results of the two backtests. 

After a backtest, we can say, that it didn’t improve it at all. It only made it worse... Here is why - the overall result stayed the same, but we added more complexity to the algorithm. Complexity decreases your chances to profit in real trading and increases possibilities of ending up with a curve fitted algorithm. So, this step is certainly a bad one. And it's fine. We are testing hypotheses here and finding something that doesn't work is also a good result. So our modification doesn't work, but I still don't want to have the logic we described above. How about we make a reverse modification? If I don't want trades to be triggered by crossovers with the moving average, how about deleting the rule about the average at all? Let's leave the only one condition to enter - long if the close is above (below) the high (low) of the lowest (highest) bar among the last 10 bars. The code for this version is here.

Let's take a look at the result. 

Getting better, isn't it? It was a pretty good step - we decreased complexity and increased almost all statistics of the algorithm. I believe we are ready to move forward. In the next post I will show you how to search for flaws and how to add more trading dimensions into your analysis.