News: An honest review

A bit over two years have gone by since we launched the TuringTrader project in December 2018. Since then, we have developed and implemented over 50 strategies on the platform and have researched at least another 50 concepts and ideas with varying complexity. With this experience under our belt, it is time to take a critical look at our project. Was it worth it? What works well? What doesn’t?

Was it worth it?

This question aims at the project as a whole. Did we really need to implement yet another backtesting engine? Or was this just a somewhat arrogant, self-centric pet project? It is essential to answer this question, as it also sets the stage for the future. Where should the project go? Should we invest more into it?

From our perspective, we can quickly answer the question with a resounding yes. The project not only brought us personal joy, but it achieved a broad range of objectives. Thanks to the project, we have connected with like-minded quants around the world. And the TuringTrader engine has made our life easier. Many of the portfolios we have developed in the past two years would have been much harder to research, implement, and test with any other platform. So this open-source project has made sense for us from a business perspective, powering our two boutique ventures.

As a consequence, we are more committed to the project than ever. We will maintain our own backtesting engine, and we will continue to share it with the world.

What works well?

There are many aspects that we love about TuringTrader. Let’s have a closer look.

Programming in C#. We certainly like C#. It’s a modern object-oriented language with an excellent standard library. We love LINQ, which is incredibly useful for developing portfolio strategies. The language is strongly typed, which has helped us avoid bugs and confidently release code to production. The language runs fast, even though further down, we will revisit speed. What’s most limiting is that not everybody shares our love for C#. Many quants develop in Python, and we sometimes wish there was an easier way to reuse their code. On the desktop, we remain committed to C#. Server-side or in the cloud, we aren’t so sure anymore.

Automagic Indicators. We absolutely love this feature. Coming from a signal processing background, it is a mystery to us why other simulators make it so hard to wire up and instantiate indicators. We are using more indicators than ever, and we are more open to experimenting around with them. And this ability to experiment is something that is tightly linked to productivity. We will certainly keep this feature.

Custom Reports. We use this feature all the time. Whatever we are working on, we need to visualize our strategies’ inner states to understand their decisions better. User-defined charts are an invaluable feature, and we certainly want to keep them. The custom templates are nice to have but may be less essential. By now, our native reports are pretty complete, and we feel that there is little missing that other quants might want. But this template-based approach enables another feature: the ability to feed simulator reports directly into Excel. This is a feature we can’t do without and that we use daily to run further analysis. In summary, we will undoubtedly keep this structure.

Splicing Data. Most of our strategies are trading ETFs. And with that, we are constantly running into needing to extend the history using proxy instruments. TuringTrader’s data-splicing feature makes this a trivial task. By now, we are wondering how anybody could seriously code without this feature.

Algorithms as Data Sources. Since we started developing on TuringTrader, we have been able to create increasingly complicated systems and meta-portfolios. And at the same time, we have been able to simplify our code through better modularization. The key to this modularization is to use algorithms as data sources. And once you start coding this way, it is addictive. There is no way we will ever drop that feature.

Universes. Thanks to TuringTrader’s universes, we can experiment with various universes, avoid the need to maintain them manually, or worry about survivorship bias. This feature has helped us develop better stock-market strategies, and it is clearly a keeper.

What works not so well?

But where there is light, there is also shadow. Here is a summary of things we don’t like.

Share-based Paradigm. The TuringTrader engine follows the same paradigm as any other simulator we know: Running simulations based on the concept of trading integer numbers of shares. Instead, we should have chosen a paradigm of calculating an index based on rebalancing operations that set the asset’s allocation to a percentage of the portfolio’s net asset value. If need be, share numbers can easily be derived from that. This choice caused many headaches, including issues with implementing meta-strategies or strategies trading inverse and leveraged assets. It can be mostly fixed by implementing a feature for fractional shares, but ultimately the simulator should not enforce calculating share numbers.

Running Everything in Sync. The TuringTrader core aims to run all data-sources and indicators in sync to implement a real-time streaming architecture. While this is beautiful, it also creates a lot of complexity and does not align well with how we are using the simulator. Instead, we should have adopted a simpler architecture, optimized to calculate time-series in their entirety and not one bar at a time. While TuringTrader’s simulation speed is reasonable compared to others, we believe we missed opportunities to multi-task and further increase simulator speed. Also, block-based processing would allow algorithms to access an unlimited number of historical bars, which is a feature we missed more than once.

Lack of Generalization. On our journey of developing TuringTrader, we learned that ultimately indicators, data-sources, and algorithms are quite similar in that they provide time-series data to the simulator. And while this is a step up from every simulator we are aware of, it is also an area where we fell short. Instead of following different paradigms and using incompatible APIs, indicators, data-sources, and algorithms should all be derived from the same base. This lack of commonality currently makes running strategies of strategies harder than necessary, and it is one of our biggest regrets with the project.

Conclusion

TuringTrader is already an opinionated backtesting engine. It does many things differently than other simulators, making it one of a kind. The things it does well support that claim. But when looking at the things it doesn’t do that well, it becomes clear that we haven’t gone far enough. We should have focused even more on doing things right, even if that results in taking different approaches than everybody else.

So ultimately, this is what we want from a backtesting engine:

• nice and clean APIs, making instantiation of indicators a breeze
• designed for implementing portfolio strategies
• streamlined methods for calculating strategies of strategies
• optimized for multi-threading and fast execution
• focus on calculating performance indices rather than account values

Could we have known this earlier? Maybe. But the reality is that some of the findings above required hands-on experience from implementing dozens of strategies for ourselves and clients. We haven’t decided yet, where we will go from here. We have taken a considerable investment in porting all of our algorithms to TuringTrader, so TuringTrader is alive and kicking, and abandoning the platform is certainly not on the table. However, we might consider a major overhaul of the engine. Whatever we do, we will aim to maintain source-code compatibility to allow for a smooth upgrade path.

Stay tuned and happy coding!