Memory management and speeding up the portfolio backtester
Posted: May 16 2018
I'm a bit confused how the memory management in MC work. After some digging (mainly testing different scenarios), it seems to me that MC first loads the necessary data into the memory, and then for each strategy instance (logical core) works through this data and stores the calculations (and data?) to the RAM again. After seeing this, I would expect the total RAM requirement to look something like “X + N*Y” where X is the initially loaded tick data, and Y>X is the RAM used for each of N instances, but this doesn’t seem to be the case as the requirement doesn't scale linearly with the number of cores. Often Y is larger than X though, even when I run an empty strategy that doesn't generate any orders and doesn't keep any data, which seems a bit "wasteful" as it shouldn't have to hold that much data for the instance. How come so much memory is needed for each instance, even if no trades are generated?
Are there any Multicharts specific actions the users can take, in order to speed up optimiziation and minimize RAM requirements? I'm not referring to tweaks outside of MC, but rather changes to the code/settings or any tricks to turn off further functionality in MC that might not be required in the optimization.
A few ideas are:
- Minimize the number of bars the strategy can reference.
- Set the 'NumberOfThreadsOnOptimization' registry key to equal the number of real cores instead of logical cores (will decrease the memory requirement without adding to computational time, assuming CPU utilization is 100%).
I was hoping to learn about some attributes (or similar) the strategy can use to avoid unnecessary computations, but that is probably wishful thinking.
Are there any Multicharts specific actions the users can take, in order to speed up optimiziation and minimize RAM requirements? I'm not referring to tweaks outside of MC, but rather changes to the code/settings or any tricks to turn off further functionality in MC that might not be required in the optimization.
A few ideas are:
- Minimize the number of bars the strategy can reference.
- Set the 'NumberOfThreadsOnOptimization' registry key to equal the number of real cores instead of logical cores (will decrease the memory requirement without adding to computational time, assuming CPU utilization is 100%).
I was hoping to learn about some attributes (or similar) the strategy can use to avoid unnecessary computations, but that is probably wishful thinking.