BacktestScore

Methodology

Why a methodology page exists

Most software rankings cannot explain their own numbers. Ours are reproducible: this page documents exactly how BacktestScore evaluates backtesting platforms, how our benchmarks are executed, how the scoring rubric works, and when data gets re-verified. If a claim on this site cannot be traced back to a cited source or an executed benchmark, that is a bug. Report it and we will fix it or remove the claim.

What we evaluate

We cover backtesting platforms that are realistically available to retail quants and algo traders: cloud platforms, Python libraries, charting SaaS, and desktop software. Inclusion criteria: the product must offer strategy backtesting as a core feature, be purchasable or installable by an individual (no institutional-only sales processes), and be actively available as of the review date. Inclusion is never paid. No vendor can buy their way into or out of the rankings.

Each platform is evaluated on four axes:

  • Power (35% of overall score). What the engine can express and validate: engine architecture (event-driven, vectorized, replay), asset-class and order-type coverage, maximum data resolution, portfolio-level testing, parameter optimization, walk-forward analysis, and path from backtest to live execution.
  • Data quality (25%). What data ships with the platform and what it costs to get adequate data: history depth, resolution, survivorship-bias handling, corporate actions (splits, dividends, delistings), and the real-world cost and effort of the data pipeline the platform assumes you will build.
  • Cost and value (25%). Total cost of a working setup (license or subscription plus required data, add-ons, and metered extras), measured against delivered capability rather than the headline price.
  • Ease of use (15%). Time to first meaningful backtest, documentation quality, learning curve, interface reliability, and the depth of community answers when you get stuck.

How the benchmarks are run

Sub-scores draw on a benchmark suite that is executed, not estimated:

  • One reference strategy, every engine. The current reference strategy (v1) is a deliberately simple, fully specified rule set: a two-moving-average crossover with next-bar market entries, fixed fractional position sizing, and defined commission and slippage assumptions. We picked it because nearly every engine can express it. It is implemented natively on each platform (Python, C#, Pine Script, AFL, or the platform's no-code builder) from the same written specification.
  • Seeded synthetic data with known ground truth. We generate deterministic synthetic OHLCV series from a published random seed, where every trade the reference strategy should take is known in advance. This lets us check each engine's fills, trade counts, equity curves, and reported statistics against an analytical answer, and expose where default fill assumptions diverge from reality.
  • Real historical data. The same strategy is run on real market data on each platform (using each platform's own data where bundled, and a common documented dataset where bring-your-own) to compare metric agreement across engines. Total return, maximum drawdown, and trade count for identical rules should not vary. Where they do, we report it.
  • Speed and reproducibility. Where the platform permits, we time a single backtest and a fixed-size parameter sweep on documented hardware, and re-run identical inputs to confirm outputs are deterministic.
  • All code is public. Benchmark implementations, data-generation scripts, seeds, and hardware specs live in our open repository. Anyone can re-run them.

Some platforms cannot run parts of the suite: no-code testers with capped history, or manual replay simulators with no scripting API. In those cases we execute the closest supported equivalent, mark unsupported tests as N/A in comparison tables, and say so in the review rather than silently scoring them as zero.

Scoring rubric

Each axis is scored 0-10. The overall score is the weighted sum (Power 35%, Data quality 25%, Cost and value 25%, Ease of use 15%), plus a documented adjustment of at most ±0.4 for material factors the axes do not capture, such as a frozen codebase, vendor stability, or reliability findings from the benchmark suite. Every adjustment is stated in the review's notes; there are no silent overrides. Scores are rounded to one decimal. Scores are finalized before any affiliate or monetization consideration, and no commercial relationship can change a score (see our editorial policy).

Fact verification and freshness

Every price, tier, and limit is checked against the vendor's live pricing page, not cached articles or third-party summaries, and carries a last-verified date. Specs cite vendor documentation. Claims about community sentiment (reliability complaints, support quality) are labeled as such and attributed to their source, whether that is a vendor forum, Reddit, or Trustpilot, with the date observed.

Update policy

  • Prices, tiers, and affiliate terms are re-verified at least every 90 days, and immediately when a vendor announces a change we become aware of.
  • Benchmarks are re-run when a platform ships a major engine release, and at minimum twice per year.
  • Reviews display their last-verified dates; material changes are recorded in a public changelog.
  • Errors reported by readers or vendors are investigated and corrected with a changelog entry. The corrections process is documented in our editorial policy.

Limitations

We measure software, not trading outcomes. The benchmarks tell you whether an engine is fast, honest about fills, and consistent. They cannot tell you whether any strategy will make money. We do not evaluate live execution quality under real market conditions, broker fill quality, or platform uptime beyond documented community reports. Nothing on BacktestScore is investment advice; backtested performance never guarantees future results.