Stress tests for Python trading strategies

Stress-test your strategy before you trust it.

Upload your Python strategy. Quantgrounds runs it through twelve standard tests on historical market data and shows where the backtest is likely to mislead you, with the evidence, the parameter or line of code involved, and a change to try.

  • First scan free, no signup
  • Your code runs in a sandbox and is not stored
  • PDF reports you can keep
Example report
overfit_meanrev.py
SPY · 1993–2019
tuned on 2000–2012
The edge largely disappears outside the period the strategy was developed on.
59
/ 100
Mixed results
Example score
diagnostic, your setup
Stress tests

An example from a sample strategy, rendered from its results file. Open the full PDF

Your own Python code

A PARAMS dict and an on_bar function. No drag-and-drop builder, no new platform or language to learn.

Evidence, not just metrics

Every finding states the numbers behind it, the parameter or line of code involved, and one specific change to try.

The same tests, every time

Every strategy gets the same twelve tests under published rules, so reports can be compared and every result traced.

Case file · look-ahead bias

A Sharpe ratio of 2.13 that could never have been traded.

center=True makes every moving-average value average the days after it too. The code runs without errors, and the backtest was quietly reading the future. Here is what the report did:

  1. Flagged line 15 in the static code review, before anything ran.
  2. Proved it. The indicator was recomputed with the future deleted at four cut dates, and past values changed.
  3. Measured the damage by re-running the strategy point-in-time, from past data only.
  4. Capped the score at 20 and marked every performance-based test "Not valid".
Sharpe as backtested
2.13
Sharpe point-in-time
0.23
13def prepare(prices, params):
14    # min_periods=1 avoids NaNs at the start
15    smooth = prices.rolling(params["smooth_days"],
                            center=True, min_periods=1).mean()
16    return {"slope": smooth.diff()}
Growth of $1, log scale As backtestedPoint-in-timeBuy & hold SPY
The stress tests

Twelve tests a skeptical quant would run. Run the same way on every strategy.

Out-of-sample periods

Splits one continuous backtest into the development period you declare and the data before and after it.

Catches: performance that exists only where you tuned.

Parameter sensitivity

Moves each numeric parameter in PARAMS by ±10% and ±20%, one at a time.

Catches: results that depend on one precise value.

Transaction costs

Re-costs the same trades at 0x to 5x your cost and solves for the break-even cost per trade.

Catches: an edge too thin to survive costs.

Execution delay

Fills the same decisions one trading day later than in the baseline.

Catches: an edge that needs instant execution.

Unseen assets

Runs the same code and parameters on up to 10 assets you did not choose.

Catches: a rule that only fits one asset.

Market regimes

Splits returns into bull, sideways, high-volatility and bear markets from trailing SPY rules.

Catches: dependence on one kind of market.

Look-ahead bias

Recomputes indicators from prepare() with future data deleted; if past values change, re-runs point-in-time.

Catches: code that uses prices not yet known.

Resampled histories

Resamples daily returns 1,000 times in blocks to put a range around Sharpe, CAGR and drawdown.

Catches: a Sharpe not clearly above zero.

Random timing

Rotates the strategy's own positions against the market 1,000 times: same exposure and costs, different timing.

Catches: returns that come from being invested, not from when.

Market exposure

Compares the strategy with buy & hold of the same assets and measures its beta and alpha against SPY.

Catches: a strategy that is just the market, with extra steps.

Walk-forward

Each year, re-picks the best nearby parameter set from the previous five years only, then trades it for the next year.

Catches: an edge that needs hindsight to set up.

Research selection

In "Your setup", tell it how many variants you tried; the deflated Sharpe ratio adjusts for keeping the best one.

Catches: the luck of picking the best of many backtests.

Every report also includes a static code review, a determinism check, the largest drawdowns and how concentrated the profits are. Read the exact rules and thresholds

How it works

Three steps. Usually under a minute.

  1. 1

    Write the strategy in plain Python

    Put the tunable numbers in PARAMS and return target weights from on_bar. It only ever receives data up to the current bar.

  2. 2

    Choose how to test it

    By default it runs on 2,000 random slices of history, so no setup you pick can flatter it. Or choose "Your setup": up to five assets and the dates you developed on, which the out-of-sample test compares with the rest.

  3. 3

    Read the report

    A verdict, the Quantgrounds score with its spread, the status of every test and ranked findings with changes to try. Each PDF carries a fingerprint of your code that anyone can verify.

Open the app
import numpy as np

PARAMS = {"sma_days": 200}   # moved ±10% and ±20%

def on_bar(ctx, data):
    n = ctx.params["sma_days"]
    weights = {}
    for asset in data.assets:
        closes = data.history(asset, n)   # up to today
        if len(closes) < n:
            return None                   # warm-up
        above = closes[-1] > np.mean(closes)
        weights[asset] = 1.0 if above else 0.0
    return weights                     # filled next close
Example reports

Three example strategies. Three very different verdicts.

These are illustrations, not recommendations. Each was run in "Your setup" mode on SPY with a 2000–2012 development period, so you can see what each test catches. Every run gives you a two-page summary PDF, a full report of about nine pages, and the raw results as JSON.

Trend filter
trend_sma.py
Example

A 200-day moving-average filter. Few parameters, round numbers, low turnover.

No major weaknesses found in these tests; 1 test with warnings.

    Open this example
    Overfit mean reversion
    overfit_meanrev.py
    Example

    Parameters chosen by a 700-trial search over 2000–2012. It looked great on the data it was tuned on.

    The edge largely disappears outside the period the strategy was developed on.

      Open this example
      Look-ahead bug
      lookahead_bug.py
      Example

      A smoothed trend rule with a backtest Sharpe of 2.1. One keyword lets it read future prices.

      Not tradeable as written: the backtest uses information from the future.

        Open this example
        Honest by construction

        Quants can smell a made-up number. So we don't make any.

        No invented statistics

        Every number, finding and verdict comes from fixed rules in code and is saved in the run's results file. No "99% confidence" theater.

        Ranges, not false precision

        Uncertainty comes from a block bootstrap. Overlapping days are never counted as independent samples.

        A diagnostic, not a certificate

        No badges or leaderboards. Passing every test does not mean a strategy will make money, and the report says so.

        Your code stays yours

        Strategies run in a sandbox with network access blocked. Code is never written to disk and is discarded when the run ends.

        What it covers

        • Python strategies using numpy, pandas and standard math modules
        • Daily bars: decisions at the close, filled at the next close
        • SPY and 20 US stocks, adjusted daily closes from 1989 to 2019
        • A flat cost per side (5 basis points by default), long and short weights

        What it doesn't

        • Intraday data, order books, options and futures
        • Market impact, borrow fees and taxes
        • Prices after 2019; the ticker list has survivorship bias
        • Any guarantee: static code checks are heuristics, and results do not predict returns
        FAQ

        Questions

        What does the score mean?

        By default you get the Quantgrounds score, measured on 2,000 random shards of history. If you choose "Your setup", you get a diagnostic score for the assets, dates and costs you picked; the two can differ, because one setup can suit a strategy better than the average. Both are a weighted average of six sub-scores: edge over buy & hold of the same assets (25%), risk-adjusted return, parameter robustness, cost resilience, generalization to other assets and drawdown at equal risk (15% each). The score can't exceed 50 plus 100 times the Sharpe edge over buy & hold, so a strategy that doesn't beat holding tops out at 50, and buy & hold itself scores about 50. 90 or more is Exceptional and rare, 80–89 Strong, 65–79 Promising, 45–64 Mixed results, 25–44 Weak, below 25 Serious issues. If look-ahead bias is confirmed, the score is capped at 20. It summarizes how the strategy did in these tests. It is not a measure of skill or a forecast of future returns. The methodology has every formula.

        Why does my strategy score below 50?

        Because it doesn't beat simply holding the same assets over the same days, on a risk-adjusted basis. Holding is free, so the score only rewards what a strategy adds on top of it: without an edge over buy & hold the score tops out at 50, and buy & hold itself scores about 50. Most published rules (trend filters, momentum, moving-average crossovers) land between 30 and 50 on this data set. That doesn't make them useless: the report still shows whether yours cuts drawdowns at equal risk, survives costs and holds up on other assets. 80 and above means a clear, robust edge, and 90 and above is rare.

        What are random shards?

        The default way to score. Instead of assets and dates you pick, the strategy is scored on 2,000 random slices of history: one asset each, 2 to 8 years long, across 11 assets. The Quantgrounds score is the average, so it can't be flattered by a favorable setup. The slices are random, so a rerun gives a slightly different number, usually within about 1.5 points; the report shows the exact range and a seed that reproduces it. The methodology has the details.

        How do I know a report is genuine?

        Every PDF prints a fingerprint of the strategy code (a SHA-256 hash) and a replay code: the settings that produced the score, such as the random-shard seed, plus the score itself. Runs are deterministic, so on the verify page anyone can paste the code, check the fingerprint, and re-run it to get the same score. An edited score or a different strategy won't reproduce. It can't prove who wrote the code, only that the report belongs to it.

        How is this different from a backtesting platform?

        It doesn't replace your backtester. A backtester answers "what would this strategy have returned?" Quantgrounds takes the strategy and asks how much of that result survives new periods, nearby parameter values, higher costs, delayed fills, other assets and a check for look-ahead bias.

        Is my code safe? Do you keep it?

        Your code is checked against an import and syntax allowlist first. Code that fails the check is never run. Code that passes runs in a separate process with CPU, memory and time limits, network access blocked and file writes blocked. It is not written to disk and is discarded when the run ends. Results stay in the server's memory for up to an hour so you can download the PDFs, and are not saved to a database.

        How long does a run take?

        Usually under a minute. Each run executes your strategy many times (parameter changes, held-out assets, the look-ahead test and a determinism check), so longer histories and more assets take longer. If the site has been idle, the first page load can take a few extra seconds while the server starts. A run is stopped after four minutes.

        What data does it use?

        Adjusted daily closing prices for SPY (1993 to 2019) and 20 US stocks (1989 to 2018, depending on the listing date). The dataset is fixed, so a deterministic strategy with the same settings gets the same numbers every time. Each report states the period it used.

        What can my strategy import?

        numpy, pandas, math, statistics, collections, itertools, functools, datetime, decimal, fractions, bisect, heapq, operator, typing, dataclasses and enum. There is no file, network or operating-system access.

        How are trades filled and costed?

        A decision made at one day's close is filled at the next day's close, so a strategy never earns the return of the bar it decided on. The cost you set (5 basis points per side by default) is charged on every change in position. The engine does all the accounting; your code only returns target weights.

        Does AI write the report?

        No. Every number, finding and verdict comes from fixed rules in code. The summary paragraph is filled in from templates.

        Is it free?

        Yes. You can run one scan without an account to see a full, watermarked preview. A free account removes the watermark, saves every report to your dashboard, unlocks the raw data, and gives you unlimited scans (up to 15 runs per hour). If you sign up after your preview, that report moves into your account automatically.

        Is this investment advice?

        No. Quantgrounds is an educational tool. Results are simulated on historical data and do not predict future returns. Nothing here is a recommendation to buy or sell anything.

        Find out what your backtest isn't telling you.

        Run a strategy