Systemic risk measures such as CoVaR, CoES and MES are widely-used in finance, macroeconomics and by regulatory bodies. Despite their importance, we show that they fail to be elicitable and identifiable. This renders forecast comparison and validation, commonly summarised as ‘backtesting', impossible. The novel notion of multi-objective elicitability solves this problem. Specifically, we propose Diebold–Mariano type tests utilising two-dimensional scores equipped with the lexicographic order. We illustrate the test decisions by an easy-to-apply traffic-light approach. We apply our traffic-light approach to DAX 30 and S&P 500 returns, and infer some recommendations for regulators.