Measuring Predictiveness

Maintaining and improving performance is essential for your business. Any changes made to your investment processes must, therefore, deliver predictable benefits.

So, how does Cabot deliver reliable, predictable results? We can sum it up in two phrases — rigorous analyses and common sense. Rigorous analyses encompass all the mathematics, statistics, financial theory and extensive testing used to establish a behavioral finding. Common sense is what individuals like you, fund managers, have imbued into the Cabot approach – including more than $150 billion of equities examined. An overview of the building blocks used to construct Cabot’s predictive findings is presented below.

• In-Sample Analysis. Cabot Behavioral Analysis (CBA) starts with identifying persistent tendencies within a portfolio performance history. Pinpointed shifts are then tested to uncover unproductive behavioral tendencies. Shift results are examined for impact on performance and statistical significance. Behavioral shifts that deliver positive improvements to both return and alpha, and reflect low P-Values, are then subjected to Out-of-Sample testing.
• Out-of-Sample Testing. Shifts from one time period, that provide compelling benefits and strong statistical significance, are tested in subsequent months to learn how they perform prredictively. A sequence of Out-of-Sample tests is performed relative to a multitude of In-Sample analyses, for each shift, with each test including successive sets of months until the entire performance history is fully utilized.
• Calendar-Time Portfolio Returns. The results from all Out-of-Sample tests for each shift are combined into a single time series. The result is a continuous series of monthly returns, beginning at the start of the first Out-of-Sample test and extending through the end of the history, reflecting the Out-of-Sample benefits of a shift across all the data analyzed.
• Predictiveness. Benefit sustainability from a shift is measured as the P-Value of the alpha for the corresponding Calendar-Day Return series. Low P-Values indicate that the benefits are reliable, evenly distributed over the time series and simply believable. Shifts that are placed into implementation are those with the highest Out-of-Sample P-Values – since these represent potential benefits with the strongest level of predictability. Naturally, the higher the benefit the better.

Forthcoming, Harold Haig (our President) and Terrance Odean (our Academic Advisor) are writing a paper that describes in detail the Cabot approach to measuring predictability. Research conducted in support of this paper indicates that behaviors related to selling – and consequently the benefits from sell shifts – reflect strong predictability across a large group of professionally managed portfolios.