Fund sustainability ratings are painting by numbers

Fund sustainability ratings have been popping up everywhere over the last few months. Amid much fanfare, Morningstar launched its scoring using Sustainalytics rating of portfolio companies, MSCI was close behind with fund analysis based on its own data, and Lipper has announced a tie up with Princeton University. The idea that a fund’s sustainability credentials can be distilled to a single number is alluring, but the reality is sadly more complicated.
Environmental, social and governance concerns have been moving up investors’ agendas for years, most obviously reflected in the growth of the UN-backed Principles for Responsible Investment (PRI), which now encompasses nine in every ten of the world’s largest active fund managers. That investors should be more interested in comparing fund “sustainability” is unsurprising.
So, on the one hand, recent moves to quantify a fund’s “sustainability” is a logical step and a welcome reflection of the growing interest we are seeing from our clients. On the other, while most Schroders’ funds score well on Morningstar and MSCI ESG ratings, we feel that a single letter or number should not be the final answer.
Other fund features can be distilled into a single number; volatility, turnover, concentration and performance all require interpretation but all are objective and valuable measures. Why not sustainability?
Most critically, there is no consistent definition of what sustainability looks like, let alone how to measure it. While they use similar terminology and present their results in similar ways, ESG ratings are fundamentally different to other ratings, like those given to bonds.
Bond ratings are designed to measure the risk that companies default, and have been reasonably accuratein doing so most of the time. Without a consistent definition of what ESG ratings are supposed to measure or even a consistent view of what constitutes a relevant ESG issue, it is impossible to gauge their effectiveness. Even the UN PRI, which commits investors to incorporating ESG factors into investment analysis, decisions and ownership, has yet to provide a definition. Despite similar sounding approaches, the main ESG rating organisations can reach very different conclusions for the same companies. On average, across over 1,600 stocks in the MSCI World benchmark, only 26% of the scores assigned by one of the two largest rating agencies can be explained by the scores the other agency assigns to the same companies (the “R-squared” value). For example, online marketplace eBay is a strong top quartile performer according to Sustainalytics, but a bottom quartile laggard in MSCI’s assessment. Both sources cannot possibly represent the definitive view of companies’ ESG strengths. Neither is right or wrong; both represent a perspective and may well be considering different factors, for different purposes. Recognising that they have different approaches and generate different conclusions is critical before leaping onto ratings as evidence of a company or fund’s sustainability qualities.
To demonstrate the challenge this poses to fund analysis, we randomly divided the companies in the MSCI World benchmark into 55 hypothetical portfolios of 30 stocks each. We looked at the average ratings of stocks in each portfolio using five common ESG ratings (labelled A-E below), as well as a Super Rating we created, and then ranked those portfolios according to their average ESG strength using each type of rating. Those rankings represent a rough approximation
of the fund ESG ratings you will see proliferate. The fact there is little correlation between rankings based on different sources of data should raise a red flag; rankings based on one source of ESG information differ from those using other sources by about 25% on average. Every ESG fund rating has to be seen as one perspective of many, and of limited use without knowing exactly how it is calculated.
Focusing on the two main fund ratings already published – by MSCI and Morningstar – shows similar discrepancies. One in five of the funds that appears to be a leader using one of the main ratings is below average through the lens of the other.
The Super Rating looks like it might provide at least the most consistent view; on average, the fund ratings calculated using the Super Rating approach were more closely correlated to other key ratings than any other rating approach. The Super Rating appears to include criteria and conclusions that are common to most of the other ratings.
How similar are fund ESG ratings using different scoring systems?* Tellingly, the Super Rating is nothing more complicated than a formula based on companies’ size and region, and nothing more fundamental. The values are obviously meaningless for ESG analysis but nonetheless manage a reasonably good approximation of widely used ratings. This highlights a fundamental failing of most ESG rating methodologies; they are heavily influenced by the size andresources companies are able to spend on disclosure and the local regulations and pressures they face to report specific information. This is why we place so much value on our analysts forming their own view on ESG issues to incorporate into investment decisions. The exercise highlights the dangers fund ratings pose to investors persuaded to focus on one fund rating methodology or another. Our concerns are not with ratings in themselves. ESG rating agencies provide valuable analysis and insights to investors – across Schroders we use data from several well known sources – but they should be inputs to company evaluation rather than an answer in themselves. At Schroders we have worked hard for close to 20 years to develop tools, frameworks and data to help us better understand and integrate ESG issues into the funds we manage for our clients. We understand that the insights this yields cannot be distilled into a single value, for either a company or fund, but must be considered in the context of the investment case for each equity or fixed income instrument that we invest in. We try hard to be transparent in our approach to examining ESG trends, assessing company performances and integrating conclusions into our investment strategies. We also recognise growing interest in evaluating how well we do these things. However, while fund ratings are compelling in their simplicity, they risk misleading investors into conclusions that promise more than they can deliver.

Andy Howard is Head of Sustainable Research, ESG, at Schroders.

Related content: MSCI’s Linda-Eling Lee on ESG Ratings: Four Myths and a Truth

• Note: Based on ratings by MSCI, Sustainalytics, RepRisk, EIRIS and Asset4 through data aggregator services. Ratings are converted to numerical values and normalised on a 1-100 scale. Data extracted in March 2016.