The Chris Paul Problem isn't that big a problem for analytics
How far do you have to go down the Chris Paul Problem rabbit hole before you disappear into an empty void of obviousness?
The too long; didn't read version of the Chris Paul Problem is, to paraphrase Billy Beane: Chris Paul's shit doesn't work in the playoffs.
Chris Paul, you see, is one of basketball's greatest point guards. He has consistently posted amazing all-around numbers, and is the active leader in steals and assists. He has consistently led great regular season teams, and consistently reached the playoffs, which includes carrying some truly chump Hornets squads into the postseason early in his career.
Chris Paul is, by any reasonable accounting, an inner circle Hall of Famer. But!
But his teams ultimately haven't won diddly.
This is the fundamental underpinning of the Chris Paul Problem, and it's a fair enough one to examine. Burneko highlights some extreme statistical oddities relating to Paul and advanced analytics - namely that Paul rates not just extremely well, but as perhaps one of the two or three best players ever: "According to Basketball Reference, Paul has the sixth-highest career Player Efficiency Rating (PER) of any player ever; Win Shares Per 48 Minutes (WS/48) essentially cannot tell the difference between him (.2504) and Michael Jordan (.2505); Box Plus-Minus (BPM) says only Jordan and LeBron James are his superiors. His career 122.69 Offensive Rating (OffRtg) is the highest of all time. The judgment of the sport’s holistic metrics is that he is one of the best basketball players who ever lived."
Burneko then spends the next 1,700-some words arguing that this shows how basketball analytics are flawed. Which, sorry to be glib, but... duh?
I don't have the mathematical nous to fully dissect the Chris Paul Problem, but my intuition is this: Paul is a fantastic, world-class defensive point guard. Combined with his incredibly efficient offensive game, this skill juices the value of his metrics to historical highs.
But! It's not actually all that valuable to be super great defensively at the point guard position. Outside a few notable exceptions, point guards are very rarely their team's most dangerous scorer, ie the person most important to guard. Moreover, as Tjarks notes, Paul is just too short to be able to switch onto larger two-guards. So between stuffing out weaker point guards and never even contemplating guarding larger twos, Paul is very rarely directly scored upon.
Then you factor in how he grabs lots of steals (among the most ever!) and that his teams generally play good defense when he's on the floor, and his defensive metrics soar.
The problem is his paper defensive value far outstrips his actual basketball defensive value, because basketball analytics, as far as I'm aware right now, don't have any mechanism for adjusting for positional - really I should say role - realities.
Kyle Lowry is another good defensive point guard who makes you scratch your head a bit as he pushes the top-10 in catch-all ratings like ESPN's real plus-minus. He outranked Kevin Durant by that metric in 2016-17! Kyle Lowry is very good but Kyle Lowry is not by even the most optimistic hallucination better than Kevin Durant.
DeMarcus Cousins is a kind of inverse. He rates out as a world-beater by most metrics even though his teams routinely perform as if they don't employ a world-beater. This is because Cousins is a very very good offensive center but does not protect the rim, which is actually maybe a center's most important job in the modern game. His paper value outstrips his actual basketball value.
And that can even change! This stuff is nebulous. If you had Cousins instead guard the elbows/wings (and if he could do it well) and gave him cover with a real rim protector inside, he might suddenly be a far more valuable basketball player, though it wouldn't necessarily show up at all in the metrics.
Kevin Love is a good example of this. He arrived in Cleveland a terrible inside defender, the Cavs ultimately gave that role to Tristan Thompson (and LeBron, who guards everywhere), had Love guard the wings which he actually does pretty decently, and he was suddenly much more valuable to the team. Love's win shares/48 have hardly budged in Cleveland (.165, .169, .163) but he's been a better contributor to their success since his really underwhelming first season.
And you can tease that out in other analytics. Going back to real plus-minus, Love didn't rate as even a top-50 player in 2014-15. He found his defensive niche during the Cavs' excellent championship-winning season a year later and he's gone back to being a top-15 player by the metric the last two seasons. (Love isn't really a top-15 player, once you consider usage, by the counting-stat version of RPM he's only 25th, which when you consider his role and how he fills it is probably not terribly far off.)
So, yes, Burneko's larger point, that Chris Paul's problem is emblematic of analytics' problems, is undoubtedly true. Much much much more than baseball's advanced metrics, basketball's must be contextualized. This has always been the case, is I feel like universally acknowledged, and to be honest I'm not sure who Burneko is actually arguing to here.
If you understand that Paul's great defensive value isn't actually as valuable as it might seem, we can go back to, for example, PER, a stat that doesn't really account for defense. If you're looking almost solely at offense, which is not totally but about close to appropriate with a point guard, Paul's career PER is "just" 25.7 and his career high is "only" 30.0. Those are still very good, Hall-worthy numbers (the career number is fifth-best all-time). But his best season doesn't match Steph Curry's (31.5) and his career number is dwarfed by Jordan's 27.9 (it's a big difference I promise). Look at it that way, and Paul properly comes back to earth a bit.
(Now, is he still the fifth-best purely offensive player of all-time? No, I most certainly wouldn't argue that. I said PER doesn't account for defense, but that isn't quite the case. It does include steals and blocks, and I think a lot of the Chris Paul Problem can also go back to the fact that he's a massive steals outlier and steals aren't perhaps really all that important.)
Basketball's overall-value analytics could certainly use some tweaking, that's true. What's also true is they're already very good for pointing you in the right direction.
Basketball is not like baseball, you can't just stack WARs and call it a day. You have to tease individual value out of a messy context, from who you share the floor with to what role you play.
This is always going to be the case, regardless of how analytics improve from here.