About a month ago Pharo 3 was released. To show how much Pharo have changed from the previous version, the amazing code city visualisation was created. We cannot doubt that Pharo changed a lot, but were this changes good or bad?
“@einarwh: The code city of Pharo is on fire! #ndcoslo pic.twitter.com/dRpvqV2IHi” Even more fire coming soon. But a different one…
— Yuriy Tymchuk (@yuriy_tymchuk) June 4, 2014
In fact we can visualise quality changes too. We can use Code Critics (Small Lint rules) to generate critics about Pharo code, and then display this data with CodeCity visualisation. I've followed a simple strategy: if an entity never had critics both in Pharo 2 and Pharo 3, it remained grey. If number of critics didn't change for an entity, it was coloured in magenta. Now, if amount of critics reduced in Pharo 3, magenta was shifted towards blue. On the other hand, if number of critics increased, nodes were becoming coloured with red shade.
You can spot really interesting parts.
For example NautilusRefactoring class was improved a lot. Also you can spot the same blue buildings for a couple of other classes.
On the other hand if you look at the package called SmartSuggestions you can see that there are a lot of red classes.
And of course we had some almost completely red buildings.
Now for important stuff. You can see that there is a A LOT of red colour. Does this mean that Pharo 3 became worse in terms of code quality? Not necessarily. We don't know if Code Critics are detecting critics well. For example some packages with small red classes are test packages. Should we skip tests for code quality analysis? No. Should we treat them as all the other code? No. They are different, because they have a special way of usage. In fact all code has a special way of usage but we treat it in the same way. This leads us to a lot of false positives, and so no one uses code quality tools. So let's hope that for next Pharo release we will be able to have a better sense of quality changes.
In the end, as we don't have any proof that big amount red colour in the visualisation is caused by Code Critics fault, it still can mean that Pharo has a lot of problems. So if you spot something, please remember to report it on the bug tracker :).
On the other hand if you look at the package called SmartSuggestions you can see that there are a lot of red classes.
And of course we had some almost completely red buildings.
Now for important stuff. You can see that there is a A LOT of red colour. Does this mean that Pharo 3 became worse in terms of code quality? Not necessarily. We don't know if Code Critics are detecting critics well. For example some packages with small red classes are test packages. Should we skip tests for code quality analysis? No. Should we treat them as all the other code? No. They are different, because they have a special way of usage. In fact all code has a special way of usage but we treat it in the same way. This leads us to a lot of false positives, and so no one uses code quality tools. So let's hope that for next Pharo release we will be able to have a better sense of quality changes.
In the end, as we don't have any proof that big amount red colour in the visualisation is caused by Code Critics fault, it still can mean that Pharo has a lot of problems. So if you spot something, please remember to report it on the bug tracker :).
SmartSuggestions was introduced in Pharo3... so of course if went from "no code critique" to "lots of problems".
ReplyDeleteAll the blue comes from those (not many) cases that someone actually did sit down and painstakingly fixed the issues... last but not least: there are *lots* of false positives and things that are not things that people ever will fix.
We need to distinguish code critique rules that are to be fixed from those that, statistically, may hint problematic design when seen a lot, but that are not clear "errors" that would be fixable.
And for tests: a lot of "smells" are actually not smells for tests. People therefore never run the code critique for tests... e.g. "lots of methods". Is having lots of test methods in a test really bad?
I can only advise everyone who tries to do research with code critique to actually use it a bit... it's more an accidental set of random things, not something that has intrinsic value at this state.
Dear Unknown,
Deleteyou are completely right. The ting is that either code is wrong or tools are wrong. If code is wrong we have to improve code, is tools are wrong, we have to improve tools.