Popularity is not the best indicator of quality, but I don’t like this chart one bit:
24 thoughts on “Is TurboGears dying?”
Turbogears switching to Pylons has killed TG (not in the sense of disappearing without a trace but in the sense of every being a major player).
I have followed the python web-framework world for many years now and I’ve seen Cherrypy hitting rock bottom due to the same reason, needless rewrites and awful documentation.
Were I to make a prediction, I’d say we’ll see a slow death of the megaframeworks (Django, too) as people take to assembling personal frameworks from components of their liking. WSGI certainly encourages this approach — and really, it just makes plain, good sense.
What chart?
Oh, it only shows if you have off-site javascripts on. That wasn’t obvious.
“lies, damn lies, and statistics”.
When you browse into the actual data which goes into the ‘daily reach’ calculation, you will notice that the search rank information for TG seems to not be collected, or is missing. As this accounts for over 50% of the Daily reach weight, I would say this graph is at best useless. You will notice that the massive drops in the django ‘Daily Reach’ also coincide with periods where that sites search rank data is also missing.
In short, nothing to see here.
If you start looking at a 1-year graph, Django also appears to be on a downward trend, although it’s still at much higher levels than TG.
What does “Daily Reach” mean anyway? It seems to be proportional to page views…
Practically everything seems to be on a downward trend… slashdot.org is zooming downward over the past year, arstechnica.com is slowly decreasing.
Have a look at Python.org vs. Ruby-lang.org vs. Perl.org / Perl.com … ALL OF THEM are decreasing!
So I’d take this information with a hefty spoonful of salt…
Or consider how Alexa gets its information. Does it know raw site hits of these websites? Nope! Alexa is installed on tons of ppl’s computers, sometimes without them even realizing it… how many geeks are likely to mistakingly have gotten it installed on their computer (I’m assuming must geeks steer clear of installing the rather pointless Alexa toolbar)?
Then consider that Django turns up for ppl searching for a fairly well known jazz musician, while TurboGears has no non-tech association. That right there can pretty easily explain the difference in graphs.
Alexa measures how many computer illiterates or SEO douchebags visit a site (i.e., people who don’t know how to clean out spyware, or who think that spyware is a good thing). If you aren’t interested in that kind of traffic, Alexa tells you nothing.
If anything, I take from this that I want to stay away from Django, as it has too much traction with people I want to avoid.
If someone wants to search on Django’s database module, what keyword do they use? ‘Django’. If they wish to search on Django’s routing module, what keyword do they use? ‘Django’. Similarly for templating and administration.
If someone wants to search for Turbogears database module, what keyword do they use? ‘SQLAlchemy’. If they wish to search for the routing module, what keyword do they use? ‘Pylons’ or ‘Routes’. Templating is divided over many keywords, including ‘Kid’, and ‘Mako’.
To do a meaningful search comparison, you would have to measure each of the Turbogear component names popularity, and compare the sum against the Django total. There are still biases (each way), but that would be a more relevant comparison. Somewhere, recently, I saw a search count tabulation that listed most of the components’ names. Comparing the Turbogears component totals with the Django total revealed a difference of 10 or 15%.
All I see is a blank space under “I don’t like this chart one bit”.
If it’s a chart, wouldn’t that be best shown as a simple PNG image? Flash isn’t a standard on the web, you know, while PNG is.
Email list traffic stats and version control activity would probably be more interesting to see.
I’ve got my TG popularity shock when I compared the number of people in #turbogears vs #django. Of course, this measurement by itself is arguable. But I believe that right now Django is more popular than TG. Nevertheless, TG is still my favorite. As you said, popularity has nothing to do with quality. 🙂
I wouldn’t say the switch to Pylons killed TG, but it definitely made me hesitate when choosing a Python webframework – what sense does it make to code for v. 1.0 when there soon will be a different v. 2.0? Django looks less like a moving target. Perhaps TG will have an advantage in the world that Zope people foresee (end of webframeworks, dawn of WSGI-component ad hoc architectures), but for the time being Django simply looks like the safer choice.
Although Compete might become a better-than-Alexa ranking site, for now their sample sizes are way too small to be particularly useful.
I’ve seen plenty of arguments that suggest Alexa (and Compete) are blunt sticks at best, useful only for trend analysis on one site rather than for comparisons of traffic between sites (but more on that below).
As David says above Django has all of its components on the one site whereas TG has SQLObject/Alchemy, Genshi/Kid and CherryPy/Pylons all on different sites.
Alexa shows a few spikes but a flat or reducing trend. My Analytics shows a steady linear increase. Look at the mini page-views picture (1.2 million page views) below the main picture – that follows exactly the same linear upward trend as for my daily-visits.
Why doesn’t Alexa agree that my daily pageviews are growing in a linear fashion? Is Alexa of any use at all if it can’t estimate this basic statistic correctly? Perhaps we should just ignore Alexa…if we can’t trust the stats then we can’t use them to draw judgements.
Is Wikipedia a useful source for measuring popularity? I don’t think so, especially in projects that are rapidly evolving (for note: TG has 8 edits since August, Django has 30 – both pages are relatively short).
What would make for better measures of popularity? I’d have thought that if you want help then you want to know how much documentation is available (nbr pages of docs?) and how many active developers talk in IRC and the newsgroups. Could anyone estimate these stats please?
How about how many well-known production sites use each framework? That gives some indication of how many developers will choose to follow a particular framework (because ‘they saw it used on other sites they like’).
Side note – we use TurboGears for ShowMeDo, we’re very happy with the code though more docs on the widgets really wouldn’t go amiss (!). We are of course much obliged to the TG community for all their great work. We’re sticking with TG – long live TG!
Wow. Lot’s of denial in this thread. But the phenomenon is corroborated by plenty of other evidence. The heyday of TG’s popularity was late 2005, early 2006. Both TG and Django use Google Groups as their primary avenue of communication, so it is easy to verify that activity on both the TG users and developers list have been on a downhill slide for quite some time, having a current activity that is only a small fraction of what once was. And it is easy to compare those levels of activity with those of their Django counterparts.
What has puzzled me is that it has taken so long for anyone to notice.
I was interested in TG from late 2005 through early 2007.
But the “best of breed” components which made up Turbogears in 2005, and which still comprise the latest official version, are now pretty much old throw aways which nobody wants. SQLObject is widely regarded as “the little oject relational mapper that couldn’t” long abandoned by its author. Mochikit is pretty much defunct. It’s still stuck using the old Cherrypy 2. And Kid is being thrown out for Genshi (which is admittedly essentially Kid 2.)
So “best of breed” TG seems perpetually, and slowly, trying to reinvent itself. Now with SQLAlchemy, Genshi, Cherrypy 3 and, surprisingly… Mochikit again, which is now pretty much at the bottom of the stack of javascript libs for popularity.
Add to that the nearly complete disregard for proper documentation and the so-so quality of the one extant book… and is it at all surprising that TG’s popularity has waned?
Constant self-reinvention, a Frankenstein API, and poor, incomplete, and obsolete documentation drove me to a framework which does not suffer from those ills.
TG desperately needs a finished 2.0 release and *proper* documentation for it in order to have a hope of regaining popularity.
Steve, great post. And I’m not just saying it because you give voice to my frustration over pouring so many hours into what looks like a doomed project.
Constant self-reinvention, a Frankenstein API, and poor, incomplete, and obsolete documentation drove me to a framework which does not suffer from those ills.
Tell me what framework doesn’t suffer these ills! More generally, are you happy with it?
Well, here I have to be quite careful, because I honestly don’t want to seem like I am trolling for a particular competing framework. But… I have moved to Django. Adrian, Zacob, and Simon have stayed the course, refining what they have and resisting the temptation to rewrite and reinvent unless it really makes sense. The project’s dedication to excellent documentation is an absolute *joy* to me. Andrian’s and Jacob’s book (“The Definitive Guide to Django: Web Development Done Right”) was published earlier this month. It is a paper book, but is also available as a pdf, and for free in html format at http://new.djangobook.com. (And isn’t it neat that way Apress allowed that?) I am just taking a break from working through the book as I type this, and am finding it to be of excellent quality.
My observation, from watching the two teams, is that ultimately it is easier, less time consuming, and makes for more polished results, to maintain an integrated whole than to glue disparate components together and keep them glued together.
That said, nothing in life is perfect. And there are two things regarding Django about which I am ambivalent::
1. The templating system was intentionally designed, based upon MVC philosophy, to discourage one from including business logic in it. This is probably a good thing. But I sometimes feel handcuffed while using it.
2. The developers have *very* strong feelings that the framework should not commit to any particular javascript library. This means that AJAX support is limited to serialization of data to XML or JSON. You won’t find any ajaxified widgets or built in javascript goodies. The philosophy is that you should be free to select your own JS library.
To that end, I just today received my copy of “Learning JQuery” from Amazon.
One other comment. Django has a reputation for being “good for content oriented sites”, and it is. But I’ve done a reporting oriented app in it, and was quite pleased with the results. And I think that once I get past my “I’m so intimidated” phase with JQuery and Javascript, that I will likely be even more pleased with Django with respect to “web apps” as opposed to “web sites”.
Thanks for the feedback. I’m starting a side project with a guy that knows Django a lot better than me, so soon I’ll be able to find out if the grass really is greener.
I agree that these uber frameworks are still to new and in their infancy. I know of alot of companies using CP and home growing their own templating environment.
Regarding you Matt’s Dec 22 question about typical framework problems, I though of Spring Framework (Java) because it is pretty good on those dimensions.
Thanks for the overview!
I dont agree with the post completely
This posting is marvelous and what a fantastic research that you have done. It has helped me a lot. thank you very much.
Turbogears switching to Pylons has killed TG (not in the sense of disappearing without a trace but in the sense of every being a major player).
I have followed the python web-framework world for many years now and I’ve seen Cherrypy hitting rock bottom due to the same reason, needless rewrites and awful documentation.
Were I to make a prediction, I’d say we’ll see a slow death of the megaframeworks (Django, too) as people take to assembling personal frameworks from components of their liking. WSGI certainly encourages this approach — and really, it just makes plain, good sense.
What chart?
Oh, it only shows if you have off-site javascripts on. That wasn’t obvious.
“lies, damn lies, and statistics”.
When you browse into the actual data which goes into the ‘daily reach’ calculation, you will notice that the search rank information for TG seems to not be collected, or is missing. As this accounts for over 50% of the Daily reach weight, I would say this graph is at best useless. You will notice that the massive drops in the django ‘Daily Reach’ also coincide with periods where that sites search rank data is also missing.
In short, nothing to see here.
If you start looking at a 1-year graph, Django also appears to be on a downward trend, although it’s still at much higher levels than TG.
What does “Daily Reach” mean anyway? It seems to be proportional to page views…
Practically everything seems to be on a downward trend… slashdot.org is zooming downward over the past year, arstechnica.com is slowly decreasing.
Have a look at Python.org vs. Ruby-lang.org vs. Perl.org / Perl.com … ALL OF THEM are decreasing!
So I’d take this information with a hefty spoonful of salt…
Or consider how Alexa gets its information. Does it know raw site hits of these websites? Nope! Alexa is installed on tons of ppl’s computers, sometimes without them even realizing it… how many geeks are likely to mistakingly have gotten it installed on their computer (I’m assuming must geeks steer clear of installing the rather pointless Alexa toolbar)?
Then consider that Django turns up for ppl searching for a fairly well known jazz musician, while TurboGears has no non-tech association. That right there can pretty easily explain the difference in graphs.
Alexa measures how many computer illiterates or SEO douchebags visit a site (i.e., people who don’t know how to clean out spyware, or who think that spyware is a good thing). If you aren’t interested in that kind of traffic, Alexa tells you nothing.
If anything, I take from this that I want to stay away from Django, as it has too much traction with people I want to avoid.
If someone wants to search on Django’s database module, what keyword do they use? ‘Django’. If they wish to search on Django’s routing module, what keyword do they use? ‘Django’. Similarly for templating and administration.
If someone wants to search for Turbogears database module, what keyword do they use? ‘SQLAlchemy’. If they wish to search for the routing module, what keyword do they use? ‘Pylons’ or ‘Routes’. Templating is divided over many keywords, including ‘Kid’, and ‘Mako’.
To do a meaningful search comparison, you would have to measure each of the Turbogear component names popularity, and compare the sum against the Django total. There are still biases (each way), but that would be a more relevant comparison. Somewhere, recently, I saw a search count tabulation that listed most of the components’ names. Comparing the Turbogears component totals with the Django total revealed a difference of 10 or 15%.
All I see is a blank space under “I don’t like this chart one bit”.
If it’s a chart, wouldn’t that be best shown as a simple PNG image? Flash isn’t a standard on the web, you know, while PNG is.
Email list traffic stats and version control activity would probably be more interesting to see.
I’ve got my TG popularity shock when I compared the number of people in #turbogears vs #django. Of course, this measurement by itself is arguable. But I believe that right now Django is more popular than TG. Nevertheless, TG is still my favorite. As you said, popularity has nothing to do with quality. 🙂
I wouldn’t say the switch to Pylons killed TG, but it definitely made me hesitate when choosing a Python webframework – what sense does it make to code for v. 1.0 when there soon will be a different v. 2.0? Django looks less like a moving target. Perhaps TG will have an advantage in the world that Zope people foresee (end of webframeworks, dawn of WSGI-component ad hoc architectures), but for the time being Django simply looks like the safer choice.
At first blush I see a similarly horrid graph at Compete.com:
http://siteanalytics.compete.com/djangoproject.com+turbogears.org/?metric=uv
Although Compete might become a better-than-Alexa ranking site, for now their sample sizes are way too small to be particularly useful.
I’ve seen plenty of arguments that suggest Alexa (and Compete) are blunt sticks at best, useful only for trend analysis on one site rather than for comparisons of traffic between sites (but more on that below).
As David says above Django has all of its components on the one site whereas TG has SQLObject/Alchemy, Genshi/Kid and CherryPy/Pylons all on different sites.
Looking at the Alexa traffic for CherryPy, TurboGears.org, SQLObject, SQLAlchemy and PylonsHQ.com we can see more traffic, though admittedly not as much as Django:
http://www.alexa.com/data/details/traffic_details/downloadhelper.net?site0=cherrypy.org&site1=turbogears.org&site2=sqlobject.org&site3=sqlalchemy.org&site4=pylonshq.com&y=r&z=3&h=400&w=700&range=1y&size=Large
How useful *is* Alexa? Compare the last year’s traffic for my ShowMeDo using Alexa’s record of page-views:
http://www.alexa.com/data/details/traffic_details/downloadhelper.net?site0=showmedo.com&y=p&z=3&h=300&w=610&range=1y&size=Medium
against my Google Analytics for the same period for daily-visits:
http://ianozsvald.com/FivePoundAppPresentations/FivePoundAppDayPerilsBootstrapping/ui/default/AnalyticsSmall.png
Alexa shows a few spikes but a flat or reducing trend. My Analytics shows a steady linear increase. Look at the mini page-views picture (1.2 million page views) below the main picture – that follows exactly the same linear upward trend as for my daily-visits.
Why doesn’t Alexa agree that my daily pageviews are growing in a linear fashion? Is Alexa of any use at all if it can’t estimate this basic statistic correctly? Perhaps we should just ignore Alexa…if we can’t trust the stats then we can’t use them to draw judgements.
Is Wikipedia a useful source for measuring popularity? I don’t think so, especially in projects that are rapidly evolving (for note: TG has 8 edits since August, Django has 30 – both pages are relatively short).
What would make for better measures of popularity? I’d have thought that if you want help then you want to know how much documentation is available (nbr pages of docs?) and how many active developers talk in IRC and the newsgroups. Could anyone estimate these stats please?
How about how many well-known production sites use each framework? That gives some indication of how many developers will choose to follow a particular framework (because ‘they saw it used on other sites they like’).
Side note – we use TurboGears for ShowMeDo, we’re very happy with the code though more docs on the widgets really wouldn’t go amiss (!). We are of course much obliged to the TG community for all their great work. We’re sticking with TG – long live TG!
Wow. Lot’s of denial in this thread. But the phenomenon is corroborated by plenty of other evidence. The heyday of TG’s popularity was late 2005, early 2006. Both TG and Django use Google Groups as their primary avenue of communication, so it is easy to verify that activity on both the TG users and developers list have been on a downhill slide for quite some time, having a current activity that is only a small fraction of what once was. And it is easy to compare those levels of activity with those of their Django counterparts.
What has puzzled me is that it has taken so long for anyone to notice.
I was interested in TG from late 2005 through early 2007.
But the “best of breed” components which made up Turbogears in 2005, and which still comprise the latest official version, are now pretty much old throw aways which nobody wants. SQLObject is widely regarded as “the little oject relational mapper that couldn’t” long abandoned by its author. Mochikit is pretty much defunct. It’s still stuck using the old Cherrypy 2. And Kid is being thrown out for Genshi (which is admittedly essentially Kid 2.)
So “best of breed” TG seems perpetually, and slowly, trying to reinvent itself. Now with SQLAlchemy, Genshi, Cherrypy 3 and, surprisingly… Mochikit again, which is now pretty much at the bottom of the stack of javascript libs for popularity.
Add to that the nearly complete disregard for proper documentation and the so-so quality of the one extant book… and is it at all surprising that TG’s popularity has waned?
Constant self-reinvention, a Frankenstein API, and poor, incomplete, and obsolete documentation drove me to a framework which does not suffer from those ills.
TG desperately needs a finished 2.0 release and *proper* documentation for it in order to have a hope of regaining popularity.
Steve, great post. And I’m not just saying it because you give voice to my frustration over pouring so many hours into what looks like a doomed project.
Tell me what framework doesn’t suffer these ills! More generally, are you happy with it?
Well, here I have to be quite careful, because I honestly don’t want to seem like I am trolling for a particular competing framework. But… I have moved to Django. Adrian, Zacob, and Simon have stayed the course, refining what they have and resisting the temptation to rewrite and reinvent unless it really makes sense. The project’s dedication to excellent documentation is an absolute *joy* to me. Andrian’s and Jacob’s book (“The Definitive Guide to Django: Web Development Done Right”) was published earlier this month. It is a paper book, but is also available as a pdf, and for free in html format at http://new.djangobook.com. (And isn’t it neat that way Apress allowed that?) I am just taking a break from working through the book as I type this, and am finding it to be of excellent quality.
My observation, from watching the two teams, is that ultimately it is easier, less time consuming, and makes for more polished results, to maintain an integrated whole than to glue disparate components together and keep them glued together.
That said, nothing in life is perfect. And there are two things regarding Django about which I am ambivalent::
1. The templating system was intentionally designed, based upon MVC philosophy, to discourage one from including business logic in it. This is probably a good thing. But I sometimes feel handcuffed while using it.
2. The developers have *very* strong feelings that the framework should not commit to any particular javascript library. This means that AJAX support is limited to serialization of data to XML or JSON. You won’t find any ajaxified widgets or built in javascript goodies. The philosophy is that you should be free to select your own JS library.
To that end, I just today received my copy of “Learning JQuery” from Amazon.
One other comment. Django has a reputation for being “good for content oriented sites”, and it is. But I’ve done a reporting oriented app in it, and was quite pleased with the results. And I think that once I get past my “I’m so intimidated” phase with JQuery and Javascript, that I will likely be even more pleased with Django with respect to “web apps” as opposed to “web sites”.
Thanks for the feedback. I’m starting a side project with a guy that knows Django a lot better than me, so soon I’ll be able to find out if the grass really is greener.
I agree that these uber frameworks are still to new and in their infancy. I know of alot of companies using CP and home growing their own templating environment.
Regarding you Matt’s Dec 22 question about typical framework problems, I though of Spring Framework (Java) because it is pretty good on those dimensions.
Thanks for the overview!
I dont agree with the post completely
This posting is marvelous and what a fantastic research that you have done. It has helped me a lot. thank you very much.