Printable View
We want every search engine spider on the planet here, as often as they like. Since the upgrades, all the sites running the new software show improvement in SE ranking. If you search e-commerce forums, our e-commerce site comes up #1 and that 1 was upgraded just before this one.
I have to agree 100% on this. Once we are in a SE database, the bots will keep visiting to make sure the data they have is correct. And the more active VirDr is the more often it will get queried by the SE. And a popular site or frequently used site will get visited more often. However, the more active a website, the more often it will get visited. If a website changes frequently, the search engine will send bots by more often. This is also true if the website is extremely popular, or heavily trafficked.Quote:
Originally Posted by JPNYC
This could account for the jump in the individual queries as we change alot in a twenty four hour period. Now the question is can we separate the true guest queries from the SE queries. If we can do that then we could run more accurate statistics. Or compile our information differently.
And are there any other types of queries that should be considered?
And what are there, nine major search engines to consider?
This may be the skewing I referred to in an earlier post.
What we have then are Search Engines that because of our software are finding us user freindly. This is what we want to have. We can take and increase our marketing by looking at this site through the eyes of a SE looks like to me.
Well as far as I can see, the number of users online has only a very minor level of importance. We use separate software to check usage, and this software also has # of post stats by month available to admins.
Wow, let’s calm down here a little. Nobody is under attack here. No one said anyone or anything someone was doing was “suspicious”. The term was used strictly in the context of the guests online number jumping dramatically (the rise being suspicious). This is just a discussion going on here of something noticed (not just here but in other threads) since the upgrade. Something that is on one hand a curiosity, and on the other is something (as it relates to people actually online at vdr throughout the day, etc) that does tie in to the health and well being of vdr. And this is something we all have a stake and an interest in, and something in our collective province.
First, no one was or is talking about or advocating preventing legitimate search engine probing of this site. No one has said that we would want to do anything that would negatively impact things that would legitimately result in the raising of our search engine rankings and presence. But there can be activities by rouge/bad bots/spiders and probing by engines that is useless (or less useful), and it is these types of things that can fall in to the category of stuff perhaps to be blocked if possible. Now we already prevent a lot of useless and unwanted activity by restricting the viewing/accessing of certain things to only logged on users (thank god). But there’s certainly no downside to reducing (further reducing) useless/unwanted probing and bandwidth utilization where it might exist. This is all just brought out in the context of a “discussion”.
But let’s back up maybe a little bit here… The dramatic increase in “guests” online was noticed and has come up before since the upgrade (by me, us, other members). It may not have been something keeping people awake at night worrying about, but… And up to this point it is pretty clear we (all) did not have a real (valid) understanding of what was going on in regard to the “guest” number (by virtue of postings about it previously). The assumption seems to have been that this number by and large represented members who hadn’t logged on and/or people visiting/browsing the site that had not yet registered. In other words, “guests” were seen pretty much as real people.
But now (by virtue of seeing what other site owners and vbulletin creators/developers are saying) it seems this is not true necessarily, at least not with VB3. In fact, the guest number can be and often is largely not representative of real people lurking at a board. Because of VB3’s much greater spider/bot friendliness it turns out that “substantial” connections can be generated by these spidering/bot activities, and these connections show up as “guests”. And from what others seem to say/see the guest number often is comprised mostly of connections that are not by real people. With our prior software this was not the case. Because it was much less spider/bot friendly the connections generated through this activity were far less (seemingly negligible). So, whereas under our prior board software we could pretty much look at guest numbers as a valid (much more valid) indication of real people (browsing) on vdr at a given time, now it seems this is, or often may not be, the case. So if nothing else some new and greater understanding of or insight into a phenomenon going on since the upgrade was done.
Now it may well be that now we can no longer really look at online numbers, or at least the guest part, for any meaningful info/measures. And if so, then that is good to know. But, when the online numbers do (or do largely) reflect “real people”, or can be made to do so, they can be of some degree of value. They can provide one piece of information in bigger pie. In the “vdr seems a little sick” thread it was discussed and considered of some meaning. It can (when the numbers are real) be of some value in such areas as looking at lurkers vs those actually registering, looking at (existing) member participation (another key priority we have moving forward), doing some site-to-site comparisons, etc. Bottomline: it isn’t that these numbers are the end all to beat all, just that they could (if real) be added info.
Now… Sparks, you mention that you are showing numbers of new members registering since the upgrade inline with the online numbers. I’m not sure what you mean by this. It might help if ya gave some actual numbers and breakdowns. As a comparison, it looks to me (based on memberlist data) that there have been (give or take a couple) 372 new registrations since the new board went in and through yesterday. Now that's something like a doubling in the rate compared to the period in 2005 prior to the upgrade. But we very frequently/generally see our online guest numbers in higher proportions than this relative to what they were pre-upgrade. Just out of curiosity what are your numbers, how are you correlating registrations to online users, etc?
On the new members registrations… It is good to see that we appear (can’t jump the gun quite yet) to have taken a good step in the right direction as far as increasing new member registrations. In the period from the beginning of 2005 until the upgrade, 75 days, we had added 567 new members (down from 893 during the same period a year ago). Since the upgrade, 24 days, we have added 372. That’s a real good and positive rate change. Now, to be fair, we didn’t do quite as well (close though) as we did (even without search engine presence) in the same period (3/17-4/09) last year when we added 418 members (hmmm, makes ya wonder what our numbers would have been last year had we had the better SE presence). Nonetheless, positive steps must be seen as positive steps. We’ll have to see how the second quarter of 2005 shapes up. If we can maintain our current average rate we can come in inline with new member additions for the same quarter last year. Time will tell…
Im sure this will ruffle someones feathers....but....darn...dont you all have anything better to do....spend this time helping with problems or doing research.....worrying about the number of guests and if they are accurate....seems like a terrible waste of a gorgeous Sunday afternoon....I for one...am going to waste it watching the NASCAR race and The Masters.
I have to admit ya cracked me with that post Sarr.:D Ya gotta admit it's pretty funny when you are making suggestions about spending more time helping users with problems. That was good. ;)
Well I suppose I could be cleaning my apt. On the whole, I'd rather be doing this. :)
Maybe odd would be better than suspicious.
The fact that Vbulletin 2 was not search engine friendly didn't stop the bots trying to index the site, it's simply that the dynamic nature of the forums were essentially unindexable. You used to be able to see the bots with the old software, because whenever they tried to access parts of the site such as the moderator tools (move thread, etc.) it would show in Who's Online as a Guest doing "Moderating duties". And there were a lot of guests doing that sort of thing. That oddity seems to have been fixed in VB3, so it is not so easy to guestimate the number of bots now. But I cannot think of any reason why the number of bots will have altered much, they were always around, the difference is that now they actually get something tangible to index, which is why the search engine rankings have improved so much.
Yeah the dynamic links just bollix up the bots entirely. Because they crawl the links but the links, like latest post, remain the same but the content changes.
What I was trying to say is it does not seem to be worth the time to gripe about this particular subject....there has to be better ways to utilize your time.