I just heard that there is a bill in the senate that will force all online retailers to pay sales tax, regardless if they operate in that state. This is wrong for a few reasons:
Online retailers are at a disadvantage to traditional stores because of shipping fees. Local stores usually do not have to worry about these added fees during a purchase.
Online retailers sell items tend to be cheaper than local stores. Part of of the reasons might be that in order to compensate for the time it takes to receive your purchase, the buyer has to be enticed to save a few bucks in the process. Otherwise, you might as well go to the store, right?
Local stores play the game of increasing prices on personal items that consumers NEED to have. For example, shaving blades. You either don’t shave for 3 days, or you bite the bullet and buy it in the store even though it costs more.
Local stores, such as Best Buy, mark-up prices for HDMI cables to $50+ dollars so uninformed consumers overpay for something that can be bought online for $5 dollars. You expect me to feel sorry for local chains when they behave like this? Then they offer extended warranties on TVs that are meaningless? I don’t think so.
And finally, the online sales tax “loophole” as some people have described it, is the ONLY loophole that us common folk can use. Members of Congress and other big corporations use many tax loopholes so they end up paying less taxes or no taxes than people making $50,000. These will never be closed because it benefits them. I smell hypocracy.
Before you start taxing all online retailers, you might want to look at forcing ALL companies that operate in the United States to pay corporate taxes. Or should the 99% continue to be screwed over?
This post might bite me in the ass one day, but what the hell right?
Running your own start-up is not easy. In fact, its insanely frustrating for many reasons. Try telling ‘regular’ people your idea over and over and for it to just go straight over their heads. Either that or they’ll instantly think its crap because they don’t understand it. Then they’ll wonder, “seriously, what do you REALLY do for work?” You get to the point of asking yourself: “Why bother even mentioning it?”.
Worse yet, try pitching your start-up and/or idea to a tech blog. I recently sent an email to a tech blog explaining what FantasySP does and why its a neat place to go for fantasy sports fanatics. It really works and it really provides an incredibly useful service. I even provided my Google Analytics stats. The response? “Nice, just trying to think of an angle…“. An angle? Are you kidding me? The angle is that it’s a bootstrapped profitable start-up with loads of cool features.
He doesn’t give a shit that the business is profitable with me working on it 20 hrs a week. He doesn’t give a shit that I’ve coded a product that people use, to the tune of 700,000 pageviews in September. Or that I’ve managed to keep costs under $300 per month and make approximately 10X that in revenue. Or that my start-up outperforms some that have 10+ employees with over a million dollars in funding. Or that I’ve done it all myself with no prior experience of the business world. I can go on and on, but none of that matters.
But this is my fault for foolishly not knowing.
What matters is that my start-up needs an angle that they find interesting. It doesn’t matter if your startup even does what it claims to do or if your profitable. If I pitched some shitty idea revolving an iPad app and daily deals with facebook integration I bet they’d be at least following up my email with questions. Why? Because its on-the-surface buzzword bullshit that attracts them to write and bring in the pageviews.
If you want your product mentioned, then modesty is not going to work. You have to play their game by overvaluing your product. Make outrageous claims like its the next Twitter meets Groupon. Lets look at a start-up like Color. I mean, why the hell did anyone even write about that shithole of an idea to begin with? Because Color is a product of arrogant bullshit from people who have been successfull before, and it will outweigh any logical thought or reasoning as to if it might work.
Everything about startup culture is complete bullshit. People would rather believe and write about pompous arrogant assholes who lie through their teeth about their product. I bet half the time, the things written in articles are BARELY half-truths. Why else would all of these start-ups go belly up within a few months?
I give people in the start-up world too much credit (which, if you knew me, is actually pretty remarkable). People only skip across the surface, and I just have to face the facts and move along. I’m no longer focusing on getting an investor on board, or trying to be mentioned in a tech blog, or trying to be acquired. If these things happen, great, but I’m not about to waste my time when I could be improving my existing product.
I’ve seen a few well known entrupenurs make mention of services that they pay for and I’d like to join the club. For those of you unfamiliar with me, I run a fantasy sports news aggregator called FantasySP. You can sync fantasy leagues rosters/transactions and be alerted of real-time player tends. This is bootstrapped in every single way, as I am the sole employee and have zero funding.
The following companies are awesome and deserve my money and/or praise.
Not a flashy name in the hosting industry, but they provide solid managed services. I use their Solo Express server to run FantasySP on a standard LAMP stack. A few other sites (like this one) also run on it. It is actually about 10% cheaper than this because I pay a year in full.
For all of those background processes that need to be run for FantasySP, a cheap cloud server from Rackspace does the job with a very fair price. Highly recommend it to anyone who wants a cheap host to screw around with or run dozens of cronjobs. 🙂
Amazon Cloudfront is where I store most of my images,stylesheets, and javascript to speed things up. For the images that slip through the cracks, there is Cloudflare, which offers an amazing service that is part CDN, part firewall, and part dns optimizer. I have written about them in the past. I use their free service, the rest of the $8 is spent on Amazon’s Cloudfront.
The best real-time analytics package out there. I have no idea why people in the industry seem to blindly love chartbeat. Clicky is better in every way combing historical and real-time analytics in a great UI.
The best real-time software to monitor how your application is performing. This has enabled me to spend less money on my hosting due to optimizations based on what NewRelic data and graphics tells me. FantasySP zooms and NewRelic has a lot to do with it. With around 700,000 pageviews per month, FantasySP remains rock solid. These guys rock and are totally worth the money.
Notepad++ for windows, its free and it does just about everything those super expensive pay ones do. (Why the heck is UltraEdit so damn expensive?)
Grand Total: $295 per month
This brings my grand total of web developer / startup related expenses to about $295. The good news is that FantasySP more than makes up for these expenditures. I try to not waste any money, so how do my expenses stack up to yours? I am guessing most companies pay at least $500 per month JUST for hosting.
Full Disclosure, some of the links provided here are affiliate links.
If you are like me, then you’re sick of the Google DEV channel getting screwed up by new bugs. (I thought that was the whole point of the Canary build?) Recent hair-pulling bugs include: fonts no longer rendering properly, sluggish performance on Twitter, or random crashes that weren’t there a build ago.
Well, fear not. I just decided to downgrade from Chrome DEV to BETA and I kept ALL my user data. Here is what you do:
Backup your “User Data” folder first. For windows 7 users: C:\\Users\YOUR-USERNAME\AppData\Local\Google\Chrome\User Data\ . You’ll notice that this folder is fairly big, in fact, mine weighed in at around 1 gig. If you are on an earlier version of windows or a different OS, then head here for more paths.
Next, close Chrome and go to Add/Remove programs to uninstall. When asked, Do not remove your user data.
This is just one of those random thoughts I had. As a programmer, everything I do involves taking complex problems and breaking them down into small pieces of logic. I look for the simplest and most effective answer to a problem. Pretty cut and dry, right? But after being a programmer for almost 10 years now, I’ve realized that I apply this mindset to just about everything I do.
I can’t say this is necessarily a good thing because most people do not agree with many of my opinions and observations. I often get remarks that I am being “too harsh” or “not giving them a chance” or “being judgmental” or “I don’t know all the facts”. More often than not, I tend to be right. But why don’t other people see what I see? I’m not overly smart. I think I am just wired to think differently, and the reason is because I became a programmer. Call it the programmers curse.
Here, I’ll show you…
Have to go food shopping? Go at night, less people means less time to waste. Shop for clothes? Wait for a 30-40% off sale, then just get everything at once online with free shipping. Need a new television? Well, I’m certainly not just going to get the cheapest one out there. Time to look at reviews and add a deal alert from Slickdeals and patiently wait. Time to buy a new console? Well, which one is easier to mod because I surely don’t see myself spending $60 per game.
Is it hard to be logical? Absolutely. It’s extremely hard. In fact, I would argue that its much easier to be illogical than to be logical. To come to an illogical conclusion requires almost no thought and relies mostly on impulse and emotion. An illogical person is not looking for the simplest and most effective answer to a problem. They don’t care if they are right or wrong. I think more people are illogical rather than logical. But I don’t blame them; it’s hard.
Think about some of the toughest problems or topics that people seem to fight over.
Politics? That reeks of illogical people who really don’t care what the truth is. We have a debt problem. What’s the most logical decision to fix that? Cut spending and raise taxes? For who? The rich? Some argue that the money companies save from tax breaks eventually trickle down to us poor folk. They create jobs. Is their logic flawed? Are they aware of the wealth breakdown in the United States? Is it time for a change? The real facts are out there waiting for you to soak them in.
Religion? Again, this is another area where it seems largely devoid of logic. Think of the simplest and most effective answer. Take a step back and learn about human history and the laws of the universe. Most people won’t end up at the conclusion I come to.
What about technology? This is a very tough area and there are a lot of illogical loyal users out there. I’m sorry folks, but I just can’t bring myself to buy a $600 iPad when an iPhone or Android is essentially the same thing. I am not loyal to a company. I use a Macbook for work and a PC at home. They both work well for me. My web browser? I used to love Firefox, but they fell behind in innovation and features to a 2 year old browser. Do I stay loyal to Firefox? Hell no, logic dictates I use the fastest and feature rich browser out there.
Relationships are by far the hardest to use logic with. I don’t even to go into that one do I? 🙂 Oh hell, why not? What I do drives my wife crazy because I can’t just blurt things out or make seemingly simple decisions and move on. If she does something overtly illogical, I want to try to figure out why she did it. She will look at me and say: “WOULD YOU STOP THAT? ARE YOU JUDGING ME!”. I’m not judging. I am analyzing. I just want to know WHY. See what I mean? I can’t help myself.
What about in the business world? I see people over and over making the wrong decisions. (Surprise surprise right?) I assume there is logic behind these decisions, but perhaps they are too lazy to REALLY delve into it to fully understand the problem? Perhaps they are just looking in the wrong place for advice? The stats are there, so why don’t they listen to them? I wish I knew.
Again, don’t take this article the wrong way. I am not saying I am the smartest or the most logical person in the world. I am not saying everything I wrote in this article is the truth. There are times when my logic is wrong. All I am saying is that I try to be as logical as I can be. But is that the best way to go about things? You know, when I think about it…sometimes being illogical seems like the logical thing to do.
Anyone who has a WordPress blog with a lot of posts will eventually encounter an extremely slow query. I refer to this as the SQL_CALC_FOUND_ROWS Bug. If you have slow-query-log enabled then a query similar to this might have shown up before:
SELECT SQL_CALC_FOUND_ROWS wp_posts.* FROM wp_posts WHERE 1=1 AND wp_posts.ID NOT IN (44682, 44657, 44630) AND wp_posts.post_type = 'post' AND (wp_posts.post_status = 'publish') GROUP BY wp_posts.ID ORDER BY wp_posts.post_date DESC LIMIT 24580, 5
Just how detremential is this to your blogs performance? Well thanks to newrelic, I can show you:
SQL_CALC_FOUND_ROWS Bug
Unfortunately I don’t know why WordPress runs this query. What I do know is that it apparently only shows up on index.php. What you probably care about is how to fix this problem. I’ve located a possible workaround thanks to this open ticket at wordpress.org. The diff log on the changes are listed as well. I went ahead and applied these code changes to wp-includes/query.php. The fixed query.php can be found here.
How are the results so far? Inconclusive. I just applied this patch and nothing broke so far, which is always a plus. I suggest you give it a try and see how your blog responds in a development environment. If I STILL spot slowdowns in the revised query, then I will update this post and let you know.
Some of you might be asking, does this affect your version of WordPress? The answer is yes. I am running WordPress 3.2.1.
Please do post your thoughts, concerns, or comments to help others out.
The web is a mighty big place and the fantasy sports market has grown by leaps and bounds over the past 5 years. All of that growth spawned many sites that have come and gone, in the news one day and no where to be found 6 months later. Some of today’s best fantasy sites are having record traffic numbers, and for good reason, because they provide incredible fantasy insight.
What people often forget about, is the technology behind these sites. Are fantasy sports sites up to date using the latest technology and do they spend time optimizing the sites for speed? I will be tackling the latter category in this blog post: speed. The speed at which a website loads is a huge factor in end user satisfaction. Below you will find the best known names in fantasy sports news and I will compare them to the big boys to see how they stack up in speed. As most of you know by now, I developed a site called FantasySP and will also show you how FantasySP compares to these sites in terms of speed.
In order to test each website’s performance, I will be using Firebug 1.7.2 with YSlow 2.1.0 under Firefox 4.0.1. I will load each site’s homepage with advertisements shown and advertisements blocked (using AdBlock Plus). I will also be using Pingdom’s Site Speed tool to test load times. This should give a fair comparison on the speed of each site.
#1) Rotoworld
Rotoworld needs no introduction as it is the premier site when it comes to fantasy news and analysis. However, does the same hold true when it comes to page speed and size?
Rotoworld with Advertisements
With advertisements shown, you see 228 HTTP requests and a size of almost 2 meg on initial page load. Your second page load has 212 HTTP requests and 104k in size. Without a question, they do a terrible job at optimizing their site for speed.
Rotoworld - No Advertisements
Without advertisements, loading Rotoworld has 214 HTTP requests and around 1.8 meg in size. The second pageload is at 202 HTTP requests and page size is about 40k. Blocking advertisements at Rotoworld will save you a little bit of time, but not much since their site is in desperate need of optimization.
Perhaps the second biggest name when it comes to fantasy news is Rotowire. Are they any better when it comes to load times?
Rotowire with Advertisements
Rotowire has 165 HTTP requests and the size of the page is 1.6meg with advertisements. Second pageload will see 134 HTTP requests and around 150k in size. Slightly better than Rotoworld, but still quite poor.
Rotowire without Advertisements
Without advertisements Rotowire has 103 HTTP requests at 1.3 meg in size. The second pageload has 88 HTTP requests and 70k in size. Rotowire has room for improvement, but it seems most of the extra requests and bloat come in the form of advertisements.
kffl is another top fantasy news site in the industry, perhaps they can do a better job when it comes to speeds?
kffl with Advertisements
As you can see HTTP requests with advertisements are at 97 and the size of the page is around 1 meg. Second page load shows 88 HTTP requests and 75k in size. So far kffl does a much better job in terms of speeding up the site and minimizing requests and size of the page.
kffl without Advertisements
With no advertisements, initial page load has 83 HTTP requests with the size of the page at around 820k. Second page load has 74 requests with 77k in size. Not too bad at all. In fact, if they fixed the amount of images that load and the amount of javascript files then they would be a lot closer to 50 http requests.
Pingdom Load Time: 10 seconds. (Why 10 seconds? My guess is that their server isn’t as expensive as the other two and they are in desperate need of a CDN to help with load time)
4) RotoInfo
RotoInfo isn’t a huge name, but Fanball is no more and these guys were the only other ones I could come up with for good fantasy analysis and news. So let’s have a look at how they perform.
RotoInfo with Advertisements
Initial page load with advertisements has 158 HTTP requests and weighs in at a hefty 3.2 meg. Second page load has 46 HTTP requests and 80k in size. Looks like we have a new winner for total page size thanks to their enormous added size in images.
RotoInfo without Advertisements
With advertisements blocked, it has 147 HTTP requests at 3.1 meg. Second page load has 35 HTTP requests at 72k. They are pretty much as bad as Rotoworld and Rotowire.
Looking at the past 4 fantasy news websites, it is blatantly obvious that these guys aren’t the best when it comes to speeding up their sites. kffl is the best at keeping HTTP requests down and page size low, but that didn’t seem to translate to a fast page. Rotoworld, Rotowire, and RotoInfo are remarkably bad at utilizing modern techniques to make sure their sites are as fast as they can be. Some are lacking a CDN, others are riddled with too many HTTP requests, and most have both problems. All of these things are fixable, and I urge them to fix these problems.
So you might be wondering, how does FantasySP fit in all of this? FantasySP was built from the ground up to be fast. It was built because I wanted a centralized location to keep an eye on player news and manage my fantasy teams. What FantasySP does is grab news from your favorite fantasy sites (listed above), blogs, and newspapers and presents the information to you as fast as possible. When you read something interesting, you head over to the source site to get the full scoop.
Before I show you how fast FantasySP is, let me show you how fast ESPN and CBSSports are since they are the best in the business with an unlimited budget.
1) ESPN
ESPN is one of the premier sports sites on the internet and they have recently undergone a major redesign. Perhaps they need to show us how to create a speedy website?
ESPN with Advertisements
With advertisements, ESPN has 88 HTTP requests and weighs in at 800k. The second page load has 26 HTTP requests with 62k in size. Clearly, these guys know what they are doing.
ESPN without Advertisements
Without advertisements, ESPN has 76 HTTP requests and is 674k in size. The second page load has 15 HTTP requests and 60k in size. They do a fantastic job at minimizing HTTP requests, making sure the size of the page is small, and utilizing a CDN. Even with their embedded video on the front page and interactive scoreboard, their site does a fantastic job at keeping things speedy. Perhaps the only thing they could do to further improve things is to create a CSS sprite for some of their images.
Pingdom Load Time: 2.5 seconds. (Using the best techniques with the best hardware results in super fast load times, no surprise there)
2) CBSSports
How does CBSSports compare to ESPN? Let’s find out.
CBSSports with Advertisements
CBSSports has 51 HTTP requests with 700k in total size with advertisements. Second page load has 10 HTTP requests at 48k in size. These are extremely impressive numbers considering they have advertisements shown.
CBSSports without Advertisements
Without advertisements CBSSports has 45 HTTP requests and about 700k in size again. Second page load has 6 HTTP requests with 46k in size. If they combined their javascript and created a CSS sprite then they would save even more HTTP requests and perhaps get it down to about 30 or 40.
I’ve talked a lot about load times and speed, so hopefully my site runs more like ESPN and CBSSports and less like Rotoworld. Let’s find out:
FantasySP with Advertisements
With advertisements FantasySP has 55 HTTp requests at 358k in size. On the second page load, 25 HTTP requests at 42k in size. Clearly one of the best sites in terms of HTTP requests and the smallest in size. This sounds great, but keep in mind that some of the other sites such as ESPN, CBSSports, and Rotoinfo have a richer user experience by including videos and/or interactive scoreboards. This is the main reason why I am able to keep page size so small.
FantasySP without Advertisements
Without advertisements FantasySP has 38 HTTP requests at 241k in size. On the second pageload there are 9 HTTP requests with 29k in size. When it comes to reading fantasy news, the end user might prefer FantasySP over any other fantasy site listed here based on speeds alone. In fact, new signups to FantasySP get advertisements disabled to speed up their experience and they are replaced with Player Trends. Hows that for enticing?
FantasySP utilizes modern techniques to improve the end user’s experience. I use Cloudflare to speed up DNS, block bad traffic, and a whole bunch of other nuggets. I use Amazon as my CDN. I minify and combine my javascript and css files. I also use a CSS sprite to reduce HTTP requests. For the backend I installed NewRelic so I can keep track of realtime performance and fix inefficiencies with the site. I also automatically redirect mobile visits to my uber fast mobile site. You could say I am addicted to speed, and I hope you are too. I did all of this myself, so imagine if I actually had a team of developers and a group of people that share my vision. HINT HINT!
So What’s Your Point?
The point of this whole post is to inform sites in the fantasy news space to spend the time to improve the user experience. The size of your homepage should not be near 3 meg in size with 150+ HTTP requests. Your products and news are great, and all you need are some fine tuning. Major sports sites like ESPN and CBSSports are highly optimized and well oiled machines. There is no reason why Rotowire, Rotoworld, Kffl, and RotoInfo can’t follow their lead. I am a HUGE fan of these sites and FantasySP would not exist without them, so please use what I’ve talked about in this post and improve your sites.
Questions? Comments? Glaring mistakes? Please let me know.
UPDATE: Here is a handy spreadsheet with speed data. Light green denotes best performing, dark green good performing, orange is bad performing.
UPDATE #2: As per request, I’ve taken a look at DraftSharks. They have also been added to the spreedsheet above.
DraftSharks
DraftSharks with Advertisements
DraftSharks with advertisements has 80 HTTP Requests at 1.1 meg in size. The second pageload has 6 HTTP Requests and 38k. This puts DraftSharks on par or better than kffl. Completely respectable job optimizing the site for speed. Much like all the other sites, if they combined js files and created a CSS sprite then HTTP requests would be even lower. It also wouldn’t hurt if they looked into a CDN solution if it fits in their budget.
DraftSharks without Advertisements
Without advertisements DraftSharks have 79 HTTP requests at 1.1 meg. Second page load has 5 HTTP Requests at 38k. Not much of a difference, mainly because most of their ads aren’t blocked by default by AdBlock Plus.
Pingdom Page Load: 7.2 seconds. (The pageload slowdown is likely due to Pingdom adding 80 or so EXTRA images that I did not see based on my testing.)
Making websites javascript heavy seems to be all the rage these days. Some sites like Gawker (and related sites by the company) have been heavily criticized for their javascript heavy redesign. While others like Twitter have been mostly praised for their work. Both sites are very similar in URL structure and functionality. But did you know that Google Bot can’t seem to accurately capture a screenshot for either site?
Gawker
Gawker: Google Instant Preview
Twitter
Twitter: Google Instant Preview
Before you panic and exaggerate these findings, keep in mind that Google can crawl these sites just fine. The problem is only with the screen capture and preview that is shown in the Google SERPs. I guess the real question is, who is at fault here? Googlebot for not being able to grab an accurate screenshot or the site developers for going a bit overboard on javascript?
I have given Cloudflare a try for the past week or so for FantasySP.com and want to pass along my thoughts. I saw a few reviews that were quite astonishing and decided to find out if they are accurate. I have been reading great things about it and couldn’t resist trying it out.
Will Cloudfront Increase Pages Per Visit?
Unfortunately no. The only reason you’ll see an increase in pages per visit is if you change the setting to add your Google Analytics code to every page. I decided to use this myself and look at the results:
Pages/Visit
Notice the bump in pages/visit? Almost double. That is when I added the Analytics code to every page and it completely skewed everything out of whack. Meanwhile GetClicky and awstats both showed them to be at normal levels. So clearly this Cloudflare setting should not be used and anyone who shows these type of drastic results are just lying to themselves.
I obviously don’t think Cloudflare adds this setting to artificially increase analytics stats. Although, I can’t explain why this happens, since they claim its fine to add without removing the old code.
Will Cloudflare Decrease Bounce Rate?
Probably not. If anything, it could have a slight increase in your bounce rate depending on how much bad bot/spam traffic you were getting. Although, a bad bot might even help your bounce rate because they are often times so abusive and load many pages. If Google Analytics shows a drastic decrease in bounce rate then it is due to the Cloudflare setting to add the script to every page. Have a look at my bounce rate:
Bounce Rate
Sorry folks, Cloudflare is no miracle bounce rate solution. 🙁
Will Cloudflare Increase the Speed of the Site?
Yes, even using their free service! Cloudflare has a whole list of features that show how they can decrease the time it takes for a page to load. They will cache content, similar to a CDN and provide optimized routes to your site, and block abusive traffic. If you pay for their service then you will see even better benefits (none of which I have tested).
So I guess the real question is, does Cloudflare make FantasySP faster? Yes and no. I have most of my static content already on Amazon’s Cloudfront. My DNS before Cloudflare was Amazon’s Route 53. So how much of a benefit am I really getting when some of Cloudflare’s services were already taken care of? Here is a look at Googlebot’s time it takes to crawl FantasySP.com:
Googlebot Crawl Rate
According to Googlebot, the site isn’t faster, perhaps slightly slower if anything. It could be that Amazon’s Route 53 provides equally as good routing for end users and I don’t see much of a difference. Either way, there isn’t enough evidence to make a call on this. What really matters is if there is a difference for the end user in a real browser. For my site, javascript is what is holding back the end user experience. Which brings up my next point, Rocket Loader.
Does Rocket Loader Improve Speeds?
To test Rocket Loader’s performance, I will be using NewRelic’s End User performance tracking. I have been using this for well over a month and it should show any trends with end user speeds.
As you can see, there is a large decrease in pageload time and goes from 8 seconds to 6! Awesome right? Well, unfortunately I noticed that Rocket Loader was screwing up some of the rendering of advertisements that were on the site. Thus making my site’s javascript unusable. Rocket Loader works fine with Google AdSense, but AdSense doesn’t generally hang as much because of their recent optimizations.
What I really wanted Rocket Loader to do, was to improve performance of the other ad networks that I use. These are the ones causing all the slowndown. Alas, it wasn’t meant to be. Clearly it was making a difference, but it is a bit buggy at this point. My next step is to selectively use Rocket Loader on certain javascript components and see if it will make a difference in overall load time without breaking things. Stay Tuned for that!
So Where Does This Leave Us?
After reading all of this, you might think I’d come to the conclusion that Cloudflare isn’t worth the trouble. Exactly the opposite. Cloudflare delivers on a lot of the promises of the service. Just be sure you don’t make mistakes like enabling Google Analytics on every page. That feature just doesn’t make any sense to me. (I’d love to hear your thoughts on this.)
If you aren’t using a CDN or a customized DNS service, then switching to Cloudflare should have an impact on quicker load times. Even if you do have a CDN and a customized DNS service, the fact that Cloudflare blocks bad threats and saves your server resources and prevents lots of spam. It’s also nice to see trending data for search engine bots and outbound links.
I’d love to try out Cloudflare’s pay only services to see if they improve upon speeds, but they do not offer a trial for such a thing. Perhaps down the line I will test these as well. I also think Rocket Loader will continue to improve and will truly make a difference for some users out there.
When optimizing your web application or server, the process can be a bit daunting. In fact, its usually something that is never fully checked off your to-do-list. The reason being is that you’ll never really know how your application will scale until it starts to show its growing pains. Just when you thought that you’ve addressed the performance issues, a few weeks or months go by and new ones will crop up.
Before New Relic came along, I used to cat the slow-query-log. I would address any queries that I saw took a while to execute. Then there is the MySQL Performance Tuning Primer Script. Also a great tool and will give you a good idea on how MySQL is performing as a whole. I would continue to look at these things, but the load averages kept climbing and I could not figure out what the problem was. I also couldn’t figure out which pages took the longest to load.
Enter New Relic
I actually considered throwing more hardware at the problem, since I could not figure it out through the usual means. Then New Relic came along and I decided to give it a try. Here is my site performance graph since I first started using New Relic:
Performance
You’ll notice that when it was first turned on, the database was clearly the issue. The problem was UPDATING a table that has almost 2 million rows. (Duh. Apparently, it was way more costly than I ever imagined) Incrementing the view count on one of the main tables was the root cause to 90% of my problems. I went ahead and created dedicated tables to hold these stats and load averages eventually settled to around .20 across the board. MySQL was no longer an issue. FantasySP was silky smooth, once again!
The best part about New Relic is that once you set it up, you don’t have to spend time SSH’ing your box to find performance issues. Everything you need is much easier to read, not to mention there is a performance breakdown based on each page/script. Now I can easily see the pages, ajax requests, and background tasks that take the longest to complete. All of these can be easily optimized now. Their interface isn’t perfect, but it’s improving every day.
Impact on Google Bot & Organic Search
Of course when you optimize your website you do so for the end user experience. The added plus to this is that Google Bot gets a huge benefit from it too. I’ve been reading over and over about how load time is now a ranking factor. Sure, but it’s not that big of a deal, right? Well, lets take a look at how Google Bot responded to these tweaks:
FantasySP.com | Pages Crawled Per DayFantasySP.com | Time Spend Downloadingm.FantasySP.com | Pages Crawled Per Daym.FantasySP | Time Spend Downloading
Clearly, the faster the response time, the more pages that can be crawled in a given day. The average response time for FantasySP.com is now at around 100ms. The average response time for Mobile FantasySP is at 72ms. Google Bot quickly realized that the site is performing faster and ramped up its crawl rate.
So you might be wondering, has the faster and more responsive site resulted in more search engine traffic from Google? So far, the answer is yes. By about 5 – 10% the past 7 straight days. Coincidence? I doubt it. Getting a bump in SERPs for being a considerably faster site is very real. There is no question that the easier it is for Google Bot to crawl, the better off I’ll be in the long run.
Conclusion
New Relic has just released a whole bunch of new changes such as Real User Monitoring , an improved install process, and a much more affordable pricing structure. It doesn’t matter if your a company of 15 developers or just one developer working on a side project. . . New Relic will be worth it. In fact, for smaller sites you should be able to take advantage of New Relic’s 14 day professional trial and be done optimizing by the time your trial is finished. However, I’ve been addicted to the stats and will eventually subscribe to their “Standard Package”.
New Relic essentially takes the guesswork out of optimization. Now you can easily track down costly plugins in WordPress and even compare performance numbers from week to week. This is by far the best tool you can use to make your site or app run faster. Spend the time and install it, or submit a support ticket and have your host do it.
I can go on and on about New Relic and you will probably get a bit lost in the data for a few days. So go ahead and give it a try.
