Category: php

Categories
FantasySP optimization php

Behind the Scenes of the FantasySP Redesign

I have been working on the FantasySP redesign for over a year before it officially launched on May 20th 2018.  The redesign was something I first thought of doing over 3 years ago, but I kept putting it off due to working on other features and the sheer size of the project.

I knew it could not be delayed any longer after the announcement by Google for their mobile-first index.  I thought I could have it up before this all went into effect, but then the news broke that their mobile-first index was going live in March.

Prior to the redesign, I had a mobile site and a desktop site.  Each had its own codebase, both front-end and backend.  Yes, you heard right. Not only that, but the mobile site didn’t have ALL of the features of the desktop site and some were stripped down versions of the desktop site.

The mobile site was so old, that it used tables because that is what you still used when mobile sites were gaining popularity for maximum compatibility. 

The initial codebase for the mobile site was pre Bootstrap and pre front-end frameworks. It was back when Digg was the shit, so I decided to use their design for mobile FantasySP.  It was back when responsive layouts didn’t exist yet and a separate mobile site was the only real option.

I also knew the task at hand was monumental and I knew I needed help for two reasons.  First, I knew a professional designer would create something better than I could.  Second, I knew this would speed up the process.

I contacted Ryan Scherf, and we narrowed down a framework (Bootstrap), a new color scheme, and started to tackle a few pages.  Ryan and I worked back and forth from about January of 2017 to May of 2017.

To call this simply a ‘redesign’ is an injustice. It’s way more than just a facelift since every piece of code is being touched behind the scenes. This is basically a relaunch of FantasySP, and I’ll get into more about that in a bit.

Learning to Ride Again

The thing about web development is that you are always evolving and constantly learning new technology.  Switching to Bootstrap v4 meant ditching floats and moving to flexbox.

Flexbox in itself is a huge change and a huge learning curve, and I still need to use a cheat sheet but I am getting better with it every day. Basically, I’ll be using flex instead of using floats for layout positioning,

Not only that, but Bootstrap itself has its own laws and ways of doing things that I had to learn.  (Yes, it’s true, I never used Bootstrap before this) Needless to say, having a designer come up with the Bootstrap theme was monumental into getting started off on the right foot.

On top of Flexbox and Bootstrap I needed to learn the modern way to develop on the front-end which meant adding Gulp and Bower into the mix. (Though do not use Bower if you are starting out now, use Yarn instead.)

Ryan pointed me in the right direction so I could get started and get everything working.

Keep in mind that I didn’t even start the real work yet. This is the work before the work. I had to learn 3 brand new things that were pretty damn challenging before things even got off the ground.

My goal was to get a variety of pages designed to get a base foundation to work off of, and I would spend the time to fill in the gaps and create pages based off of those core pages.

Slowly but surely I was getting the hang of the new workflow, and I could start taking Ryan’s designs and create new pages and tweak the layouts he had already given me.

Starting Over

The redesign meant that I was starting over. Everything was going to be re-written from the ground up. If I am doing this redesign I am going to do it justice.

The new FantasySP will be fast and customizable. It will be lean (as best as I could). And it was going to be as future proof as I could possibly make it.

This meant all of my javascript functions had to be looked at and most had to be recoded.  This meant basic things like autocomplete, pop-up boxes, front-end data checks, table sorting all had to be researched. Some of it Bootstrap would handle, but many of them had to be included as extra javascript libraries.

From now on I would have one file for CSS and one file for JS and that was it.

It also meant I had to redesign the billing page and implement the newest version of Stripe on both the frontend and the backend. I was using code from 2011 and it was starting to give me problems on occasion.

I wasn’t going to try and reinvent the wheel. Honestly, I don’t have time to relearn EVERYTHING. I’m not switching to MEAN and ditching my stack but I will modernize it. (If you are looking for benchmarks, you’ll find them at the end of this article, but I’m not spoiling it here.)

And yes, I would still be using jQuery. I’d still be using nginx. I’d still be using PHP 7.x…. commonly referred to as the LEMP stack.  You may not like the stack, and hate PHP but that’s because your an idiot. 🙂

Speaking of PHP, all of my code had to be looked at and either re-written or tweaked in order to fit into a unified codebase for a responsive layout.

Like I said, this task was daunting.

The Actual Design

Now that some of the geeky stuff is out of the way, let’s actually talk about the design. There are a few things I knew I wanted the new design to have:

  1. A new color because red reminds users of ESPN
  2. A better top navigation
  3. A dedicated section for user navigation
  4. awesome player popups
  5. reimagined fantasy tools
  6. obviously be responsive

Once we figured out the colors and top navigation, the real work on the design could start.  My wife was the one to narrow down that the color orange would work for the site, and Ryan picked the perfect shade to work with.

The day I got really excited about this redesign was when work began on the reimagined start/sit tool.  One of the first mockups looked like this.

Start/Sit Take One

 

A good start, but I wanted the top section to pop colorwise. It needed something to make it stick out that I thought was missing.

Based off that feedback, he completely nailed the revamped design and is something I would have never come up with on my own.

Start/Sit Take 2
Start/Sit Take 2

One last example I’ll show are the early mockups of the Player Page and Player Popups.

I needed a much better way to organize the chaos I had going on with the player pages, and I am pretty sure the first thing I got back was almost exactly what I used in my final design.

I didn’t provide much insight into what I was looking for other than please organize it all, and allow for a bunch of different boxes to the right side that I could work with.

player page

As soon as I saw it, I could not wait until this thing went live. It looked light years better than what was already on the site

The first player popup design that we started with was great right off the bat, but much like the start/sit tool, I wanted a bit more color to the header.

Player Pop-up (1)

Which then turned into this:

Player Popup 2

The live popup as of today looks like this:

Live Popup
Live Popup

Once Ryan’s design job was done, it was time to get a good sampling of live pages to turn into real HTML/CSS.  I had no idea how to formulate a bootstrap theme, so I really needed to see how it was supposed to be done.

I would take what he gave me, which was the core styling of FantasySP and a few pages and then come up with designs for the rest of them.  I think one of the first designs I converted into CSS/HTML was the player popup and boy oh boy did that take me a long time.

This was when I had to really buckle down and learn the ins and outs of Bootstrap and flex positioning.  Needless to say, it was not fun and I felt lost for a few weeks. I still kinda get lost with flex positioning because it is just so different.

Once the rest of the designs were turned into CSS I finally had enough material to create the beta subdomain and start turning these HTML/CSS pages into actual working pages with live data.

It turns out, that day happened to be May 17th 2017. As you can see, all I had was a hello world.

At this point I would be starting over in terms of all of the backend logic for PHP and would start from scratch. In some cases, I would take my old PHP code and convert it to work with the new layouts.  It was a major milestone, but my work was far from over.

From this point, it was still another year before I officially launched the new FantasySP.

In June of 2017, I finally had enough of the beta site up and running to open it up for users to start trying.  From that point on, I tackled each page and tool and kept grinding away.

Redesigning the Fantasy Assistant, Trade Analyzer, Draft Genius, billing, etc were extremely time consuming and very challenging but totally worth it once I was finished.

The Fantasy Assistant itself probably took me at least a month on its own because of the sheer size of it, plus for the first time it had to work on mobile.

Time for Stats

Like I mentioned earlier, the redesign was supposed to be much faster than the original site. Paying members will get an experience even faster because no advertisements are shown.

The numbers do not lie.

Front End Load Times

Front end load times  are measured based on what percentage of load times are considered Satisfied, Tolerating, and Frustrating. This data is taken from New Relic.

ORIGINAL: Satisfied: 53%, Tolerating: 40%, and Frustrating: 7%

NEW DESIGN: Satisfied: 80%, Tolerating: 18%, Frustrating: 3%

According to Google Analytics, average page load times went from a range of 5 – 15 seconds, all the way down to a much more narrow range of 4.5 to 6 seconds.

Wrap-Up

I feel like this post is getting too long so I am going to end it here. Hopefully this post helped you visualize the long journey that I had to navigate. Of course, I am nowhere near done in terms of optimizations.

I’d love to go into optimization techniques I’ve used on this redesign but most of them are widely known these days for frontend development… it’s mostly a matter of just implementing them.

I still need to upgrade to PHP 7.3 and upgrade nginx and enable the pagespeed module but there is plenty of time to get to those over the next few weeks.  I will make a new blog post detailing those changes and how it affected performance and the people behind the scenes that helped me.

For now though, I am extremely happy with the speed, the design, and the response from my users. Do yourself a favor and check out the brand new FantasySP yourself.

After roughly a year and a half since I started the redesign, it went live without any downtime and just a few minor bugs that needed to be fixed.

Now that this redesign is finished, I can implement features much faster since I am working with one unified codebase.  Thanks for reading!

Categories
php rant

RE: PHP Sucks

An article hit hackernews today with the title “PHP Sucks“.  The reasons for it sucking are as follows:

  • There is a lot of old code out there that’s shitty
  • There is a bad stigma for PHP developers
  • You may get paid less as a PHP developer compared to other languages (Proof?)

I’m not sure if you can get paid less to be a PHP developer, but it’s important to be a full stack engineer and learn as much as possible about the stack you are on.

One of my favorite quotes from the article

Sometimes people are more straightforward and will just respond with “Oh, I’m sorry about that”. Recently I talked to a CEO who more carefully said “Ah, that’s pretty old school right?”. Developers who more-so live in the Java-dominant corporate bubble will likely silently dismiss me as a incompetent programmer.

I find this to be pretty funny.  If you care about what other people think about the language you choose then you’re worried about the wrong stuff.

If you told me that I should switch away from the LAMP stack because it’s slow, then I am all for that.  Speed would be my #1 reason to move away from PHP.

You want to swap out Apache for nginx?  Sure thing. Why? Because nginx is much faster and uses less RAM.  (Though it’s not a drop in replacement in most instances)

If you want to swap out MySQL for MariaDB then I would not be opposed since most say that it’s 100% compatible and slightly faster.

The problem, or lackthereof, is that PHP is among the fastest languages out there. If you switch from PHP to Rails then you would be bummed to find out how slow Rails is compared to PHP7. Cool, hip programming language, but much slower. Hmm, decisions decisions.

Do you want a fast app or do you want to worry about what other people think?

If you care more about what others think than making sure your application runs as fast as possible, then I feel sorry for you. If I cared what people say I’d be working for some 9 to 5 job as just-another-developer being underappreciated and underpaid and listening to a CEO who almost knew what the fuck he was talking about.

I run the LAMP stack for FantasySP and give zero fucks. When I finally migrate to PHP7, then I’ll hopefully have around double the speed. My apps average response time is 120ms. I reckon I can get that to around 60-90 ms with PHP7. What’s not to like?

Categories
apache case-study php

Apache mod_fcgi (FastCGI) vs DSO (mod_php)

My experiment with FastCGI has come to an end.

You may remember my last post about upgrading my server to the latest Apache, PHP 5.4.x, and switching to FastCGI.

The biggest issue with this new setup was the fact that APC was not shared amongst all the PHP processes, which really made things more difficult.  Did I really want to recode my caching solution?  An alternative would be to use Redis, but that is assuming FastCGI was faster than DSO and worth using.

Caching aside, FastCGI is slower than DSO based on testing in a production environment.  In some high traffic situations FastCGI performed significantly slower.

On the other hand, FastCGI was great with resource management compared to DSO.  It managed to keep memory usage low during both real world testing and benchmarks that I threw at it.

In the end, I guess it depends on what you are looking for.  Speed or better resource management?

 

 

 

Categories
apache php wamp

Setting up 64bit WAMP Server under Windows 8 Using Latest Builds

Setting up WAMP Server can be tricky depending on your configuration.  This post is going to walk you through some of the tougher steps to get your local machine up and running so you can get back to development.

Keep in mind this is not a newbies guide to WAMP.  Look elsewhere if you can’t get the basics working.  What we are trying to tackle here are more advanced problems that you may experience.

A couple of snags I hit were WAMP running slowly, APC refusing to install, and getting cURL working.

At the time of writing this, the current bleeding edge WAMP Bundle has the following specs:

Apache 2.4.2 – Mysql 5.5.24 – PHP 5.4.3 XDebug 2.1.2 XDC 1.5 PhpMyadmin 3.4.10.1 SQLBuddy 1.3.3 webGrind 1.0

In my particular case, I opted to run 64bit code but I do not see any benefits other than making things more difficult. You may want to stick with 32bit.

Initial Steps

Installing WAMP at the start is easy and I am going to assume you can handle clicking next on the install screen and get the basics working.  Since you are using Windows 7/8 it is important that you install the latest runtimes for your setup:

http://www.microsoft.com/download/en/details.aspx?id=8328 x86 (32-bit)

http://www.microsoft.com/download/en/details.aspx?id=13523 x64 (64-bit)

It is also extremely important to make sure IIS is not running on your computer.  To do that go to: “Turn Windows Feature On and Off” and disable IIS, otherwise it will take up Port 80 and waste resources.

After a few reboots you should be at the point where you can configure PHP/Apache and add additional extensions.

Apache Configuration

This is where the fun begins.  To get to the Apache configuration file Left Click on WAMP Server systray and browse to Apache -> httpd.conf.  I am not going to go through the entire process, but will add some useful code snippets.

If you have any subdomains then this is the time to set those up and it looks something like this:

##########################
NameVirtualHost *:80

<VirtualHost *:80>
DocumentRoot “G:\wamp\www\FantasySP”
ServerName localhost
</VirtualHost>

<VirtualHost *:80>
DocumentRoot “G:\wamp\www\reddit-ama”
ServerName rlocalhost
</VirtualHost>

<VirtualHost *:80>
DocumentRoot “G:\wamp\www\FantasySP\m”
ServerName m.localhost
ErrorLog logs/subdomain_error.log
</VirtualHost>

As you can see, I have two separate projects FantasySP and Top IAmA, one running on localhost and the other on rlocalhost.  Most of you will have your DocumentRoot as \www\ and can probably skip this bit.  (Also, don’t forget to update your Windows HOSTS file.)

One of the snags you may encounter at this point might be getting Apache to recognize .htaccess files.  I modified my rules to look like this:

<Files “.ht*”>
Order allow,deny
Allow from all
Satisfy All
</Files>

Another snag might be the fact you can’t access the server at all.  By default it may “Order Deny,Allow”, then specify a specific IP that has access.  You can either update your localhost IP or just use “Allow from all”.

By default Apache has a few modules disabled that should be enabled.  Left Click on WAMP Server systray and browse to Apache -> Apache Modules.  You will want to enable deflate_module and rewrite_module.  Without the rewrite module your clean URLs won’t load.  Without deflate enabling gzip compression will cause a configuration error.  If you have expires rules in your htaccess file then include the expires_module.

If Apache is still having errors then chances are you need another missing module or you screwed up your httpd.conf file.  Make sure you back this up before you start fiddling around.

Configuring PHP

Now that Apache is up to snuff, it’s time to add missing PHP extensions and configurations.

Immediately you may want to enable the PHP short open tag.  Left Click LAMP Systray -> PHP -> PHP Settings.  Otherwise things like <? and ?> will cause errors.

(However, as septor in the comments pointed out, the short open tags will affect XML documents.)

Next up is making sure the cURL is working (assuming you want this).  In my case, cURL would not work with the included extension so I had to find an updated version that would work on my PC.

Grab the latest cURL extension that fits your OS.  Overwrite php_curl.dll in your WAMP folder similar to this: C:\wamp\bin\php\php5.4.3\ext

Restart Apache and run <? phpinfo(); ?> to see if cURL shows up this time.  If it doesn’t then try a different DLL file until it does show up.

The biggest pain in the ass will be to get APC working.  After a bit of frustration I finally figured out the easiest way to get it working.

In your php.ini file add the following line at the end to first DISABLE APC.  apc.enabled=0.  It is important APC is disabled first because it will make debugging a lot easier.

Find the correct php_apc.dll file that will work for your setup. You can find that on this handy page full of modules.

You can either try each one and restart apache if you’re lazy.  Or you can look at your phpinfo output to see what version you need.  TS stands for thread safe.  VC9 stands for the runtime library it was compiled with. (Remember when you installed those runtimes a bit earlier?).

Copy the .dll into the ext folder and restart apache.  If you picked the wrong one then Apache will crash.  Otherwise the systray icon will turn Green.  Once it appears in phpinfo output then you’ve found the right one.

Go back to your php.ini configuration and change apc.enabled=1.  There is a good chance Apache will crash once APC is enabled.  I think this has to do with the fact Serialization Support shows as Disabled (or broken).  That value should show up as “php”.

After dealing with this problem for a day or so I read that you had to copy the php_apc.dll file into one of your Windows System folders.

If you are running 64bit WAMP then copy php_apc.dll into C:\Windows\SysWOW64. For 32bit copy into C:\Windows\System32.

Only do that last step if Apache errors when APC is enabled.  I have no idea why these files need to be copied there, so if you know the reason then feel free to post in the comments.

At this point you can specify your own advanced APC configurations if you want.

Fixing Slow Performance

Right after I started testing FantasySP on localhost I quickly realized that it was much slower under this configuration than my last configuration.

Last time I was running PHP 5.3 with Apache 2.2.x.  So either the new Apache was slow or PHP.

As of this writing WAMP comes with Apache 2.4.2.  The latest version right now is Apache 2.4.4.   Apparently Apache 2.4.4 uses VC10 runtimes instead of VC9 to enhance performance under Windows.

I figured it might be best to update Apache to see if that would improve the speeds.  I got the latest Apache Build and created a new directory in wamp/bin/apache/apache2.4.4.

I coped over wampserver.conf from the apache2.4.2 directory and my httpd config file.  I also re-enabled my PHP extensions.  You can now specify which version of Apache you would like by Left Clicking the WAMP systray -> Apache -> Version.

Once WAMP Restarted using the newer version of Apache it was much faster. Though there was still something slowing it down.

I watched task manager in Windows 8 to see which applications were using a lot of CPU.  (A great reason to upgrade to Windows 7+ is the improved task manager).  Anyways, I noticed that Windows Defender was using a lot of CPU when httpd.exe or mysqld.exe would run.

As it turns out, real-time spyware protection was slowing things down!  Open up Windows Defender (or whatever you use for spyware/virus protection) and exclude real-time protection for httpd.exe and mysqld.exe.

It also makes sense to disable Windows Indexer from indexing C:/Wamp/*.

Final Thoughts

Phew.  After all of those configurations and changes, localhost runs as fast as my previous setup under Apache 2.2 and PHP 5.3.  As long as you take the time to set things up properly then things should run smooth under Windows 7 or Windows 8.

Even though you will experience growing pains by updating your WAMP environment I highly suggest taking a weekend to do so to make sure your applications run well on an updated stack.

Hopefully this post has proved useful to easing your pain during the WAMP setup process.  I have a strong feeling that I will be consulting this very blog post 3 years down the road when I have to do it all over again…

Categories
apache cpanel how-to php

APC Uptime Restarts Every 2 Hours

Do you happen to have a really bad uptime for PHP’s APC, and it seems to randomly restart after 2 hours or so?  Well, chances are that you are running cPanel/WHM on this server as well?

The problem is that you need to enable Piped Loggin for Apache.   This can be found under Apache Configuration in WHM.

It says:

Configure Apache to use a single log target for all virtual host access and bandwidth logs. The combined logs will be piped to a helper application where they can be split based upon domain. This option will reduce the number of log files Apache manages freeing up system resources. Piped logging is recommended for systems with a large number of domains. By default this feature is disabled, and Apache will create distinct log files for each virtual host entry.

Once you do this, your log files will be processed (it defaults to every 2 hours), without APC restarting.  For even more information, head over to this post on the cPanel forums.

This problem is obviously very frustrating and it took me quite a while to dig up the solution.  Hopefully this helps others out there.

Categories
guide how-to php

FantasySP, Building a Better Upgrade Flow

A few weeks ago I came to the realization that my current implementation of upgrading and billing users on FantasySP using Paypal was pretty terrible.

It was a clunky process where the user left my site to go to paypal, then they could sign in and/or input their credit card information.  After that, they get directed back to my site to a “success” page.

The user was unable to upgrade from a monthly subscription to a yearly subscription, if he so chose.  I never implemented automatic membership downgrades/upgrades because I thought I would be leaving paypal sooner rather than later.  To top it off, the design of the page was just plain bad.

Talk about a mess.  This is what my upgrade/billing page used to look like:

 

Old Billing Page

Simplicity is Key

When it comes to designing and implementing the new upgrade flow, I knew I wanted something extremely simple and easy to use:

  1. User should stay on the fantasysp.com domain during the process (SSL would have to be installed)
  2. Only 2 subscription choices to pick from with a suggested audience.
  3. Able to upgrade from monthly to yearly with a prorated cost.
  4. Able to apply a coupon code during selection BEFORE he checks out.
  5. Easily see the breakdown of features.

For me, Stripe seemed like the best option.  There are countless posts out there detailing what Stripe is all about and why its cool.  During the checkout process, stripe uses a javascript library that handles the form submission and processing of credit card information.  If it all checks out, then the form is submitted to my server with an authenticated token that stripe recognizes to perform the transaction, create a new customer, or subscribe them to a plan.  Therefore no credit card information is actually handled by my server.

Design the Page

Instead of hiring a professional designer, I decided to take a crack at this one myself.  If what I made looked like shit then I would bite the bullet and hire a pro.  I would never say that I am a designer, but I can pretend to be one.   I looked at quite a few checkout pages ranging from 37signals to Old Navy.  I skipped the Photoshop process of a mockup and went straight to designing in HTML/CSS because I knew what I wanted in my head:

User Upgrade Options

The boxes look pretty much like Monopoly cards, right? Instead of smashing everything together, I wanted to make sure I utilized the entire width of the website.  The user is presented with a CLEAR option of free, monthly, or yearly plus.   Underneath the price suggests who would benefit best from the plan.  Monthly is for the seasonal fantasy player, whereas Yearly is for hardcore players, and Standard is for the casual player.  When mousing over each box, the background turns yellow.

You may notice that I decided to use different fonts (via Google Fonts) because I think it makes it appear and feel more unique.  Arial gets boring real quick.  I also wanted to make sure that the page relied solely on CSS for styling.  Everything you see there are some fairly simple and common CSS techniques to add a bit of flare to each box.

The coupon code is shown just below the choices WITHOUT a button.  My site will analyze the text as the user types or pastes the coupon into the box.  In order to proceed to the next page, the user must click a box.  If a valid/expired coupon code is detected then it gives a message like so:

Coupon Code Entered

But of course, the user needs to see the full breakdown of the features.  The user also needs to see what the site would look like as a Yearly Plus subscriber, which means the top banner of the site will show Player Trends instead of banner ads:

More of the Page

The user can mouse over the bottom 4 choices and see more information about each via tooltips powered by qTips.  Now lets move onto the actual form to submit the subscription order:

 

Checkout Form

Thanks to Stripe, I don’t need their billing address or even full name to process an order.  We want the bare minimum here so the user can quickly move on.  Next to the plan is an option to change their plan.  There is also an option to “Go Back” incase the user changes their mind. The “Submit Payment” button is BIG and BOLD because the user should be able to quickly find the “Submit” button.

Again, clicking the “Submit Payment” button sends the information to Stripe and then responds back with a token if valid.  Once that happens, then communication between Stripe occurs and the user is upgraded to their selected plan.  If all goes well, the form is finally submitted and the user is presented with a Success page.

But Wait, There’s More

You thought that was it, huh?  That is the basic checkout process, but the user needs to see an email invoice to ensure they have something for their records.  Sounds like something else I have to design that also needs to look just as good and be just as simple.  Luckily I recently redesigned the email newsletter that goes out, so I used that template for the invoice:

Email Invoice

Looks like I should be nearly done with the design of the billing process, but there are a few more possibilites to take into account.  What about Cancelling memberships?  What about Upgrading from a Monthly subscription to a Yearly Plus?

Enrolled in Monthly
Clicked to Upgrade

As you can see, choosing to upgrade a subscription prompts the user with a new box via jQuery UI.  After clicking yes, the user is shown the price and is sent a new prorated invoice. Similarly, if the user wishes to cancel, then a similar box would pop up.

Wrap Up

Revamping a billing page takes a lot of planning, testing, and fine tuning.  Just the design implementation alone took about 2 weeks with lots of tweaking.  Lots of frontend technologies are used including jQuery, jQuery UI, Google Fonts, Stripe, and qTip.  There are still a few things that need to be tweaked on the frontend and backend, but overall the new billing is exactly what I envisioned and is light-years better than what I had.

Hopefully this post will help others who are thinking about revamping their upgrade flow. Nearly 3 weeks of hard-work launched a few days ago and it was great to see a few users already go through the process with no problems.  Merry Christmas to me.

Categories
optimization php

Fix the WordPress SQL_CALC_FOUND_ROWS Bug

Anyone who has a WordPress blog with a lot of posts will eventually encounter an extremely slow query. I refer to this as the SQL_CALC_FOUND_ROWS Bug. If you have slow-query-log enabled then a query similar to this might have shown up before:

SELECT SQL_CALC_FOUND_ROWS wp_posts.* FROM wp_posts WHERE 1=1 AND wp_posts.ID NOT IN (44682, 44657, 44630) AND wp_posts.post_type = 'post' AND (wp_posts.post_status = 'publish') GROUP BY wp_posts.ID ORDER BY wp_posts.post_date DESC LIMIT 24580, 5

Just how detremential is this to your blogs performance?  Well thanks to newrelic, I can show you:

SQL_CALC_FOUND_ROWS Bug

Unfortunately I don’t know why WordPress runs this query.  What I do know is that it apparently only shows up on index.php.  What you probably care about is how to fix this problem.   I’ve located a possible workaround thanks to this open ticket at wordpress.org.  The diff log on the changes are listed as well.  I went ahead and applied these code changes to wp-includes/query.php.  The fixed query.php can be found here.

How are the results so far?  Inconclusive.  I just applied this patch and nothing broke so far, which is always a plus.  I suggest you give it a try and see how your blog responds in a development environment.  If I STILL spot slowdowns in the revised query, then I will update this post and let you know.

Some of you might be asking, does this affect your version of WordPress? The answer is yes.  I am running WordPress 3.2.1.

Please do post your thoughts, concerns, or comments to help others out.

Categories
optimization php

My Review of New Relic, with some Performance Numbers

When optimizing your web application or server, the process can be a bit daunting.  In fact, its usually something that is never fully checked off your to-do-list.  The reason being is that you’ll never really know how your application will scale until it starts to show its growing pains.  Just when you thought that you’ve addressed the performance issues, a few weeks or months go by and new ones will crop up.

Before New Relic came along, I used to cat the slow-query-log.  I would address any queries that I saw took a while to execute.   Then there is the MySQL Performance Tuning Primer Script.  Also a great tool and will give you a good idea on how MySQL is performing as a whole.  I would continue to look at these things, but the load averages kept climbing and I could not figure out what the problem was.  I also couldn’t figure out which pages took the longest to load.

Enter New Relic

I actually considered throwing more hardware at the problem, since I could not figure it out through the usual means.  Then New Relic came along and I decided to give it a try. Here is my site performance graph since I first started using New Relic:

Performance

You’ll notice that when it was first turned on, the database was clearly the issue. The problem was UPDATING a table that has almost 2 million rows. (Duh.  Apparently, it was way more costly than I ever imagined)  Incrementing the view count on one of the main tables was the root cause to 90% of my problems.  I went ahead and created dedicated tables to hold these stats  and load averages eventually settled to around .20 across the board.  MySQL was no longer an issue.  FantasySP was silky smooth, once again!

The best part about New Relic is that once you set it up, you don’t have to spend time SSH’ing your box to find performance issues.  Everything you need is much easier to read, not to mention there is a performance breakdown based on each page/script.  Now I can easily see the pages, ajax requests, and background tasks that take the longest to complete.  All of these can be easily optimized now.  Their interface isn’t perfect, but it’s improving every day.

Impact on Google Bot & Organic Search

Of course when you optimize your website you do so for the end user experience.  The added plus to this is that Google Bot gets a huge benefit from it too.  I’ve been reading over and over about how load time is now a ranking factor.  Sure, but it’s not that big of a deal, right?  Well, lets take a look at how Google Bot responded to these tweaks:

FantasySP.com | Pages Crawled Per Day
FantasySP.com | Time Spend Downloading
m.FantasySP.com | Pages Crawled Per Day
m.FantasySP | Time Spend Downloading

Clearly, the faster the response time, the more pages that can be crawled in a given day.  The average response time for FantasySP.com is now at around 100ms.  The average response time for Mobile FantasySP is at 72ms.  Google Bot quickly realized that the site is performing faster and ramped up its crawl rate.

So you might be wondering, has the faster and more responsive site resulted in more search engine traffic from Google?  So far, the answer is yes.  By about 5 – 10% the past 7 straight days. Coincidence?  I doubt it.  Getting a bump in SERPs for being a considerably faster site is very real.  There is no question that the easier it is for Google Bot to crawl, the better off I’ll be in the long run.

Conclusion

New Relic has just released a whole bunch of new changes such as Real User Monitoring , an improved install process, and a much more affordable pricing structure.  It doesn’t matter if your a company of 15 developers or just one developer working on a side project. . . New Relic will be worth it.  In fact, for smaller sites you should be able to take advantage of New Relic’s 14 day professional trial and be done optimizing by the time your trial is finished.  However, I’ve been addicted to the stats and will eventually subscribe to their “Standard Package”.

New Relic essentially takes the guesswork out of optimization.  Now you can easily track down costly plugins in WordPress and even compare performance numbers from week to week.  This is by far the best tool you can use to make your site or app run faster.  Spend the time and install it, or submit a support ticket and have your host do it.

I can go on and on about New Relic and you will probably get a bit lost in the data for a few days.  So go ahead and give it a try.

Categories
guide how-to php

Web Developers, You’re Not Alone

Web developers who plan to work alone, or are already doing so have a lot of obstacles to overcome.  One of which is being isolated and having no like-minded professionals to turn to for advice and feedback.   I know this one all too well and have been working alone  for the past 5+ years on various projects.  My current project, FantasySP, has been extremely challenging and rewarding at the same time.  Reinventing how people manage their fantasy sports teams can be quite a challenge.

Smashing magazine wrote a great article detailing why you should not be working alone.  However, some of us have no choice but to work alone. It could be that you are doing a solo project that no one believes in, or you simply do not have enough funding to get anyone else on board.  Fear not, this article will show you where to go to get some helpful feedback.

  1. Stack Overflow
    This is my personal favorite for websites to turn to when you are stuck in a bind.  As a developer on your own you will eventually run into programming problems that you need to get some outside advice on.  Whether it be a MySQL scalability issue or how to parse JSON in PHP.  You are free to post about whatever problem you are having, even newbie questions are often given great responses.  Questions are mostly geared towards programing, but can occasionally touch on server configuration or just plain old HTML questions.

    Best part about the site?  The community is friendly, extremely knowledgeable, and eager to help.  I’ve asked questions on that site that directly and indirectly led to answers.  I can’t say enough good things about this site and suggest anyone who is involved in coding to sign up and participate.   You’ll feel compelled to help others and try to be the first one to get selected as “Best Answer” to rack up points.

  2. Forrst
    Forrst is a site that can be used for networking with like-minded individuals and provide a spot to ask a wide range of questions that usually don’t pop up on Stack Overflow.  This site isn’t specifically limited to questions.  You have the option to post code snippets, screenshots of projects, to gain feedback.  You can even post useful links that you think can be beneficial to others.  Your posts can be private or made public, it’s completely up to you.

    If you are looking for feedback on a blog layout, design mockups, or coding then Forrst can be a great spot to check out.  I do not have much experience here, but from what I’ve seen thus far the community seems helpful and eager to help.  Getting started on Forrst you may feel a bit isolated, since it has a social aspect to it.  The site requires you to apply for it, but after waiting a week or so then you will recieve your invite code in your mail.  However, if you are helpful and engaged in the site then you should gain quite a few followers and get the feedback your looking for.

  3. Dribbble
    Dribbble’s sole focus is to post your designs and try to gain feedback from it’s userbase.  The site is invite only and is fairly difficult to become a member of without getting lucky or knowing someone already on the site.  I am not a member of Dribbble, but a designer friend is and told me the site is less about feedback and more about showing off your awesomeness.  Egos abound.

    So should Dribbble be on this list of helpful sites?  Yes and no. It’s still a great spot to post your designs and get feedback, but if you aren’t up to par then expect to hear about it.  That isn’t necessarily a bad thing if you are a pro, but amateurs beware.

  4. Web Hosting Talk
    Here we have a throwback site that you probably heard of.  Unlike the other sites mention, this is a forum and is geared towards web hosting questions.  Being a developer on your own means making web hosting decisions as well, so this site is a must to learn which places to avoid.

    It may not look the best, but the forum is loaded with people who have lots of experience with multiple web hosts and it even has employees from dozens of hosting companies.  Ask your questions about domain registrars, VPS’s, cPanel, or Plesk and you’ll get lots of great feedback

  5. Webmasterworld

    Another blast from the past, webmasterworld is the place to go for questions related to your server and SEO.  This is a fantastic place to go to learn about harmful webbots that should be blocked and the latest rumors/news about search engines.  Users on this site are anonymous, and for good reason.

    This site is about  asking questions that you probably wouldn’t ask elsewhere.  For example, if you plan to partake in questionable practices such as cloaking content for Googlebot.  The community is helpful and there is a large mix of newbies and experts on the topic of SEO.

    This list is far from perfect and I have a feeling I’ve missed quite a few useful sites.  Be sure and leave comments to other helpful websites to go to for feedback.

Categories
jquery php

jQuery vs Prototype, a newbies perspective

When I first started to dabble with javascript and ajax, Prototype was the only game in town that was making any headlines. So of course I used prototype with script.aculo.us .  I was able to make some simple stuff, but my coding with Prototype wasn’t very elegant and it certainly wasn’t the best way to code things.  But you know what, I’m a javascript newbie and  it worked!

Over the last few years it seems like Prototype development has slowed down a lot, while jQuery is making huge headway.  Every tutorial I see is using jQuery.  So I FINALLY decided to buy a book and learn it.  They are vastly different to say the least.  My prototype code had an awful lot of onclick=”” kinda stuff inside the HTML, while jQuery let’s your code appear more transparent and natural.  I love being able to pass my own variables using html attributes.

I went through a lot of WTF, why isn’t this working moments during my first attempts at coding using jQuery techniques.  However, overall I love jQuery.  Now my code is optimized, cleaner, and I feel as though I am not limited in what I want to do.  If I can dream it, I can code it with jQuery.  Not a small feat for someone like myself.

For example, with Prototype I simply didn’t know how to show a loading gif when an ajax command was in use.  It sounds simple and a lot of you are probably thinking what an idiot.  I turn to jQuery and my first attempt I find that it’s so simple to achieve.  Not only that but I can easily manipulate form values, check their validity,and pass them along with no problem at all.

To learn jQuery I completely rewrote all of my javascript code for FantasySP.  It took me about 2 LONG weeks, but now I feel as though I can code just about anything I want.  My code is a mere 7kb in size now, compared to 49kb in Prototype.  I am actually making reusable functions now and you can too.

My only quibble with jQuery is that coding an advanced autocomplete function is a huge pain in the ass.  Prototype let me customize it from the backend (PHP), whereas jQuery forced me to use JSON and adding extra javascript code.

So for those of you who haven’t used a framework before, or are still using Prototype, give jQuery a try.  You won’t be disappointed.