Magento image not loading when using cron, only getting placeholder

I have a module which sends out an email via Mailchimp with a scheduled task for Magento’s internal cron and in doing so looks up the product image.

Mage::helper('catalog/image')->init($product, 'small_image')->resize(150));

The problem being only the placeholder was being returned when the cron job was executed. To confuse matters when I ran the script in testing from the browser it would return the correct image.

This is a nasty problem at first I thought this might be the answer Sangay details there that if your config does not specify a M,GB etc that it will return the wrong calculation, but then why the different results depending on how I ran the script?

To get to the bottom of this I ran what is being run in Mage_Catalog_Helper_Image

$model = Mage::getModel('catalog/product_image');
$model->setBaseFile($this->product->getData('small_image'))->resize(600,600);
$url = $model->saveFile()->getUrl();

PHP Fatal error:  Uncaught exception 'Varien_Exception' with message 'Memory limit has been reached.' in /lib/Varien/Image/Adapter/Gd2.php:58

Returning this shows the actual exception I did not get any Exception in the Mage exception log, and not in the host’s error log. On my part a server configuration oversight as the CLI php.ini did not have the error log set up. In any case that’s the error.

Because the value returned from _convertToByte function in Mage_Catalog_Helper_Image is -1 as the CLI php.ini sets the memory_limit to -1 (unlimited).

This could be an oversight by Magento or a design feature not to run this task on unlimited memory, personally I think it’s a bug. I modified the function to the following to set a 2G limit in this case.

   if (stripos($memoryValue, 'M') !== false) {
            return (int)$memoryValue * 1024 * 1024;
        }
        elseif (stripos($memoryValue, 'KB') !== false) {
            return (int)$memoryValue * 1024;
        }
        elseif ($memoryValue == "-1") {
            return '2147483648'; //2G
        }

Clearing expired magento shopping/sales rules programatically

Here’s a nice cleanup script I use to clear out expired shopping cart rules and catalog price rules and apply those changes. It’s one of those annoying things that you would think there would be a button for.

I’ve set the expiry check to the previous day and date comparison using strtotime you might want to adjust that, and use php’s dateTime library for a more thorough comparison as this relies on your server’s timezone configuration matching magento’s and being set up correctly.

If your wondering what setState does that’s to remove the message in magento admin about un-applied rules.

require '../app/Mage.php';

umask(0);
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);


$model = Mage::getModel('catalogrule/rule')
    ->getCollection();


foreach($model as $rule) {
    $today = strtotime('Yesterday');

    if(strtotime($rule->getData('to_date')) < $today) {
      $rule->delete();
    }

}

$model = Mage::getModel('salesrule/rule')
    ->getCollection();


foreach($model as $rule) {
    $today = strtotime('Yesterday');

    if(strtotime($rule->getData('to_date')) < $today) {
       $rule->delete();
    }

}

try {
    Mage::getModel('catalogrule/rule')->applyAll();
    Mage::getModel('catalogrule/flag')->loadSelf()
        ->setState(0)
        ->save();
    Mage::app()->removeCache('catalog_rules_dirty');
    echo Mage::helper('catalogrule')->__('The rules have been applied.');
} catch (Exception $e) {
    echo Mage::helper('catalogrule')->__('Unable to apply rules.');
    print_r($e);
}

The essential go-live checklist for any website

You’ve finally got to the stage where you site can go live, design,development all done and now it’s just a matter of flicking the switch. Research has shown in other industries such as medical, aviation and others that a checklist can prevent errors. The digital world isn’t quite so keen to have a consistent approach due to it’s very nature but perhaps we should.

Need a website review or just want a second opinion?, please get in touch and for a fixed fee I can provide a detailed report on your site.

Here’s my definitive checklist broken down into areas

Cross browser appearance ✓

Check for broken layouts on IE, Firefox and others

10-helpful-resources-for-cross-browser-testing

How to check:

Browsershots
Virtualbox
IE Tester
IE Developer Toolbar
Mac in the Cloud

Don’t always rely on developer toolbars or testing apps, looking at the real thing is the best way to ensure your site is glitch free.

Responsive design ✓

If your site is responsive test across as many devices as possible, ideally the real devices themselves, depending on the framework your using or your media query implementation using preview tools that try to emulate the devices may not provide an entirely accurate preview.

Responsive-Web-Design-stats

How to check:

Iphone Tester – Quick iphone preview
Ipad Peek – Quick ipad preview
Perfecto Mobile – Cross device testing for a fee
Browser Stack – Run official mobile emulators, alternatively download them yourself
The Responsinator – Preview a multitude of devices
Your browser – Firefox, Chrome and even IE11 now have built in developer tools

SEO ✓

schema

Check all page titles and meta descriptions, the most important part of the website, make sure the titles are different on every page, and relate to the content. The description is your selling window to the world so make it short and snappy.

How to check:

SEOMoz – Sign up to the 30 day trial if you don’t have an account and you can run a detailed analysis on every page
Google Webmaster Tools – Enable this to check for 404’s and any other issues
Google Structured testing tool – Check your schema markup. Haven’t done this? It’s easy enough to add last minute.

Content ✓

Check every page, and there’s only one real way to do it – proof reading, if it’s a large site assign a section to various people to help with the workload the more people that read through it the better.

That’s assuming the tone and feel of the text is consistent throughout, if it isn’t consider getting a copywriter to re-write your text.

Also check for broken links, there’s a number of automated tools to do this.

How to check:

Spellery – Online tool to spell check a page
Copify – Cheap and professional copywriting service
Integrity – Great affordable tool for Mac
Sleuth’s Link Checker – An oldie but a goodie PC based tool

 

Feedback from real people! ✓

solutions_emotional_trends

Most projects don’t have the budget or the time for this, you can instead farm out the link to as many people as you know. If you own a business send it out to everyone.

How to check:

Another option is to set out your own goals for example fill out the contact form and watch how people interact with the site in person. You will be surprised!

Usabilia – Set up real tests for real people

Developer Tests ✓

mysql-database-connection-utf-8

The developers play the most crucial role, there’s been many times where some small mistake turns into a major issue.

  • Data input/output in the correct formats
  • Data saved in the right format
  • Allowed for internationalisation?
  • Form submissions
  • Removed test files
  • Uploaded all required files
  • Dummy test of live URL with hosts file
  • File permissions set for uploading

How to check:

Not an easy answer where possible unit testing can achieve much of the above. There are some automated tools you could use to perform simple/advanced checks.

Selenium – Automated browsing for testing purposes
There’s various Firefox, Chrome plugins as well
Monitis offer this sort of testing too

Speed ✓

Often overlooked Google has long indicated that it considers this as a ranking factor how important we don’t know, but consider your visitors they don’t want to hang around either.

Get a decent hosting solution many small business’s often use poor hosting solutions from the likes of GoDaddy and 1and1 whilst they are cheap and cheerful backup is often not included and the service can be slow at times not to mention the poor customer support.

How to check:

Pingdom
Google Page speed
Dotcom Monitor

Run tests at different times, I cover some other aspects in one my Magento articles.

Load Testing ✓

Static-loading-test-in-progress

Email campaign or launching a product load testing is an essential part of going live. First get a decent monitoring package and then blast visitors to your site. That’s probably a crude approach but may be satisfactory depending on your needs you might be better with goal based load testing.

How to check:

New Relic
Monitis
Blazemeter
Jmeter

 

Peopleperhour PPH be prepared to pay for nothing – Dubious Developers

I recently put a job on peopleperhour as an hourly rate quite a decent hourly rate as I had no luck with a previous developer I sought to get a more experienced professional. After some hours of frustration on my part the problem was not resolved I subsequently raised a refund request.

Requesting a refund from Peopleperhour

The developer disputed the request and it went to a review with PPH , the developer had sneakly recorded our sessions which I thought worked in my favour as much of the time the developer spent Googling solutions for the problem in which my screen was still.

But if you check their terms and conditions the hourlie jobs are excluded from many of the safegaurds ie. poor quality of work or time deadlines so the buyer has no protection, basically your gambling your money away as the seller can claim they have done work and PPH do not consider the technical aspects of a project.

Section 7 of their Terms & Conditions

c) for Disputes concerning the quality of the work delivered: PPH will consider whether the Seller has met general quality standards as defined in Section 3.3. Assessments on quality i) down to perceived taste, or ii) requiring specialist technical or subject matter expertise, will not form part of the resolution decision;

So how do they then assess it? I’ve emailed them to ask.

No protection for Buyers

A developer can simply be sat there recording the screen clicking around edit a few variables and people per hour will consider it job done, work completed!

Thanks Peopleperhour for alerting me to this fact before putting my money in Escrow.

I’m a developer myself and I would never expect money for a poor job and definitely not in the case where I did not resolve the issue, but PPH and the developer in question have different values.

Getting your money back?

The only option left to get my money back is to claim a fraudulent transaction from my bank and from PayPal as the funds where taken without my consent.

Seems I’m not the only one

http://www.complaintsboard.com/complaints/peopleperhourcom-it-is-illegal-c627718.html


http://www.realscam.com/f10/please-careful-peopleperhour-com-2330/

Yet to hear a response from Peopleperhour.

Update:

Here’s my request to support

—————

Hi,

I posted a job on here and worked with two sellers, as you can see one of the sellers accepted my refund request because they where not able to help me with my issue.

Please can you justify why you think I should pay £80 when I have not received anything for my money.

Kind regards,
Joel

———

Peopleperhour response:

Hello Joel

Thank you for your message.

We are sorry for the inconvenience.

We found that your seller has raised dispute for the refund request, please rest assured our disputes team will soon check the workstream and resolve the issue. We suggest you to update the details of the issue on your workstream, so that our disputes team can know about the issue clearly.

Please let me know if I can help you with anything else.

Thanks

Sivakesh
PPH Customer Support

———

They decided in the sellers favour in the end, I’m now trying to claim under PayPal’s buyer protection, and then a final option is to claim from the credit card company, once I opened my PayPal dispute PPH immediately cancelled my account.

PPH final response was this:

Peopleperhour response:

From the WorkStream we can see that the job was agreed on an hourly rate basis. As defined in the PPH terms and conditions, for hourly rate jobs (unlike for fixed price jobs) the Seller is agreed to be paid for the time spent, not for deliverables produced. From the activity in the WorkStream there is sufficient evidence that time was spent on this job and that the number of hours invoiced is credible. As such, the seller will be allocated the sum in escrow.

——–

Update Two:

Finally I got all my money back through PayPal’s buyer protection thanks PayPal!

 

So I guess from my part it wasn’t clear that there is no protection the person the other end could have no developer experience yet they will be paid anyways and be paid in full.

Developers, Freelancers and Employers looking to hire freelance designers you should post jobs as a fixed priced job so you at least get some protection as their terms and conditions.

Had your own PPH nightmare get in touch?

 

Configuring Magento for Trade only

Magento never fails to amaze me in it’s flexibility and it’s ability to configure the site for almost any business objective. One popular choice might be a trade only site to use it as a platform for wholesalers or partners to purchase large quantities of goods.

trade

When thinking about this configuration usually you would want some of the below:

 

– Independent store per customer

– Customer can only access the catalog once logged in

– Remove registration or have approval only

– Prices only visible to logged in members

– Sell the same SKU to different customers at different prices

– Remove checkout payment option

– Restrict address and delivery address alterations

 

Independent store per customer

Magento does support multiple store’s per website, so a new store view can be set up per customer, but to get this to work as intended as Magento allows the customer to login to any store, which you may want to restrict. You can add a store attribute to the customer profile and then check this attribute upon login and redirect the customer to the correct site, no matter where they log in at.

Customer logged in Catalog

This is pretty straightforward again adding a check in the header to see if the customer is logged in, otherwise redirect to the login page.

Remove registration

You may want to disable registration if your using the API to update Magento or have an approval process , there is a free module on Magento Connect that allows approval. It’s also worth disabling persistent login in the configuration so there is only one login page to edit.

Prices only visible to logged in members

As above , if you wanted the site to remain accessible to the public but just hide the pricing, a simple customer session check will do the job.

Customer groups for different pricing

Supported out of the box, although if your running multiple stores this probably becomes redundant.

Remove checkout payment option

This is a little bit more tricky, and involves editing several files there’s a guide on doing just that here there are some changes needed for 1.9.1 though that aren’t covered in that article.

Restrict address and delivery address alterations

You will most likely want to restrict address changes as these are orders on account, here you can set the input/selects to disabled.

 

Thinking about starting a trade only Magento website and require a consultant? Please get in touch.

Extending magento’s REST Api v2 – Tutorial

In this post I’m going to go through all the steps you need to make in order to extend the REST Api, there is very little documentation scattered around on this and I had to pull pieces together from various official sources, not only that but the responses Magento gives you can at times be confusing and does not really give you a hint to what the problem is, finally this is a full guide so no knowledge is assumed other than setting up the rest roles/users and assuming you have ssh access on the server.

This guide is split into two parts. Part 1 configuring REST access and Oauth setup, Part 2 extending the Api if you already have the api access and oauth configured correctly then skip to part 2.

Part 1 – configuring REST access and Oauth setup

Firstly set up your roles/users follow the guide on the Magento site

  • Add a new user with an admin role
  • Add all resources to the role in Role Api Resources
  • In Rest attributes for the admin role select All resources
  • Add a new consumer

Install/Configure Oath

it’s likely you won’t have this set up on your server, this needs to be installed via PECL, if you do want to check that you don’t already have it, check phpinfo(); and see if you have these lines:

oauth

It’s important here to have curl in the Request engine support, if you don’t this means you need to install curl (apt-get install php5-curl) or oauth has not been installed correctly. If you do have it you can try upgrading

pecl upgrade oauth

or you may have to uninstall/reinstall if for example you didn’t have curl before installing oauth.

pecl uninstall oauth

There is another caveat with 1.2.3 of oauth and that is Magento uses the multi update HTTP_MULTI_STATUS with a 207 response code when you issue PUT requests to update multiple items. In this version of oauth it incorrectly gives Invalid/bad auth response when using this response code, to get around this you can compile your own version with a fix (the bug has been fixed in the latest SVN release).

If your not bothered about using the multi update via PUT then go ahead and install oauth

pecl install oauth 

To compile your own version download it from the PECL site http://pecl.php.net/package/oauth

Replace oauth.c with the fixed version

Upload all the files into a directory, if you haven’t already then issue the following commands a line at a time: (replace extname with the directory your working with)

$ cd extname
$ phpize
$ ./configure
$ make
$ make install

This should now have installed the extension correctly, you can check by doing:

pecl info oauth

And by looking at the phpinfo(), ok so now oauth is configured correctly we can run some tests.

Create a directory in the root of your site, e.g /tests and create a file called customers.php place the following code below changing:

  • mydomain.com
  • CONSUMEY_KEY
  • CONSUMER_SECRET

You can get the key and secret from System > Web Services > Oauth Consumers > Select the appropriate key you set up earlier.


<?php

require '../app/Mage.php'; //Path to Magento
Mage::app();

// $callbackUrl is a path to your file with OAuth authentication example for the Admin user
$callbackUrl = "http://mydomain.com/tests/customers.php";
$temporaryCredentialsRequestUrl = "http://mydomain.com/oauth/initiate?oauth_callback=" . urlencode($callbackUrl);
$adminAuthorizationUrl = 'http://mydomain.com/admin/oauth_authorize';
$accessTokenRequestUrl = 'http://mydomain.com/oauth/token';
$apiUrl = 'http://mydomain.com/api/rest';
$consumerKey = 'CONSUMER_KEY';
$consumerSecret = 'CONSUMER_SECRET';

session_start();
if (!isset($_GET['oauth_token']) && isset($_SESSION['state']) && $_SESSION['state'] == 1) {
 $_SESSION['state'] = 0;
}
try {
 $authType = ($_SESSION['state'] == 2) ? OAUTH_AUTH_TYPE_AUTHORIZATION : OAUTH_AUTH_TYPE_URI;
 $oauthClient = new OAuth($consumerKey, $consumerSecret, OAUTH_SIG_METHOD_HMACSHA1, $authType);
 $oauthClient->enableDebug();

if (!isset($_GET['oauth_token']) && !$_SESSION['state']) {
 $requestToken = $oauthClient->getRequestToken($temporaryCredentialsRequestUrl);
 $_SESSION['secret'] = $requestToken['oauth_token_secret'];
 $_SESSION['state'] = 1;
 header('Location: ' . $adminAuthorizationUrl . '?oauth_token=' . $requestToken['oauth_token']);
 exit;
 } else if ($_SESSION['state'] == 1) {
 $oauthClient->setToken($_GET['oauth_token'], $_SESSION['secret']);
 $accessToken = $oauthClient->getAccessToken($accessTokenRequestUrl);
 $_SESSION['state'] = 2;
 $_SESSION['token'] = $accessToken['oauth_token'];
 $_SESSION['secret'] = $accessToken['oauth_token_secret'];
 header('Location: ' . $callbackUrl);
 exit;
 } else {
 $oauthClient->setToken($_SESSION['token'], $_SESSION['secret']);

$resourceUrl = "$apiUrl/customers";
 $oauthClient->fetch($resourceUrl, array(), 'GET', array('Content-Type' => 'application/json'));
// print_r($_SESSION);
// die();
 echo $oauthClient->getLastResponse();
 }
} catch (OAuthException $e) {
 print_r($e->getMessage());
 echo "<br/>";
 print_r($e->lastResponse);
}

This should first ask you to authorise , proceed and accept the request and if all goes well you should now have a list of customers in JSON format, if you don’t want JSON then replace the fetch with


$oauthClient->fetch($resourceUrl, array(), 'GET', array('Accept' => 'text/xml'));

If you’ve got this far your now ready to extend the API, If you get any other response check that you have assigned the role properly and everywhere there is a resource dropdown it’s set to ALL. I’m not going to cover how to do this in a browser as well as Magento provides a guide on this , follow the guide you can get the access token/secret by uncommenting the two lines in the above script:


// print_r($_SESSION);
// die();

Part 2 extending the Api

Assuming you’ve set up the access and oauth correctly, (note incorrect oauth set up may look like it’s working when it isn’t!).

Let’s create a custom call for managing customer groups, create the following structure:

app/code/local/Custom
app/code/local/Custom/Restapi
app/code/local/Custom/Restapi/Groups
app/code/local/Custom/Restapi/Groups/Model
app/code/local/Custom/Restapi/Groups/Model/Api2
app/code/local/Custom/Restapi/Groups/Model/Api2/Group
app/code/local/Custom/Restapi/Groups/Model/Api2/Group/Rest
app/code/local/Custom/Restapi/Groups/Model/Api2/Group/Rest/Admin
app/code/local/Custom/Restapi/Groups/Model/Api2/Group/Rest/Admin/V1.php
app/code/local/Custom/Restapi/Groups/etc
app/code/local/Custom/Restapi/Groups/etc/api2.xml
app/code/local/Custom/Restapi/Groups/etc/config.xml

Enable our extension, create the following file

app/etc/modules/Custom_Restapi_Groups.xml
<config>
    <modules>
        <Custom_Restapi_Groups>
            <active>true</active>
            <codePool>local</codePool>
        </Custom_Restapi_Groups>
    </modules>
</config>

Set up the config.xml

<?xml version="1.0"?>
<config>
    <modules>
        <Custom_Restapi_Groups>
            <version>0.1.0.0</version>
        </Custom_Restapi_Groups>
    </modules>
    <global>
        <models>
           <groups>
                <class>Custom_Restapi_Groups_Model</class>
            </groups>
        </models>

    </global>
</config>

set up the api2.xml

<?xml version="1.0"?>
<config>
    <api2>
        <resource_groups>
            <catalog translate="title" module="api2">
                <title>Catalog</title>
                <sort_order>10</sort_order>
            </catalog>
        </resource_groups>
        <resources>
            <groups translate="title" module="api2">
                <group>catalog</group>
                <model>groups/api2_group</model>
                <title>Groups</title>
                <sort_order>10</sort_order>
                <privileges>
                    <admin>
                        <retrieve>1</retrieve>
                        <create>1</create>
                              <update>1</update>
                              <delete>1</delete>
                    </admin>

                </privileges>
                <attributes>

                    <name>Name</name>
                </attributes>

                <routes>
                    <route_entity>
                        <route>/groups/group/:id</route>
                        <action_type>entity</action_type>
                    </route_entity>
                    <route_collection>
                        <route>/groups/</route>
                        <action_type>collection</action_type>
                    </route_collection>

                </routes>

                <versions>1</versions>
            </groups>
        </resources>
    </api2>
</config>

Here the resource groups refer to which group it appears in the backend role configuration in the admin, it’s title and sort order and the privileges available, The attributes section is important for when your using POST if you do not specify the variables your posting you will get an error 400 bad request.

Finally we define our routes another important note here is when using PUT/POST you need to use the action_type collection as certain methods are only available to the collection type, if you unsure or trying to replicate how the default api works have a look in the api2.xml of the module for example this is the same as /app/code/core/Mage/Customer/etc/api2.xml

 <routes>
                    <route_entity>
                        <route>/customers/:id</route>
                        <action_type>entity</action_type>
                    </route_entity>
                    <route_collection>
                        <route>/customers</route>
                        <action_type>collection</action_type>
                    </route_collection>
                </routes>

Now that’s set up lets populate our model in V1.php

<?php
class Custom_Restapi_Groups_Model_Api2_Group_Rest_Admin_V1 extends Mage_Api2_Model_Resource
{

    /**
     * Create a customer group
     * @return array
     */

    public function _create() {
        //Create Customer Group
        $requestData = $this->getRequest()->getBodyParams();
        $groupName = $requestData['name'];
        Mage::getSingleton('customer/group')->setData(
            array('customer_group_code' => $groupName,'tax_class_id' => 3))
            ->save();

        $targetGroup = Mage::getSingleton('customer/group');
        $groupId = $targetGroup->load($groupName, 'customer_group_code')->getId();

        if($groupId) {
            $json = array('id' => $groupId);
            echo json_encode($json);
            exit();
        }

    }

     /**
     * Retrieve a group name by ID
     * @return string
     */

    public function _retrieve()
    {
        //retrieve a group name by ID
        $customerGroupId = $this->getRequest()->getParam('id');
        $groupname = Mage::getModel('customer/group')->load($customerGroupId)->getCustomerGroupCode();

        return $groupname;

    }

}
?>

Now we have our extension completed let’s try it out in a test, create the file groupInsert.php in the tests folder as before replace mydomain and the keys..

<?php

require '../app/Mage.php'; //Path to Magento
Mage::app();

// $callbackUrl is a path to your file with OAuth authentication example for the Admin user
$callbackUrl = "http://mydomain.com/tests/index.php";
$temporaryCredentialsRequestUrl = "http://mydomain.com/oauth/initiate?oauth_callback=" . urlencode($callbackUrl);
$adminAuthorizationUrl = 'http://mydomain.com/admin/oauth_authorize';
$accessTokenRequestUrl = 'http://mydomain.com/oauth/token';
$apiUrl = 'http://mydomain.com/api/rest';
$consumerKey = 'CONSUMER_KEY';
$consumerSecret = 'CONSUMER_SECRET';
session_start();
if (!isset($_GET['oauth_token']) && isset($_SESSION['state']) && $_SESSION['state'] == 1) {
    $_SESSION['state'] = 0;
}
try {
    $authType = ($_SESSION['state'] == 2) ? OAUTH_AUTH_TYPE_AUTHORIZATION : OAUTH_AUTH_TYPE_URI;
    $oauthClient = new OAuth($consumerKey, $consumerSecret, OAUTH_SIG_METHOD_HMACSHA1, $authType);
    $oauthClient->enableDebug();

    if (!isset($_GET['oauth_token']) && !$_SESSION['state']) {
        $requestToken = $oauthClient->getRequestToken($temporaryCredentialsRequestUrl);
        $_SESSION['secret'] = $requestToken['oauth_token_secret'];
        $_SESSION['state'] = 1;
        header('Location: ' . $adminAuthorizationUrl . '?oauth_token=' . $requestToken['oauth_token']);
        exit;
    } else if ($_SESSION['state'] == 1) {
        $oauthClient->setToken($_GET['oauth_token'], $_SESSION['secret']);
        $accessToken = $oauthClient->getAccessToken($accessTokenRequestUrl);
        $_SESSION['state'] = 2;
        $_SESSION['token'] = $accessToken['oauth_token'];
        $_SESSION['secret'] = $accessToken['oauth_token_secret'];
        header('Location: ' . $callbackUrl);
        exit;
    } else {
        $oauthClient->setToken($_SESSION['token'], $_SESSION['secret']);
        $resourceUrl = "$apiUrl/groups/";
        $productData = Mage::helper('core')->jsonEncode(array(
            'name'      => 'test_group'
        ));
        $headers = array('Content-Type' => 'application/json');
        $oauthClient->fetch($resourceUrl, $productData, 'POST', array('Content-Type' => 'application/json'));
        echo $oauthClient->getLastResponse();
    }
} catch (OAuthException $e) {
    print_r($e);
}

Run this file in your browser and you should get a JSON response with the name and ID, likewise we can get the group ID from a name by replacing $resourceURL and $oauthClient with the following:

 $resourceUrl = "$apiUrl/groups/group/1";
 $oauthClient->fetch($resourceUrl, array(), 'GET', array('Content-Type' => 'application/json'));

That’s it, you’ve now extended the API! If you want to use the other methods for example to GET all groups you will need to write the various functions, check your exception log to see what function is required as you will get an error similar to:

#0 /var/www/website/app/code/core/Mage/Api2/Model/Resource.php(301): Mage_Api2_Model_Resource->_critical('Resource method...')
#1 /var/www/website/app/code/core/Mage/Api2/Model/Resource.php(251): Mage_Api2_Model_Resource->_errorIfMethodNotExist('_update')
#2 /var/www/website/app/code/core/Mage/Api2/Model/Dispatcher.php(74): Mage_Api2_Model_Resource->dispatch()

Then simply add this to your class , monitor the exception log for other errors too, sometimes I would receive the error

exception 'Mage_Api2_Exception' with message 'oauth_problem=token_rejected' in /var/www/website/app/code/core/Mage/Api2/Model/Auth/Adapter/Oauth.php:61

When I didn’t have the correct attribute set up or something else was wrong event though the token was valid, the other errors your likely to encounter

exception 'Mage_Api2_Exception' with message 'Decoding error.'

The input here is wrong check your JSON/XML formatting

exception 'Mage_Api2_Exception' with message 'The request data is invalid.'

Did you add the correct attribute?

exception 'Mage_Api2_Exception' with message 'Resource data pre-validation error.'

The data is not in the expected format, for example if your trying to populate a multi-select attribute this needs to be in an array with the correct ID’s

Please do get in touch if you have anything to add to my example or i’ve missed something or alternatively if you would like your own API developed or help writing one.

Magento 1.9 upgrade template changes needed

The main things I’ve found on upgrading:

Form Keys
Form keys are required on all the forms so in update cart,login,register etc…

 <?php echo $this->getBlockHtml('formkey'); ?>

Checkout SSL
If your on Nginx and have the redirect issue using SSL it’s because you forgot to add this in index.php

if( isset($_SERVER['HTTP_X_FORWARDED_PROTO']) ) {
 if ($_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https') {
 $_SERVER['HTTPS'] = 'on';
 $_SERVER['SERVER_PORT'] = 443;
 }
}

Custom templates not loading

Make sure they are surrounded by the <cms> tag in your config.xml

<config>
 <global>
 <cms>
 <layouts>
 <new_template1>

Passwords disappearing
I had to enter all the configuration password info again e.g for Mandrill, Fishpig wordress, centinel 3ds etc.. Not sure if that’s because of the way I upgraded

Payment Gateways
Even re-entering and saving the data did not work here , I had to delete all data > save and then re-enter the data.

Review step not place order not working

Make sure your template looks like this

review = new Review('<?php echo $this->getUrl('checkout/onepage/saveOrder', array('form_key' => Mage::getSingleton('core/session')->getFormKey())) ?>', '<?php echo $this->getUrl('checkout/onepage/success') ?>', $('checkout-agreements'));

 

Skip review step magento 1.9 UK PayPal

I’m not sure if i’m missing a setting somewhere but it appears to me that in the UK version e.g if you have Website Payments Pro Payflow Edition (Includes Express Checkout) enabled, this does not skip the review step as per the 1.9 update suggests.

You can find the setting in /app/code/core/Mage/Paypal/etc/config.xml

By default it is set to 1


<skip_order_review_step>1</skip_order_review_step>

and in  /app/code/core/Mage/Paypal/Controller/Express/Abstract.php around lines 194 it checks for this parameter.


if ($this->_checkout->canSkipOrderReviewStep()) {
$this->_forward('placeOrder');
} else {
$this->_forward('placeOrder'); // skip review steps for all checkouts
// $this->_redirect('*/*/review'); // default
}

The only possible thought is that I am using the crius skip step 1 and that is changing the name of the checkout. I haven’t the time to debug fully at the moment.

 public function canSkipOrderReviewStep()
    {
        $isOnepageCheckout = !$this->_quote->getPayment()
            ->getAdditionalInformation(Mage_Paypal_Model_Express_Checkout::PAYMENT_INFO_BUTTON);
        return $this->_config->isOrderReviewStepDisabled() && $isOnepageCheckout;
    }

As the functions only check is for the config.php function isOrderReviewStepDisabled() which in turn checks the configuration value, which we hope is set to 1. So as a quick hack I commented out the review and changed it to placeorder as above.

Kind of annoying that they didn’t include this as a configurable option from the admin area and that it does not seem to work in the UK versions. Anyways that’s how to get rid of the place order/review step in 1.9.

If your using 3ds centinel then you also need this:

in review/info.phtml to auto place the order after the customer successfully completes the 3ds secure authentication


<?php
$centinel = Mage::getSingleton('centinel/service');

if($centinel->isAuthenticateSuccessful()==1) {?>
review.save();
<? }
?>

Responsive design for Magento Checkout

One part of Magento that performs particularly poorly on Mobiles is the use of tables mainly because there just isn’t enough room to see everything in a legible manner.

Some clever people have invented ways to overcome this through scrolling tables http://zurb.com/playground/responsive-tables or http://elvery.net/demo/responsive-tables/ but for ecommerce I don’t think it works well as the information often important information is hidden without scrolling the table.

For example on the cart table you typically would get something like this

responsive

Which on mobile looks like this

squashed

I find it’s easier to have a hidden alternative layout for mobile

mobile

 

[code]
@media screen and (max-width: 800px) {

#shopping-cart-table {
display:none;
}

#shopping-cart-mobile {
display:block;
}

}

[/code]

This is only shown at the breakpoint you define in my example 800px at the same time hiding the normal table output.

This method does create extra markup though and a more organised approach might be to have a separate theme but this would rely on sniffing out the user agent and has the expense of keeping another theme up to date with any changes.

Perhaps the best solution is to ditch the table markup all together for DL’s, I would like to hear your thoughts.

Speed up Magento without Varnish Cache – The Alternatives

More often than not you should probably stay clear of implementing varnish in Magento until you’ve exhausted all other options. It seems like the Holy Grail in terms of performance and will literally make your site fly, the problem is Magento is a complicated beast and any extension you use that claims a quick integration with Magento couldn’t be further from the truth! (unless of course your using a stock site, unmodified.) It’s likely you will need a ton of customisation and debugging to any of the Varnish plugins you can buy off the shelf. Not only that you will need to set up additional servers, learn how to debug varnish and it’s various admin tools and generally spend a lot of time getting it to work properly.

Turpentine offers the most reliable product from the one’s I tested but with limited documentation and help your going to struggle to get it set up correctly. The official Varnish Pagecache extension from the makers of Varnish isn’t much better, with a number of unfixed bugs and generally poor documentation, but before getting into all that you should really consider your options on improving the site speed without the use of Varnish.

Firstly set a benchmark for how your site is now, go over to http://tools.pingdom.com/fpt/ and record the speed of the homepage, category pages etc as you make each change you can run the tests again to track your progress

Optimisation for any platform

  • Tune .htaccess for speed , Creare have an excellent blueprint here
  • Use a CDN for images Pica CDN works with Amazon, Rackspace and many others
  • Install SOLR search free for CE edition alternatively this plugin works well and offers a more comprehensive set of features (be prepared for some debugging depending on your site) both these plugins require tomcat server to be installed
  • Use a CDN for content delivery e.g CloudFlare
  • Remove redundant code , php comments, html comments anything that adds to the page load however small
  • Minify your media files (JS,CSS etc) there’s a few extensions for this most widely used is the fooman speedster
  • Remove any unused styles or JS
  • Organise your main theme assets into CSS sprites
  • Reduce the quality of the images (save your images in the smallest file size possible) ideally combine the main images into a sprite
  • Use the cloud for your DNS, this can shave off an extra 50MS+ depending on your current provider, CloudFlare and other providers offer this service.
  • Use DNS pre-fetching , resolve the IP address for your assets before you use them. Something for modern browsers – Find out more in this guide from Mozilla
  • Use HTML5 browser caching through an app.manifest there’s an excellent guide on html5rocks
  • Move blocking JS to the footer, Google’s pagespeed insights offers a great tool for this (you can only do this for fonts, and external JS libaries. Don’t try moving Magento’s Core JS to the footer). Some libraries need to be in the header.

Client Side

A modern approach

Most browsers now support HTML5’s new app cache,  a cache on the users browser but browser already do that right? Yes they do – However AppCache works differently in that it’s designed for Apps where the user isn’t always online and in doing so when a cache is created on the clients side and when a page is loaded, cached resources are loaded directly from the cache, no connections to the live site, no checking if there’s a newer version –  a big speed increase.

However there’s some downsides to using this approach on how the cache expires and clearing it,  rather than re-iterate a very good explanation can be found on http://alistapart.com/article/application-cache-is-a-douchebag. I’m working on a module for Magento the makes use of AppCache if you have any thoughts please leave a comment.

Server Side

Tune Apache

Apache users can achieve quite a performance boost from tweaking the configuration:

  • Put .htaccess rules directly in your vhost conf and turn off htaccess – Magento has hundreds of directories and files each time a call is made to a file apache has to recursively loop through all these to check for .htaccess files. It might not sound like much but it can make a difference. Not only this you will be securing your server from exploits.
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>

You can find more about some of the settings here http://httpd.apache.org/docs/2.2/misc/perf-tuning.html

  • Check the apache config,  are the settings correct for your server spec? There’s numerous guide on this so I won’t detail this here, you could also try adjusting settings and running the benchmark several times to see what works best, in normal load and under stress.
  • There’s 25 tips here on tweaking apache

Use Nginx + PHP-FPM

Nginx uses much less resources as it works a bit differently from Apache, if your already familiar with Apache and don’t have the time to learn how to use Nginx it’s probably best to take it as far as you can with Apache before considering this option, if you already use it then it works pretty well out of the box. What you might need though is some extra configuration settings there’s a guide here on that.

Make sure you install php-fpm if your running nginx.

apt-get install php5-fpm

In most cases the above configuration can be left as is in nginx, if your using port 9000, you will want to edit the php-fpm configuration usually www.conf found in:

/etc/php5/fpm/pool.d/

Change the user and group to match that of your nginx installation as default this is www-data

; Unix user/group of processes
; Note: The user is mandatory. If the group is not set, the default user's group
;       will be used.
user = www-data
group = www-data

There are three different ways of running php-fpm and that is ondemand,dynamic or static I won’t go into the detail of these, but i’ve found ondemand to work better for Magento which basically runs as many process’s as is required up to a maximum rather than fixing a certain amount to be running all the time. It’s worth noting here you can enable a status page that gives you some information about what’s going on.

pm.status_path = /status

In your php-fpm uncomment this line, you can then visit website.com/status to get an output, there’s a great tutorial on using php-fpm status page here.

Tweak Mysql

There’s many guides out there on mysql and I make no claims to be an expert these settings have helped

skip-name-resolve
log_slow_queries       = /var/log/mysql/mysql-slow.log

If your mysql is listening locally e.g bind-address in my.cnf says localhost or 127.0.0.1 etc then you don’t need to resolve the name , this will avoid any delay from the DNS.

The rest of your config should be configured as per the report from mysqltuner, if that’s new to you check out this post on how to use it.

Redis

I have recently come across Redis It’s an extremely fast cache storage engine and works seamlessly with Magento, it does require some server configuration but I would say this is one of the easiest to configure and get up and running quickly. There’s also support for sessions via Redis Sessions have not tried this yet but also looks very solid as of CE 1.8 Redis comes part of the default install so it’s as simple as configuring it via the local.xml if you want to find out more there’s some benchmark information here and a guide to using Magento with Redis on the Magento Site.

HHVM

Review coming soon on this.

APC caching

Install it onto your server

sudo apt-get install php-apc
sudo service apache2 restart

Whilst it’s installing make a note of the version, we will need this later to tweak the settings e.g

Get:1 http://uk.archive.ubuntu.com/ubuntu/ precise/universe php-apc i386 3.1.7-1 [79.2 kB]

Enable it via/magento/app/etc/local.xml and add the following lines (note if you have more than one Magento Install on the same server make sure the prefix is unique for each one)

<global>
        ...
        <cache>
            <backend>apc</backend>
            <prefix>alphanumeric</prefix>
        </cache>
        ...
</global>

Tweaking APC couldn’t be easier, first check what version is installed (as noted earlier) download and browse the archive which matches on http://pecl.php.net/package/APC

Inside there will be a file called apc.php , you need to put this somewhere that is served by Apache or Nginx ideally on a password protected area of your site.

What you want to achieve here is a high Hit ratio and little fragmentation, usually this happens because there is not enough memory allocated to APC. You can alter this setting by changing the config file in /etc/php5/conf.d/apc.ini or wherever your php install is located until you achieve the desired result.

it should look something like this:

extension=apc.so
apc.enabled=1
apc.shm_size=500M
apc.max_file_size=3M
apc.enable_cli=1

shm_size
The maximum amount of memory APC can use, one it runs out it has to purge cached items which leads to fragmentation.

max_file_size
The maximum file size that can be cached by APC, this defaults to a low value and I would recommend changing this to 3M or 5M.

Memcache

There’s a decent guide here on installing Memcache once you’ve done that follow this guide on enabling it in Magento

Single split servers

This is generally a good idea for failover, and that’s to have a separate MySQL and Web Server, this takes the load off one server and allows you to upscale each individually. Ideally if you have enough traffic also separate out the SOLR instance. Splitting the connection between mysql and your web server can have negative effect though depending on the connection between each you will need a gigabit connection to remove network latency so if you haven’t got much load coming from apache/nginx then it’s probably not worth it.

Google PageSpeed

This can be installed server side on Apache as a module or Nginx (Nginx requires a re-build) , this allows you to do a lot of optimisation on the fly like removing whitespace, minifying JS/CSS and even optimising images. more about Google PageSpeed

App code

Non-Varnish Caching (build your own)

Magento’s built in cache

There’s a few ways to add caching that don’t involve varnish, first is that Magento comes with comprehensive caching out of the box.

There’s really only four methods available to us.

save($value, $key, $tags = array(), $lifeTime=null)
load($key)
remove($key)
clean($tags = array()

So let’s add something to the cache in this example we are retrieving the lowest price from a grouped product, it’s pretty intensive as it has to loop through each simple product to retrieve it’s price. The prices don’t change very often so we don’t have to do this everytime!

// load the cache
$cache = Mage::app()->getCache();

// The Code we are caching
if(!$cache->load($_product->getId())) {

// The cache doesn't exist
$aProductIds = $_product->getTypeInstance()->getChildrenIds($_product->getId());
                                    $prices = array();
                                    foreach ($aProductIds as $ids) {
                                        foreach ($ids as $id) {

                                            $aProduct = Mage::getModel('catalog/product')->load($id);
                                            if($aProduct->isSaleable()) {
                                            $prices[] = $aProduct->getPriceModel()->getPrice($aProduct);
                                            }
                                        }
                                    }
                                    asort($prices);
                                    $prices = array_shift($prices);
                                    $grouped_price = $helper->currency($prices,true,false);
// save
$cache->save($grouped_price, $_product->getId(), array("grouped_prices"), 3600);
} else {

// load the saved price
$grouped_price = $cache->load($_product->getId());

}

echo $grouped_price;

So in the above block we are:

  1. First checking if the the cache named “id of product” exists
  2. If it returns false we then run the code to save the price to the cache with a lifetime of 1 hour (3600 seconds) and with a tag of array(“grouped_prices”)
  3. If the cache doesn’t exist we run the code as usual
  4. In all scenarios $grouped_price is returned with the price

If you don’t set a lifetime value then the item would be cached until it’s removed manually, to remove this value earlier than 1 hour we would do so by using remove

$cache->remove($_product->getId());

We can also remove by the tag if for instance you wanted to clear the cache for all grouped product prices you can use clean.

$cache->clean(array("grouped_prices"));

Full Page caching and others

Unicache Inchoo

This builds on the default caching system but really only offers convenience and an admin section allowing you to clear the cache for individual items. check it out here

Lesti FPC

Gordon Lesti wrote his own FPC for Magento, it’s easy to install follow the guide on using it, although this suffers from the same problems as using Varnish if you have custom blocks you will need to configure them for the site to work properly. However it does not require additional servers or software so it cuts a lot of set up time.

Do you know of any other ways to speed up Magento?, get in touch.

Quafzi Performance Tweaks

Recently came across this module, that offers a lot of optimisations based on recommendations from Ecommerce devs

https://github.com/quafzi/magento-performance-tweaks

One thing I had to disable on this particular module is the CMS block caching if for example like me you are using it to load a template that changes for each product category. It’s well commented so it’s easy to see the particular changes that might affect your site.