Magento collection filtered

Here’s a simple setup for a product collection with the following filters used in most of my modules, ideal for things like product feeds etc.

It has the following filters

– By Store
– Must have price > 0
– Must cost > 0
– Only simple products
– Visible either Catalog,Search or Both

$visibility = array(
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_BOTH,
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_IN_CATALOG,
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_IN_SEARCH
);

$storeId = 1;
$collection = Mage::getResourceModel('catalog/product_collection');
$collection->getEntity()->setStoreId($storeId);

// Reset the select.
$collection->getSelect()->reset();

// Update table name.
$reflectionMethod = new ReflectionMethod($collection, '_initSelect');
$reflectionMethod->setAccessible(true);
$reflectionMethod->invoke($collection);

$collection
 ->addAttributeToSelect('sku')
 ->addAttributeToSelect('name')
 ->addAttributeToSelect('description')
 ->addAttributeToSelect('small_image')
 ->addAttributeToSelect('manufacturer')
 ->addAttributeToSelect('msrp')
 ->addAttributeToFilter('visibility', $visibility)
 ->addAttributeToFilter('status', '1')
 ->addAttributeToFilter('type_id', array('eq' => 'simple'))
 ->addFinalPrice()
 ->setPageSize(900000)
 ->setCurPage(1);

Calculating selling price from the cost price

Recently in my price matching system I had to calculate the selling price of a product in a way that also calculates a transaction fee percentage on the final selling price without knowing the selling price, and without knowing what the transaction fee is until you’ve calculated the total price. Had fun chasing your tail yet?

You can do this in Excel with circular references but you will get circular reference warnings and the only way is to allow these warnings and an iteration limit of 100.

Stepping aside from the magic of excel I was able to work out the exact formula for this and below is a calculation for anyone else struggling to work out how to do this.

$cp = 9.56; // cost price
$d = 2.50; // delivery fee
$m = 0.15; // margin 15%
$tf = 0.019; // transaction fee

$sellingPrice = (($cp+$d)*(1+($m)))/(1-((1/6)+$tf)*(1+($m)));

$costPrice = $x / (1+(.15));

echo $sellingPrice."\n";
echo $costPrice;

Magento image not loading when using cron, only getting placeholder

I have a module which sends out an email via Mailchimp with a scheduled task for Magento’s internal cron and in doing so looks up the product image.

Mage::helper('catalog/image')->init($product, 'small_image')->resize(150));

The problem being only the placeholder was being returned when the cron job was executed. To confuse matters when I ran the script in testing from the browser it would return the correct image.

This is a nasty problem at first I thought this might be the answer Sangay details there that if your config does not specify a M,GB etc that it will return the wrong calculation, but then why the different results depending on how I ran the script?

To get to the bottom of this I ran what is being run in Mage_Catalog_Helper_Image

$model = Mage::getModel('catalog/product_image');
$model->setBaseFile($this->product->getData('small_image'))->resize(600,600);
$url = $model->saveFile()->getUrl();

PHP Fatal error:  Uncaught exception 'Varien_Exception' with message 'Memory limit has been reached.' in /lib/Varien/Image/Adapter/Gd2.php:58

Returning this shows the actual exception I did not get any Exception in the Mage exception log, and not in the host’s error log. On my part a server configuration oversight as the CLI php.ini did not have the error log set up. In any case that’s the error.

Because the value returned from _convertToByte function in Mage_Catalog_Helper_Image is -1 as the CLI php.ini sets the memory_limit to -1 (unlimited).

This could be an oversight by Magento or a design feature not to run this task on unlimited memory, personally I think it’s a bug. I modified the function to the following to set a 2G limit in this case.

   if (stripos($memoryValue, 'M') !== false) {
            return (int)$memoryValue * 1024 * 1024;
        }
        elseif (stripos($memoryValue, 'KB') !== false) {
            return (int)$memoryValue * 1024;
        }
        elseif ($memoryValue == "-1") {
            return '2147483648'; //2G
        }

Clearing expired magento shopping/sales rules programatically

Here’s a nice cleanup script I use to clear out expired shopping cart rules and catalog price rules and apply those changes. It’s one of those annoying things that you would think there would be a button for.

I’ve set the expiry check to the previous day and date comparison using strtotime you might want to adjust that, and use php’s dateTime library for a more thorough comparison as this relies on your server’s timezone configuration matching magento’s and being set up correctly.

If your wondering what setState does that’s to remove the message in magento admin about un-applied rules.

require '../app/Mage.php';

umask(0);
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);


$model = Mage::getModel('catalogrule/rule')
    ->getCollection();


foreach($model as $rule) {
    $today = strtotime('Yesterday');

    if(strtotime($rule->getData('to_date')) < $today) {
      $rule->delete();
    }

}

$model = Mage::getModel('salesrule/rule')
    ->getCollection();


foreach($model as $rule) {
    $today = strtotime('Yesterday');

    if(strtotime($rule->getData('to_date')) < $today) {
       $rule->delete();
    }

}

try {
    Mage::getModel('catalogrule/rule')->applyAll();
    Mage::getModel('catalogrule/flag')->loadSelf()
        ->setState(0)
        ->save();
    Mage::app()->removeCache('catalog_rules_dirty');
    echo Mage::helper('catalogrule')->__('The rules have been applied.');
} catch (Exception $e) {
    echo Mage::helper('catalogrule')->__('Unable to apply rules.');
    print_r($e);
}

Magento Review Generator

Adding fake reviews to your site is in the long run a bad idea, it can not only damage your site’s reputation but it’s bad practice. That said I have developed a module that will seed Magento with reviews which could be based on a specific criteria, it does this by creating a unique combination of words to create a readable sentence.

Not only this but it can add the reviews over time at a pace that is SEO friendly and organic.

You might think Google could easily foil such a system but in reality there is such a huge combination of sentences that it would take some genius algorithm to work out that these where being added programmatically in combination with a random time sequence I don’t see how Google would know the difference as long as it’s not overdone.

Getting back on topic the idea of this module is just to seed, marginally boost or test you product ratings for example adding a minimum of 1 or 2 reviews per product to get you started.

Get in touch if this is something you need on your ecommerce store.

You can now buy this script from my shop here

Magento price not saving multi scope/store

Just ran into this issue on a multi store view website in the admin backend, it seems because the product price scope is global unless you have a module to override this functionality the price should only have one attribute value, and each time you save the product in any store view you are essentially saving one price to store 0 the admin store.

If you used an importer or some other process to import your products you may find that there are incorrectly two records in the catalog_product_entity_decimal table

entity_tbl

You can look at the data via this query, replacing ATTRIBUTE_ID with your product ID

SELECT * FROM catalog_product_entity_decimal WHERE attribute_id = ATTRIBUTE_ID

How to fix?

In my store the price attribute is 75 it may differ though, there should only be one entry for store_id 0. If you now delete that row you will be able to save the product price. You can delete all these values in one line. (please backup your database first.)

DELETE FROM magento_db.catalog_product_entity_decimal WHERE attribute_id = 75 AND store_id = 2

Where store_id is the additional store entry and magento_db is your database name.

I’m Joel a magento developer/freelancer based in St Albans

I’m a developer for Magento, I have developed many custom modules for my clients, some of my work includes:

  • Banner Management
  • Product display grid allowing the user to control the homepage inserting blocks of video,products,reviews,offers etc
  • Landing page builder tool – making it easy to create many pages at once around several different keywords
  • Price checking module – Automatically checks google shopping for defined competitor prices, matches pricing and updates the products via Google’s Merchant API in realtime.
  • Customisation of SOLR Bridge plugin
  • WordPress Integration
  • Magento Custom Theming
  • Custom PayPal response for error messages
  • Cardinal Commerce 3DS integration
  • Trade only configuration
  • Removing shipping and payment steps from the checkout
  • Extending Magento’s REST API and building a custom REST API
  • Integrating Loyalty points system, custom events to update MailChimp
  • MailChimp Integration
  • Social discount module – offering discounts at the checkout for social actions
  • Custom email templates
  • Integrating Amazon Payment module

I’m based in the London, St Albans, Watford area and available for a consultation, primarily I prefer to work by email.

Configuring Magento for Trade only

Magento never fails to amaze me in it’s flexibility and it’s ability to configure the site for almost any business objective. One popular choice might be a trade only site to use it as a platform for wholesalers or partners to purchase large quantities of goods.

trade

When thinking about this configuration usually you would want some of the below:

 

– Independent store per customer

– Customer can only access the catalog once logged in

– Remove registration or have approval only

– Prices only visible to logged in members

– Sell the same SKU to different customers at different prices

– Remove checkout payment option

– Restrict address and delivery address alterations

 

Independent store per customer

Magento does support multiple store’s per website, so a new store view can be set up per customer, but to get this to work as intended as Magento allows the customer to login to any store, which you may want to restrict. You can add a store attribute to the customer profile and then check this attribute upon login and redirect the customer to the correct site, no matter where they log in at.

Customer logged in Catalog

This is pretty straightforward again adding a check in the header to see if the customer is logged in, otherwise redirect to the login page.

Remove registration

You may want to disable registration if your using the API to update Magento or have an approval process , there is a free module on Magento Connect that allows approval. It’s also worth disabling persistent login in the configuration so there is only one login page to edit.

Prices only visible to logged in members

As above , if you wanted the site to remain accessible to the public but just hide the pricing, a simple customer session check will do the job.

Customer groups for different pricing

Supported out of the box, although if your running multiple stores this probably becomes redundant.

Remove checkout payment option

This is a little bit more tricky, and involves editing several files there’s a guide on doing just that here there are some changes needed for 1.9.1 though that aren’t covered in that article.

Restrict address and delivery address alterations

You will most likely want to restrict address changes as these are orders on account, here you can set the input/selects to disabled.

 

Thinking about starting a trade only Magento website and require a consultant? Please get in touch.

Extending magento’s REST Api v2 – Tutorial

In this post I’m going to go through all the steps you need to make in order to extend the REST Api, there is very little documentation scattered around on this and I had to pull pieces together from various official sources, not only that but the responses Magento gives you can at times be confusing and does not really give you a hint to what the problem is, finally this is a full guide so no knowledge is assumed other than setting up the rest roles/users and assuming you have ssh access on the server.

This guide is split into two parts. Part 1 configuring REST access and Oauth setup, Part 2 extending the Api if you already have the api access and oauth configured correctly then skip to part 2.

Part 1 – configuring REST access and Oauth setup

Firstly set up your roles/users follow the guide on the Magento site

  • Add a new user with an admin role
  • Add all resources to the role in Role Api Resources
  • In Rest attributes for the admin role select All resources
  • Add a new consumer

Install/Configure Oath

it’s likely you won’t have this set up on your server, this needs to be installed via PECL, if you do want to check that you don’t already have it, check phpinfo(); and see if you have these lines:

oauth

It’s important here to have curl in the Request engine support, if you don’t this means you need to install curl (apt-get install php5-curl) or oauth has not been installed correctly. If you do have it you can try upgrading

pecl upgrade oauth

or you may have to uninstall/reinstall if for example you didn’t have curl before installing oauth.

pecl uninstall oauth

There is another caveat with 1.2.3 of oauth and that is Magento uses the multi update HTTP_MULTI_STATUS with a 207 response code when you issue PUT requests to update multiple items. In this version of oauth it incorrectly gives Invalid/bad auth response when using this response code, to get around this you can compile your own version with a fix (the bug has been fixed in the latest SVN release).

If your not bothered about using the multi update via PUT then go ahead and install oauth

pecl install oauth 

To compile your own version download it from the PECL site http://pecl.php.net/package/oauth

Replace oauth.c with the fixed version

Upload all the files into a directory, if you haven’t already then issue the following commands a line at a time: (replace extname with the directory your working with)

$ cd extname
$ phpize
$ ./configure
$ make
$ make install

This should now have installed the extension correctly, you can check by doing:

pecl info oauth

And by looking at the phpinfo(), ok so now oauth is configured correctly we can run some tests.

Create a directory in the root of your site, e.g /tests and create a file called customers.php place the following code below changing:

  • mydomain.com
  • CONSUMEY_KEY
  • CONSUMER_SECRET

You can get the key and secret from System > Web Services > Oauth Consumers > Select the appropriate key you set up earlier.


<?php

require '../app/Mage.php'; //Path to Magento
Mage::app();

// $callbackUrl is a path to your file with OAuth authentication example for the Admin user
$callbackUrl = "http://mydomain.com/tests/customers.php";
$temporaryCredentialsRequestUrl = "http://mydomain.com/oauth/initiate?oauth_callback=" . urlencode($callbackUrl);
$adminAuthorizationUrl = 'http://mydomain.com/admin/oauth_authorize';
$accessTokenRequestUrl = 'http://mydomain.com/oauth/token';
$apiUrl = 'http://mydomain.com/api/rest';
$consumerKey = 'CONSUMER_KEY';
$consumerSecret = 'CONSUMER_SECRET';

session_start();
if (!isset($_GET['oauth_token']) && isset($_SESSION['state']) && $_SESSION['state'] == 1) {
 $_SESSION['state'] = 0;
}
try {
 $authType = ($_SESSION['state'] == 2) ? OAUTH_AUTH_TYPE_AUTHORIZATION : OAUTH_AUTH_TYPE_URI;
 $oauthClient = new OAuth($consumerKey, $consumerSecret, OAUTH_SIG_METHOD_HMACSHA1, $authType);
 $oauthClient->enableDebug();

if (!isset($_GET['oauth_token']) && !$_SESSION['state']) {
 $requestToken = $oauthClient->getRequestToken($temporaryCredentialsRequestUrl);
 $_SESSION['secret'] = $requestToken['oauth_token_secret'];
 $_SESSION['state'] = 1;
 header('Location: ' . $adminAuthorizationUrl . '?oauth_token=' . $requestToken['oauth_token']);
 exit;
 } else if ($_SESSION['state'] == 1) {
 $oauthClient->setToken($_GET['oauth_token'], $_SESSION['secret']);
 $accessToken = $oauthClient->getAccessToken($accessTokenRequestUrl);
 $_SESSION['state'] = 2;
 $_SESSION['token'] = $accessToken['oauth_token'];
 $_SESSION['secret'] = $accessToken['oauth_token_secret'];
 header('Location: ' . $callbackUrl);
 exit;
 } else {
 $oauthClient->setToken($_SESSION['token'], $_SESSION['secret']);

$resourceUrl = "$apiUrl/customers";
 $oauthClient->fetch($resourceUrl, array(), 'GET', array('Content-Type' => 'application/json'));
// print_r($_SESSION);
// die();
 echo $oauthClient->getLastResponse();
 }
} catch (OAuthException $e) {
 print_r($e->getMessage());
 echo "<br/>";
 print_r($e->lastResponse);
}

This should first ask you to authorise , proceed and accept the request and if all goes well you should now have a list of customers in JSON format, if you don’t want JSON then replace the fetch with


$oauthClient->fetch($resourceUrl, array(), 'GET', array('Accept' => 'text/xml'));

If you’ve got this far your now ready to extend the API, If you get any other response check that you have assigned the role properly and everywhere there is a resource dropdown it’s set to ALL. I’m not going to cover how to do this in a browser as well as Magento provides a guide on this , follow the guide you can get the access token/secret by uncommenting the two lines in the above script:


// print_r($_SESSION);
// die();

Part 2 extending the Api

Assuming you’ve set up the access and oauth correctly, (note incorrect oauth set up may look like it’s working when it isn’t!).

Let’s create a custom call for managing customer groups, create the following structure:

app/code/local/Custom
app/code/local/Custom/Restapi
app/code/local/Custom/Restapi/Groups
app/code/local/Custom/Restapi/Groups/Model
app/code/local/Custom/Restapi/Groups/Model/Api2
app/code/local/Custom/Restapi/Groups/Model/Api2/Group
app/code/local/Custom/Restapi/Groups/Model/Api2/Group/Rest
app/code/local/Custom/Restapi/Groups/Model/Api2/Group/Rest/Admin
app/code/local/Custom/Restapi/Groups/Model/Api2/Group/Rest/Admin/V1.php
app/code/local/Custom/Restapi/Groups/etc
app/code/local/Custom/Restapi/Groups/etc/api2.xml
app/code/local/Custom/Restapi/Groups/etc/config.xml

Enable our extension, create the following file

app/etc/modules/Custom_Restapi_Groups.xml
<config>
    <modules>
        <Custom_Restapi_Groups>
            <active>true</active>
            <codePool>local</codePool>
        </Custom_Restapi_Groups>
    </modules>
</config>

Set up the config.xml

<?xml version="1.0"?>
<config>
    <modules>
        <Custom_Restapi_Groups>
            <version>0.1.0.0</version>
        </Custom_Restapi_Groups>
    </modules>
    <global>
        <models>
           <groups>
                <class>Custom_Restapi_Groups_Model</class>
            </groups>
        </models>

    </global>
</config>

set up the api2.xml

<?xml version="1.0"?>
<config>
    <api2>
        <resource_groups>
            <catalog translate="title" module="api2">
                <title>Catalog</title>
                <sort_order>10</sort_order>
            </catalog>
        </resource_groups>
        <resources>
            <groups translate="title" module="api2">
                <group>catalog</group>
                <model>groups/api2_group</model>
                <title>Groups</title>
                <sort_order>10</sort_order>
                <privileges>
                    <admin>
                        <retrieve>1</retrieve>
                        <create>1</create>
                              <update>1</update>
                              <delete>1</delete>
                    </admin>

                </privileges>
                <attributes>

                    <name>Name</name>
                </attributes>

                <routes>
                    <route_entity>
                        <route>/groups/group/:id</route>
                        <action_type>entity</action_type>
                    </route_entity>
                    <route_collection>
                        <route>/groups/</route>
                        <action_type>collection</action_type>
                    </route_collection>

                </routes>

                <versions>1</versions>
            </groups>
        </resources>
    </api2>
</config>

Here the resource groups refer to which group it appears in the backend role configuration in the admin, it’s title and sort order and the privileges available, The attributes section is important for when your using POST if you do not specify the variables your posting you will get an error 400 bad request.

Finally we define our routes another important note here is when using PUT/POST you need to use the action_type collection as certain methods are only available to the collection type, if you unsure or trying to replicate how the default api works have a look in the api2.xml of the module for example this is the same as /app/code/core/Mage/Customer/etc/api2.xml

 <routes>
                    <route_entity>
                        <route>/customers/:id</route>
                        <action_type>entity</action_type>
                    </route_entity>
                    <route_collection>
                        <route>/customers</route>
                        <action_type>collection</action_type>
                    </route_collection>
                </routes>

Now that’s set up lets populate our model in V1.php

<?php
class Custom_Restapi_Groups_Model_Api2_Group_Rest_Admin_V1 extends Mage_Api2_Model_Resource
{

    /**
     * Create a customer group
     * @return array
     */

    public function _create() {
        //Create Customer Group
        $requestData = $this->getRequest()->getBodyParams();
        $groupName = $requestData['name'];
        Mage::getSingleton('customer/group')->setData(
            array('customer_group_code' => $groupName,'tax_class_id' => 3))
            ->save();

        $targetGroup = Mage::getSingleton('customer/group');
        $groupId = $targetGroup->load($groupName, 'customer_group_code')->getId();

        if($groupId) {
            $json = array('id' => $groupId);
            echo json_encode($json);
            exit();
        }

    }

     /**
     * Retrieve a group name by ID
     * @return string
     */

    public function _retrieve()
    {
        //retrieve a group name by ID
        $customerGroupId = $this->getRequest()->getParam('id');
        $groupname = Mage::getModel('customer/group')->load($customerGroupId)->getCustomerGroupCode();

        return $groupname;

    }

}
?>

Now we have our extension completed let’s try it out in a test, create the file groupInsert.php in the tests folder as before replace mydomain and the keys..

<?php

require '../app/Mage.php'; //Path to Magento
Mage::app();

// $callbackUrl is a path to your file with OAuth authentication example for the Admin user
$callbackUrl = "http://mydomain.com/tests/index.php";
$temporaryCredentialsRequestUrl = "http://mydomain.com/oauth/initiate?oauth_callback=" . urlencode($callbackUrl);
$adminAuthorizationUrl = 'http://mydomain.com/admin/oauth_authorize';
$accessTokenRequestUrl = 'http://mydomain.com/oauth/token';
$apiUrl = 'http://mydomain.com/api/rest';
$consumerKey = 'CONSUMER_KEY';
$consumerSecret = 'CONSUMER_SECRET';
session_start();
if (!isset($_GET['oauth_token']) && isset($_SESSION['state']) && $_SESSION['state'] == 1) {
    $_SESSION['state'] = 0;
}
try {
    $authType = ($_SESSION['state'] == 2) ? OAUTH_AUTH_TYPE_AUTHORIZATION : OAUTH_AUTH_TYPE_URI;
    $oauthClient = new OAuth($consumerKey, $consumerSecret, OAUTH_SIG_METHOD_HMACSHA1, $authType);
    $oauthClient->enableDebug();

    if (!isset($_GET['oauth_token']) && !$_SESSION['state']) {
        $requestToken = $oauthClient->getRequestToken($temporaryCredentialsRequestUrl);
        $_SESSION['secret'] = $requestToken['oauth_token_secret'];
        $_SESSION['state'] = 1;
        header('Location: ' . $adminAuthorizationUrl . '?oauth_token=' . $requestToken['oauth_token']);
        exit;
    } else if ($_SESSION['state'] == 1) {
        $oauthClient->setToken($_GET['oauth_token'], $_SESSION['secret']);
        $accessToken = $oauthClient->getAccessToken($accessTokenRequestUrl);
        $_SESSION['state'] = 2;
        $_SESSION['token'] = $accessToken['oauth_token'];
        $_SESSION['secret'] = $accessToken['oauth_token_secret'];
        header('Location: ' . $callbackUrl);
        exit;
    } else {
        $oauthClient->setToken($_SESSION['token'], $_SESSION['secret']);
        $resourceUrl = "$apiUrl/groups/";
        $productData = Mage::helper('core')->jsonEncode(array(
            'name'      => 'test_group'
        ));
        $headers = array('Content-Type' => 'application/json');
        $oauthClient->fetch($resourceUrl, $productData, 'POST', array('Content-Type' => 'application/json'));
        echo $oauthClient->getLastResponse();
    }
} catch (OAuthException $e) {
    print_r($e);
}

Run this file in your browser and you should get a JSON response with the name and ID, likewise we can get the group ID from a name by replacing $resourceURL and $oauthClient with the following:

 $resourceUrl = "$apiUrl/groups/group/1";
 $oauthClient->fetch($resourceUrl, array(), 'GET', array('Content-Type' => 'application/json'));

That’s it, you’ve now extended the API! If you want to use the other methods for example to GET all groups you will need to write the various functions, check your exception log to see what function is required as you will get an error similar to:

#0 /var/www/website/app/code/core/Mage/Api2/Model/Resource.php(301): Mage_Api2_Model_Resource->_critical('Resource method...')
#1 /var/www/website/app/code/core/Mage/Api2/Model/Resource.php(251): Mage_Api2_Model_Resource->_errorIfMethodNotExist('_update')
#2 /var/www/website/app/code/core/Mage/Api2/Model/Dispatcher.php(74): Mage_Api2_Model_Resource->dispatch()

Then simply add this to your class , monitor the exception log for other errors too, sometimes I would receive the error

exception 'Mage_Api2_Exception' with message 'oauth_problem=token_rejected' in /var/www/website/app/code/core/Mage/Api2/Model/Auth/Adapter/Oauth.php:61

When I didn’t have the correct attribute set up or something else was wrong event though the token was valid, the other errors your likely to encounter

exception 'Mage_Api2_Exception' with message 'Decoding error.'

The input here is wrong check your JSON/XML formatting

exception 'Mage_Api2_Exception' with message 'The request data is invalid.'

Did you add the correct attribute?

exception 'Mage_Api2_Exception' with message 'Resource data pre-validation error.'

The data is not in the expected format, for example if your trying to populate a multi-select attribute this needs to be in an array with the correct ID’s

Please do get in touch if you have anything to add to my example or i’ve missed something or alternatively if you would like your own API developed or help writing one.