Upgrading to Magento 2

Having had the opportunity to work on a new build using Magento 2 the architecture of the application has changed significantly, to me it has definitely increased the difficulty of getting a Magento project up and running but in return offers a more granular and organised application to work with.

With this view I think it’s unlikely that you would upgrade your site to v2. Magento have not given an upgrade path to 2.0 and I doubt they will, so it’s probably worth waiting for the opportunity to re-design the site before moving over to the new platform.

Magento have committed to 3 years from general release of 2.0 so that gives you till around 2019 to move over. In the meantime that means they will continue to provide security patches and support for the 1.x platform.

Enhancing Magento Security – Best practices

After watching the recent panorama documentary on the recent TalkTalk hack, it made me wonder how vulnerable many of the Magento sites where and what can be done to tighten Magento’s security.

Many opensource platforms are often hacked for example WordPress is a regular culprit , though it’s not as widespread that the core code has vulnerabilities it’s usually a rogue plugin or one that hasn’t been updated, with WordPress’s built in automatic updating it has a bit more protection.

Magento doesn’t have such frequent easy updating but recently the Magento team have released a flurry of patches, in what seems to be either that they have woken up to the fact that Magento in it’s previous incarnation had many exploits or that retailers have fed back to Magento. Whatever the case like any opensource platform continual updates are required especially in an ecommerce environment.

Magento and other ecommerce systems will be a prime target for hackers, thousands of un-encrypted customer details and in most cases it’s in-practical to encrypt them so your left with the only option of tightening up server security and the application itself.

So what steps can you take to secure Magento?

Patch/Upgrade

Install the most up to date patches
http://magento.com/security-patch

As of writing there are around 15 patches for community edition, depending on which version you have installed.

https://www.magentocommerce.com/download

Alternatively upgrade Magento to the most current release, as it includes all current patches 1.9.2.2

Upgrade to the next version 2.0
Probably not viable for most retailers just yet

PCI Compliance

Any ecommerce should be PCI compliant even if your taking payments offsite it’s worth having a PCI scanner in place. For that I would recommend using Trustwave some PayPal integrations require it anyways.

This will scan your site for common exploits, you might be surprised by what it picks up.

Trustwave

https://www.trustwave.com

Strong SSL

Ensure your using a strong cipher suite and your webserver is set up to use the correct protocols.

SSL Rating

https://www.ssllabs.com/ssltest/

Run your site through SSL Labs for any recommendations, A or above rating is recommended.

Change the admin path

A simple yet effective change to move the /admin to somewhere else, edit your config xml including the following.

<admin>
<routers>
<adminhtml>
<args>
<frontName><![CDATA[admin]]></frontName>
</args>
</adminhtml>
</routers>
</admin>

If this isn’t an option as it can break some plugins, then request your server admin add (if not already installed) a rule to fail2ban to prevent attacks on the /admin folder.

Prevent access

To magento’s directories, see a complete list in the bottom example.

directly in the vhost (Apache):

LocationMatch ^/(app/|var/) >
    Require all denied

Nginx:

location ^~ /app/ { deny all; }
location ^~ /var/ { deny all; }

Alternatively you could return a 404

location /app/                { return 404; }
location /downloader/         { return 404; }
location /errors/             { return 404; }
location /media/              { return 404; }
location /assets/             { return 404; }
location /images/             { return 404; }
location /skin/               { return 404; }
location /includes/           { return 404; }
location /lib/                { return 404; }
location /media/downloadable/ { return 404; }
location /pkginfo/            { return 404; }
location /report/config.xml   { return 404; }
location /shell/              { return 404; }
location /var/                { return 404; }

Also make sure to include any external import/export tools e.g Magmi

Turn on SSL for the admin

secure

Ideally you should have your entire site as SSL, this can affect performance on lower end servers.

Use strong passwords

Simple but obvious, this is probably the single biggest threat. Make it easy to remember but hard to guess, a password generator might not be useful here unless you use a password manager. Ideally use two-factor authentication

Two factor auth

Through this extension: http://www.xtento.com/magento-extensions/two-factor-authentication-enhanced-admin-security.html

Restrict admin by IP

Ideally this would be better sitting in the apache/nginx config, but if you don’t have that kind of access or want GUI control there is a free module.

http://www.magentocommerce.com/magento-connect/et-ip-security.html

Use a more granular admin permission module

This allows you to control in a more detailed way what each admin user can do.

http://www.aitoc.com/en/magentomods_advanced_permissions.html

Even if it’s just you managing your site, you could have a master login and editor role, to limit the use of full access.

Advanced server side

More advanced server side implementations should include:

Install fail2ban

As mentioned above you can set it up to restrict the login attempts on /admin but more globally attacks on SSH and other services.

Use of hard/soft firewall

Any server should have a software firewall in place locking down ports that should not be open, ideally a hardware firewall gives more concrete protection.

Correct permissions on folders

Configuring which folders have read/write access ideally all should be read-only accept where required.

Anti virus installation

ClamAv or any other virus install a personal favourite on windows is ESET, it’s a worthy note you should have this on your personal/work machine’s too.

Code monitors

Use of products like CodeGaurd that alert you of code changes, there’s also sucuri, this would help spot simple iframe injections or other code injection attacks where the site is kept running with an unobtrusive line of malware injected into your index.php files or other index files.

DNS level protection

Use CloudFlare for DNS level blocking and DDoS protection.

Use a load balancer

Whilst not essential this can help prevent DDoS attacks and hide your main server IP, it also gives you lots of flexibility in changing/upgrading your server.

Change SSH port

This does cut out a lot of the attacks, there’s arguments for and against this as a hacker could easily find the port, but this will prevent the thousands of automated attacks.

Disable non secure FTP

It’s not enabled by default on most linux distro’s but if you have something like cPanel installed it may be.

Jail SFTP users & Jail Apache/Nginx

Effective in locking down what users can access, and possibly preventing further access to other parts of the system.

Remove .htaccess / disable AllowOverride and put the configuration directly in the vhost file. (Apache Only)

Moving the config further up the chain makes it harder for hackers to change the site configuration in combination with the jail it would be difficult for someone to make a change to the apache configuration for the site.

Disable Postfix and other mail services and use Mailgun/Mandrill instead

Recently a server I look after was compromised and was sending out spam emails via postfix. It’s best to disable these services unless you understand properly how to secure them. Mailgun and Mandrill both have limits in place that would prevent this kind of attack.

 

Magento vs in-house solution

I recently came across a business who decided to move away from Magento to using their own in-house solution, albeit without the features of Magento, but something they are willing to invest the time and money in building into their in-house platform, their primary reason for doing this? Because they could not find a full time developer in their area and for their budgeted salary, and this they felt posed a long term business threat.

This is something many business’s consider when first starting an ecommerce shop, which platform to use, outsource or insource, opensource or bespoke?

It’s a choice that can only be worked out by looking at how much time you have, your budget, and technical resources you can allocate to a project.

I can see how some business owners might get frustrated with Magento, something which could be considered a niche area. The stats website builtwith suggests around 1% of the top 1 million websites are built with Magento, at first the seems rather pathetic but the huge variety of platforms out there makes this number pretty reasonable. Shopify for example only has half this figure or IBM Websphere a mere 0.2%. Not that being popular should be the only reason to choose a platform.

There are less developers than there are for standard PHP but it’s no different from using Ruby,Shopify or requiring an iOS developer to develop an app a specific ecommerce platform is a specialist area that’s a no brainer.

And then there’s the logic of building a complex ecommerce system in-house that only those developers know, what happens when the founding devs leave the business? Your left in an even worse situation. Having a community around your business gives you some extra security, tried and tested code, security patches, accredited professionals and documentation.

Magento is one of the few applications that gives you total control of your shop in an unprecedented way. Allowing all sorts of configurations that the initial Magento team probably never even dreamt the platform would be used for.

And here in lies the issue, when you do have a heavily customised platform especially one with 3rd party extensions, it can be become cumbersome in updating and supporting. In this case there are lots of areas of the site where products are only shown if some other conditions are met, this kind of logic on top of Magento’s default requirements for a product to be visible on the site creates a lot of support queries. As a developer you will often get the query of “why is this product not showing” and 99% of the time it’s because a field isn’t set or a in a particular scope a field isn’t set, the product is out of stock etc. Sometimes you don’t know yourself , I’ve spent countless hours looking into these type of queries. Part of the solution to that is more training for the people who use the backend and serving up a FAQ or decision tree that helps the user figure out why.

The second issue with Magento is that if you do get a reasonable amount of traffic or sales everyday, is that you either need a dedicated Magento hosting specialist or people on your team that know how to manage servers. And if you’ve gotten to that level in comes the caching headache.

But these are problems any platform faces one I feel with a small in-house team will not easily be overcome unless it’s built with a similar flexible design pattern and you have a broad skillset based team. Magento comes into it’s own when you want a site that is rich in functionality but don’t have time to develop whole eco systems to deliver that idea, like templating, multi-currency, multi scopes and hole punching, user authentication and REST Api’s and really why do it unless you have something that’s really taking ecommerce in a different direction.

What does matter is getting the configuration right from day one, finding an experienced developer or agency who has a proven track record of Magento development and scoping out a project development roadmap that looks at how the various parts of the site work together. Limit the use of 3rd party modules to only be used where it will save a substantial amount of development time and make sure they don’t interfere with any custom functionality you plan to have.

3rd party modules work best where you need core parts of functionality like payment gateways, as that’s where substantial costs can be saved not having to develop your own interface as your unlikely to want to customise these at a later date.

After that if you plan on having a lot of features for example customer points, promotions and lots of rules about how the products are displayed it may be better to size up the development costs rather than using a lot of different 3rd party modules as it can take just as much time writing the code that communicates between them.

In my view this is a much better, long term and cost effective route than going completely for a bespoke ecommerce solution.

Child products from parent ID by store

Child products from a parent ID by store

 $storeId = Mage::app()->getStore()->getStoreId();
        $coreResource = Mage::getSingleton('core/resource');
        $conn = $coreResource->getConnection('core_read');
        $query = "SELECT * FROM catalog_product_relation AS c
LEFT JOIN catalog_product_website AS w
ON c.parent_id = w.product_id
LEFT JOIN catalog_product_link AS l
ON l.product_id = w.product_id
WHERE c.child_id = $childId AND w.website_id = $storeId AND link_type_id = 4
GROUP BY parent_id";

Adding website/store filter to category product grid in Magento

It appears the Magento developers didn’t think this one through enitrely as this to me is something that should be there by standard. Not everyone has a multi-website set up and products that span two stores, yet that’s the very reason this kind of thing can be a pain to add on not just here but also in the main catalog view and in other views like reports.

website_filter

I am detailing this by editing core files, this is not the correct way and should be added via a module or local override, which in this case would be very little extra work,  but in essence the steps you need to take are modifying/overriding

Mage_Adminhtml_Block_Catalog_Category_Tab_Product

First find the function

_addColumnFilterToCollection

Add the following if statement at the top, this is checking for our column which we later add and looking up the website ID’s

 if ($column->getId() == 'website_id') {
            $this->getCollection()->joinField('websites',
                'catalog/product_website',
                'website_id',
                'product_id=entity_id',
                null,
                'left');
            return parent::_addColumnFilterToCollection($column);
        }

At the bottom of the function you need to modify the lines like so, to bring in the website names.

parent::_prepareCollection();
$this->getCollection()->addWebsiteNamesToResult();
return $this;

Next add a new function directly underneath the above, this is a custom column callback for our new column which we add next, which is simply a way of adding more criteria onto our $collection, in this case we add the store filter.

   protected function _websiteFilter($collection, $column)
    {

        if (!$value = $column->getFilter()->getValue()) {
            return $this;
        }

        $store = Mage::app()->getWebsite($value);
        $collection->addStoreFilter($value);

        return $this;
    }

Finally add the website column, with our custom filter callback

  if (!Mage::app()->isSingleStoreMode()) {
                         $this->addColumn('websites',
                                 array(
                                     'header'=> Mage::helper('catalog')->__('Websites'),
                     'width' => '100px',
                     'sortable'  => false,
                     'index'     => 'websites',
                     'type'      => 'options',
                     'options'   => Mage::getModel('core/website')->getCollection()->toOptionHash(),
                                     'filter_condition_callback' => array($this, '_websiteFilter'),
             ));
         }

And you’re done.

Module_Helper_Data not found any module magento

If you receive the Helper_Data.. not found message when trying to install a new module either manually or through the Downloader and get this message it’s simply a case of disabling Compilation mode before enabling the new module.

Quick and simple one, might be overlooked if you don’t tend to use Compilation mode, but that’s all you need to do simply disable compilation mode.

 

Magento collection filtered

Here’s a simple setup for a product collection with the following filters used in most of my modules, ideal for things like product feeds etc.

It has the following filters

– By Store
– Must have price > 0
– Must cost > 0
– Only simple products
– Visible either Catalog,Search or Both

$visibility = array(
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_BOTH,
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_IN_CATALOG,
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_IN_SEARCH
);

$storeId = 1;
$collection = Mage::getResourceModel('catalog/product_collection');
$collection->getEntity()->setStoreId($storeId);

// Reset the select.
$collection->getSelect()->reset();

// Update table name.
$reflectionMethod = new ReflectionMethod($collection, '_initSelect');
$reflectionMethod->setAccessible(true);
$reflectionMethod->invoke($collection);

$collection
 ->addAttributeToSelect('sku')
 ->addAttributeToSelect('name')
 ->addAttributeToSelect('description')
 ->addAttributeToSelect('small_image')
 ->addAttributeToSelect('manufacturer')
 ->addAttributeToSelect('msrp')
 ->addAttributeToFilter('visibility', $visibility)
 ->addAttributeToFilter('status', '1')
 ->addAttributeToFilter('type_id', array('eq' => 'simple'))
 ->addFinalPrice()
 ->setPageSize(900000)
 ->setCurPage(1);

Calculating selling price from the cost price

Recently in my price matching system I had to calculate the selling price of a product in a way that also calculates a transaction fee percentage on the final selling price without knowing the selling price, and without knowing what the transaction fee is until you’ve calculated the total price. Had fun chasing your tail yet?

You can do this in Excel with circular references but you will get circular reference warnings and the only way is to allow these warnings and an iteration limit of 100.

Stepping aside from the magic of excel I was able to work out the exact formula for this and below is a calculation for anyone else struggling to work out how to do this.

$cp = 9.56; // cost price
$d = 2.50; // delivery fee
$m = 0.15; // margin 15%
$tf = 0.019; // transaction fee

$sellingPrice = (($cp+$d)*(1+($m)))/(1-((1/6)+$tf)*(1+($m)));

$costPrice = $x / (1+(.15));

echo $sellingPrice."\n";
echo $costPrice;

Magento image not loading when using cron, only getting placeholder

I have a module which sends out an email via Mailchimp with a scheduled task for Magento’s internal cron and in doing so looks up the product image.

Mage::helper('catalog/image')->init($product, 'small_image')->resize(150));

The problem being only the placeholder was being returned when the cron job was executed. To confuse matters when I ran the script in testing from the browser it would return the correct image.

This is a nasty problem at first I thought this might be the answer Sangay details there that if your config does not specify a M,GB etc that it will return the wrong calculation, but then why the different results depending on how I ran the script?

To get to the bottom of this I ran what is being run in Mage_Catalog_Helper_Image

$model = Mage::getModel('catalog/product_image');
$model->setBaseFile($this->product->getData('small_image'))->resize(600,600);
$url = $model->saveFile()->getUrl();

PHP Fatal error:  Uncaught exception 'Varien_Exception' with message 'Memory limit has been reached.' in /lib/Varien/Image/Adapter/Gd2.php:58

Returning this shows the actual exception I did not get any Exception in the Mage exception log, and not in the host’s error log. On my part a server configuration oversight as the CLI php.ini did not have the error log set up. In any case that’s the error.

Because the value returned from _convertToByte function in Mage_Catalog_Helper_Image is -1 as the CLI php.ini sets the memory_limit to -1 (unlimited).

This could be an oversight by Magento or a design feature not to run this task on unlimited memory, personally I think it’s a bug. I modified the function to the following to set a 2G limit in this case.

   if (stripos($memoryValue, 'M') !== false) {
            return (int)$memoryValue * 1024 * 1024;
        }
        elseif (stripos($memoryValue, 'KB') !== false) {
            return (int)$memoryValue * 1024;
        }
        elseif ($memoryValue == "-1") {
            return '2147483648'; //2G
        }