Reset npm/bower permissions

Useful snippet for resetting bower/npm permissions if you get errors like this

Error: EACCES: permission denied, open '/Users/jdavey/.config/configstore/bower-github.json'
You don't have access to this file.

sudo chown -R $USER:$GROUP ~/.npm
sudo chown -R $USER:$GROUP ~/.config

You can also run this on whatever folder is causing the permissions error. This resets them to the current user.

Upgrading to Magento 2

Having had the opportunity to work on a new build using Magento 2 the architecture of the application has changed significantly, to me it has definitely increased the difficulty of getting a Magento project up and running but in return offers a more granular and organised application to work with.

With this view I think it’s unlikely that you would upgrade your site to v2. Magento have not given an upgrade path to 2.0 and I doubt they will, so it’s probably worth waiting for the opportunity to re-design the site before moving over to the new platform.

Magento have committed to 3 years from general release of 2.0 so that gives you till around 2019 to move over. In the meantime that means they will continue to provide security patches and support for the 1.x platform.

Setting different meta title,description per social network

The only way to get different title,meta responses per social media channel is to have the response set on the backend, in this case when Facebook, Pinterest and Google Plus hit the page check the user agent and display the relevant info, Twitter you can still force the text/image that’s shown, the rest rely on the open graph tags or whatever the title/description is set to. From my experience it’s impossible to set this via plugins like addthis etc.. although it may be possible through some of the API’s it’s easier to check the user agent though.

jstl example

<c:choose>
<c:when test="${fn:contains(header['User-Agent'],'Facebot') || fn:contains(header['User-Agent'],'facebookexternalhit/1.1') || fn:contains(header['User-Agent'],'facebookexternalhit/1.0')}">
<c:if test="${not empty facebookTitle}">
<c:set var="ogTitle" value="${facebookTitle}" />
</c:if>
<c:if test="${not empty facebookDesc}">
<c:set var="ogDescription" value="${facebookDesc}" />
</c:if>
<c:if test="${not empty facebookImage}">
<c:set var="ogImage" value="${facebookImage}" />
</c:if>
</c:when>
<c:when test="${fn:contains(header['User-Agent'],'pinterest/0.1 +http://pinterest.com/')}">
<c:if test="${not empty pinterestTitle}">
<c:set var="ogTitle" value="${pinterestTitle}" />
</c:if>
<c:if test="${not empty pinterestDescription}">
<c:set var="ogDescription" value="${pinterestDescription}" />
</c:if>
<c:if test="${not empty pinterestImage}">
<c:set var="ogImage" value="${pinterestImage}" />
</c:if>
</c:when>
<c:when test="${fn:contains(header['User-Agent'],'Google (+https://developers.google.com/+/web/snippet/)')}">
<c:if test="${not empty gplusTitle}">
<c:set var="ogTitle" value="${gplusTitle}" />
</c:if>
<c:if test="${not empty gplusDesc}">
<c:set var="ogDescription" value="${gplusDesc}" />
</c:if>
<c:if test="${not empty gplusImage}">
<c:set var="ogImage" value="${gplusImage}" />
</c:if>
</c:when>
<c:otherwise>
<!-- use default title value -->
</c:otherwise>
</c:choose>

PHP example

function get_meta()
{
switch (true) {
case stristr($_SERVER['HTTP_USER_AGENT'], 'Facebot') || stristr($_SERVER['HTTP_USER_AGENT'],'facebookexternalhit'):
$title = 'Facebook Title';
$description = 'Facebook Description';
break;
case stristr($_SERVER['HTTP_USER_AGENT'], 'developers.google.com'):
$title = 'Google Title';
$description = 'Google Description';
break;
case stristr($_SERVER['HTTP_USER_AGENT'], 'pinterest'):
$title = 'Pinterest Title';
$description = 'Pinterest Description';
break;
default:
$title = 'Default title';
$description = 'Default Description';
}
}

Cq5 formatting string to a number with decimal places

How to format from a string to a number, the date coming from a request param. The only thing not covered here is to catch exceptions if the parameter passed was not a number. In all other cases falls back safely.

// e.g ?myStringNum=2.44433 in request URL
String myStringNum= request.getParameter("myStringNum");
if (myStringNum!= null) {
// cast as number first and decode the string if required
    Number formattedNumber=Float.valueOf(URLDecoder.decode(request.getParameter("myStringNum"),"UTF-8"));
// formatting template , to one decimal place
	DecimalFormat df = new DecimalFormat("#.#");
// set the attribute for cq5
     request.setAttribute("formattedNumber",df.format(formattedNumber));
}

//Then in your CQ5 template add
${formattedNumber}

Enhancing Magento Security – Best practices

After watching the recent panorama documentary on the recent TalkTalk hack, it made me wonder how vulnerable many of the Magento sites where and what can be done to tighten Magento’s security.

Many opensource platforms are often hacked for example WordPress is a regular culprit , though it’s not as widespread that the core code has vulnerabilities it’s usually a rogue plugin or one that hasn’t been updated, with WordPress’s built in automatic updating it has a bit more protection.

Magento doesn’t have such frequent easy updating but recently the Magento team have released a flurry of patches, in what seems to be either that they have woken up to the fact that Magento in it’s previous incarnation had many exploits or that retailers have fed back to Magento. Whatever the case like any opensource platform continual updates are required especially in an ecommerce environment.

Magento and other ecommerce systems will be a prime target for hackers, thousands of un-encrypted customer details and in most cases it’s in-practical to encrypt them so your left with the only option of tightening up server security and the application itself.

So what steps can you take to secure Magento?

Patch/Upgrade

Install the most up to date patches
http://magento.com/security-patch

As of writing there are around 15 patches for community edition, depending on which version you have installed.

https://www.magentocommerce.com/download

Alternatively upgrade Magento to the most current release, as it includes all current patches 1.9.2.2

Upgrade to the next version 2.0
Probably not viable for most retailers just yet

PCI Compliance

Any ecommerce should be PCI compliant even if your taking payments offsite it’s worth having a PCI scanner in place. For that I would recommend using Trustwave some PayPal integrations require it anyways.

This will scan your site for common exploits, you might be surprised by what it picks up.

Trustwave

https://www.trustwave.com

Strong SSL

Ensure your using a strong cipher suite and your webserver is set up to use the correct protocols.

SSL Rating

https://www.ssllabs.com/ssltest/

Run your site through SSL Labs for any recommendations, A or above rating is recommended.

Change the admin path

A simple yet effective change to move the /admin to somewhere else, edit your config xml including the following.

<admin>
<routers>
<adminhtml>
<args>
<frontName><![CDATA[admin]]></frontName>
</args>
</adminhtml>
</routers>
</admin>

If this isn’t an option as it can break some plugins, then request your server admin add (if not already installed) a rule to fail2ban to prevent attacks on the /admin folder.

Prevent access

To magento’s directories, see a complete list in the bottom example.

directly in the vhost (Apache):

LocationMatch ^/(app/|var/) >
    Require all denied

Nginx:

location ^~ /app/ { deny all; }
location ^~ /var/ { deny all; }

Alternatively you could return a 404

location /app/                { return 404; }
location /downloader/         { return 404; }
location /errors/             { return 404; }
location /media/              { return 404; }
location /assets/             { return 404; }
location /images/             { return 404; }
location /skin/               { return 404; }
location /includes/           { return 404; }
location /lib/                { return 404; }
location /media/downloadable/ { return 404; }
location /pkginfo/            { return 404; }
location /report/config.xml   { return 404; }
location /shell/              { return 404; }
location /var/                { return 404; }

Also make sure to include any external import/export tools e.g Magmi

Turn on SSL for the admin

secure

Ideally you should have your entire site as SSL, this can affect performance on lower end servers.

Use strong passwords

Simple but obvious, this is probably the single biggest threat. Make it easy to remember but hard to guess, a password generator might not be useful here unless you use a password manager. Ideally use two-factor authentication

Two factor auth

Through this extension: http://www.xtento.com/magento-extensions/two-factor-authentication-enhanced-admin-security.html

Restrict admin by IP

Ideally this would be better sitting in the apache/nginx config, but if you don’t have that kind of access or want GUI control there is a free module.

http://www.magentocommerce.com/magento-connect/et-ip-security.html

Use a more granular admin permission module

This allows you to control in a more detailed way what each admin user can do.

http://www.aitoc.com/en/magentomods_advanced_permissions.html

Even if it’s just you managing your site, you could have a master login and editor role, to limit the use of full access.

Advanced server side

More advanced server side implementations should include:

Install fail2ban

As mentioned above you can set it up to restrict the login attempts on /admin but more globally attacks on SSH and other services.

Use of hard/soft firewall

Any server should have a software firewall in place locking down ports that should not be open, ideally a hardware firewall gives more concrete protection.

Correct permissions on folders

Configuring which folders have read/write access ideally all should be read-only accept where required.

Anti virus installation

ClamAv or any other virus install a personal favourite on windows is ESET, it’s a worthy note you should have this on your personal/work machine’s too.

Code monitors

Use of products like CodeGaurd that alert you of code changes, there’s also sucuri, this would help spot simple iframe injections or other code injection attacks where the site is kept running with an unobtrusive line of malware injected into your index.php files or other index files.

DNS level protection

Use CloudFlare for DNS level blocking and DDoS protection.

Use a load balancer

Whilst not essential this can help prevent DDoS attacks and hide your main server IP, it also gives you lots of flexibility in changing/upgrading your server.

Change SSH port

This does cut out a lot of the attacks, there’s arguments for and against this as a hacker could easily find the port, but this will prevent the thousands of automated attacks.

Disable non secure FTP

It’s not enabled by default on most linux distro’s but if you have something like cPanel installed it may be.

Jail SFTP users & Jail Apache/Nginx

Effective in locking down what users can access, and possibly preventing further access to other parts of the system.

Remove .htaccess / disable AllowOverride and put the configuration directly in the vhost file. (Apache Only)

Moving the config further up the chain makes it harder for hackers to change the site configuration in combination with the jail it would be difficult for someone to make a change to the apache configuration for the site.

Disable Postfix and other mail services and use Mailgun/Mandrill instead

Recently a server I look after was compromised and was sending out spam emails via postfix. It’s best to disable these services unless you understand properly how to secure them. Mailgun and Mandrill both have limits in place that would prevent this kind of attack.

 

Frontend Development in CQ5

I’m new to using CQ5 in this instance version 5.5, as a frontend developer I found the software a little confusing at first, mainly in working in a VM development environment like below:

dev

 

As CQ5 is platform independent it can easily be set up on Ubuntu with Apache acting as a reverse proxy to your local content repository allowing you to preview the content through your shared folder. The above can be extended to include https and the author instance.

In short Apache passes any requests to etc/project* to your localhost e.g 8080 which is serving the shared folder and so serves up your local repo.

If you want to find out more about how the repository works in terms of folder structure etc there is some useful documentation here on Adobe’s site.

Once this is set up you can get on with developing again there’s some useful guides

Component Properties and structure

Component best practices

Developing on CQ 5.6

Widget docs

All available Xtype’s

Any starting point should be copying an existing component and playing around with the attributes and settings.

JSP is server side so there’s not really the prospect of trying out your jsp without simply adding it to your component, although I did come across http://zkfiddle.org/ which does work albeit a bit slow.

If you do get an error and your not in the component.jsp you need to undo your changes and then modify/save component.js to clear any cache.

Another niggle is the VM not shutting down properly, something I’m not aware of there being a fix, which gives you a nice blank page when you fire up the author or publisher. The only solution being copying over a new instance of your VM. I did read about the Lucene indexes getting corrupt but these take hours to re-compile so it’s not worth the time unless you where really stuck (there’s a good article here on that http://www.wemblog.com/2011/12/how-to-rebuild-index-in-cq5-wem.html) and that’s if that is indeed the error.

If you have any suggestion for the above please leave a comment.

Magento vs in-house solution

I recently came across a business who decided to move away from Magento to using their own in-house solution, albeit without the features of Magento, but something they are willing to invest the time and money in building into their in-house platform, their primary reason for doing this? Because they could not find a full time developer in their area and for their budgeted salary, and this they felt posed a long term business threat.

This is something many business’s consider when first starting an ecommerce shop, which platform to use, outsource or insource, opensource or bespoke?

It’s a choice that can only be worked out by looking at how much time you have, your budget, and technical resources you can allocate to a project.

I can see how some business owners might get frustrated with Magento, something which could be considered a niche area. The stats website builtwith suggests around 1% of the top 1 million websites are built with Magento, at first the seems rather pathetic but the huge variety of platforms out there makes this number pretty reasonable. Shopify for example only has half this figure or IBM Websphere a mere 0.2%. Not that being popular should be the only reason to choose a platform.

There are less developers than there are for standard PHP but it’s no different from using Ruby,Shopify or requiring an iOS developer to develop an app a specific ecommerce platform is a specialist area that’s a no brainer.

And then there’s the logic of building a complex ecommerce system in-house that only those developers know, what happens when the founding devs leave the business? Your left in an even worse situation. Having a community around your business gives you some extra security, tried and tested code, security patches, accredited professionals and documentation.

Magento is one of the few applications that gives you total control of your shop in an unprecedented way. Allowing all sorts of configurations that the initial Magento team probably never even dreamt the platform would be used for.

And here in lies the issue, when you do have a heavily customised platform especially one with 3rd party extensions, it can be become cumbersome in updating and supporting. In this case there are lots of areas of the site where products are only shown if some other conditions are met, this kind of logic on top of Magento’s default requirements for a product to be visible on the site creates a lot of support queries. As a developer you will often get the query of “why is this product not showing” and 99% of the time it’s because a field isn’t set or a in a particular scope a field isn’t set, the product is out of stock etc. Sometimes you don’t know yourself , I’ve spent countless hours looking into these type of queries. Part of the solution to that is more training for the people who use the backend and serving up a FAQ or decision tree that helps the user figure out why.

The second issue with Magento is that if you do get a reasonable amount of traffic or sales everyday, is that you either need a dedicated Magento hosting specialist or people on your team that know how to manage servers. And if you’ve gotten to that level in comes the caching headache.

But these are problems any platform faces one I feel with a small in-house team will not easily be overcome unless it’s built with a similar flexible design pattern and you have a broad skillset based team. Magento comes into it’s own when you want a site that is rich in functionality but don’t have time to develop whole eco systems to deliver that idea, like templating, multi-currency, multi scopes and hole punching, user authentication and REST Api’s and really why do it unless you have something that’s really taking ecommerce in a different direction.

What does matter is getting the configuration right from day one, finding an experienced developer or agency who has a proven track record of Magento development and scoping out a project development roadmap that looks at how the various parts of the site work together. Limit the use of 3rd party modules to only be used where it will save a substantial amount of development time and make sure they don’t interfere with any custom functionality you plan to have.

3rd party modules work best where you need core parts of functionality like payment gateways, as that’s where substantial costs can be saved not having to develop your own interface as your unlikely to want to customise these at a later date.

After that if you plan on having a lot of features for example customer points, promotions and lots of rules about how the products are displayed it may be better to size up the development costs rather than using a lot of different 3rd party modules as it can take just as much time writing the code that communicates between them.

In my view this is a much better, long term and cost effective route than going completely for a bespoke ecommerce solution.

Adding website/store filter to category product grid in Magento

It appears the Magento developers didn’t think this one through enitrely as this to me is something that should be there by standard. Not everyone has a multi-website set up and products that span two stores, yet that’s the very reason this kind of thing can be a pain to add on not just here but also in the main catalog view and in other views like reports.

website_filter

I am detailing this by editing core files, this is not the correct way and should be added via a module or local override, which in this case would be very little extra work,  but in essence the steps you need to take are modifying/overriding

Mage_Adminhtml_Block_Catalog_Category_Tab_Product

First find the function

_addColumnFilterToCollection

Add the following if statement at the top, this is checking for our column which we later add and looking up the website ID’s

 if ($column->getId() == 'website_id') {
            $this->getCollection()->joinField('websites',
                'catalog/product_website',
                'website_id',
                'product_id=entity_id',
                null,
                'left');
            return parent::_addColumnFilterToCollection($column);
        }

At the bottom of the function you need to modify the lines like so, to bring in the website names.

parent::_prepareCollection();
$this->getCollection()->addWebsiteNamesToResult();
return $this;

Next add a new function directly underneath the above, this is a custom column callback for our new column which we add next, which is simply a way of adding more criteria onto our $collection, in this case we add the store filter.

   protected function _websiteFilter($collection, $column)
    {

        if (!$value = $column->getFilter()->getValue()) {
            return $this;
        }

        $store = Mage::app()->getWebsite($value);
        $collection->addStoreFilter($value);

        return $this;
    }

Finally add the website column, with our custom filter callback

  if (!Mage::app()->isSingleStoreMode()) {
                         $this->addColumn('websites',
                                 array(
                                     'header'=> Mage::helper('catalog')->__('Websites'),
                     'width' => '100px',
                     'sortable'  => false,
                     'index'     => 'websites',
                     'type'      => 'options',
                     'options'   => Mage::getModel('core/website')->getCollection()->toOptionHash(),
                                     'filter_condition_callback' => array($this, '_websiteFilter'),
             ));
         }

And you’re done.

Optimising WordPress for PageSpeed Insights

Recently I was commissioned to optimise a WordPress site on behalf of an SEO agency. This usually consists of reports from Google’s pagespeed insights and there’s a number of things you can do to increase the speed.

pagespeed

Before you start you should first check that the foundations are correct.

Enqueue

For example all JS and CSS files are properly enqueued in WordPress we will later need to work on minifying and combining these into one file.

HTML Structure

Ensure important page elements are near the top, move your html around if required and ensure that your CSS files are ordered in the way the page loads you might want to break the css up into two files. The first loading important structure e.g header, body content etc.. The second appearing in the footer loading the remaining part of the site.

Onto the optimisation

Optimise images

The next thing you want to look at is image optimisation there’s a plugin you can install that optimises all image uploads as you upload them which in turn utilises some command line tools to compress images e.g optipng and jpegoptim. You also need to run these tools on the existing image library.

Set an expiry on assets through nginx or apache or alternatively just use Cloudflare or any CDN of your choice.

Minify HTML

HTML Minify https://github.com/cferdinandi/html-minify does a decent job of minimising your html.

Minify CSS/JS

You then need to install a plugin to combine your css/js and again work on the order here to ensure the right delivery.

Blocking

Fonts are often blocking resources, consider using Google’s Font API instead and anything else that isn’t critical either defer or move to the bottom of the page.

A lot of the above can also be achieved in part of in full by Google’s pagespeed module you’ll achieve better results though by doing each part onsite.

Image resizing

If you have any scripts that work with images e.g sizing images on the fly or creating thumbnails and have a very large amount of images either through the creation of different sizes or in total watch out for anything that reads the folder they are contained in as this can put a serious performance hit on your server.

Magento collection filtered

Here’s a simple setup for a product collection with the following filters used in most of my modules, ideal for things like product feeds etc.

It has the following filters

– By Store
– Must have price > 0
– Must cost > 0
– Only simple products
– Visible either Catalog,Search or Both

$visibility = array(
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_BOTH,
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_IN_CATALOG,
    Mage_Catalog_Model_Product_Visibility::VISIBILITY_IN_SEARCH
);

$storeId = 1;
$collection = Mage::getResourceModel('catalog/product_collection');
$collection->getEntity()->setStoreId($storeId);

// Reset the select.
$collection->getSelect()->reset();

// Update table name.
$reflectionMethod = new ReflectionMethod($collection, '_initSelect');
$reflectionMethod->setAccessible(true);
$reflectionMethod->invoke($collection);

$collection
 ->addAttributeToSelect('sku')
 ->addAttributeToSelect('name')
 ->addAttributeToSelect('description')
 ->addAttributeToSelect('small_image')
 ->addAttributeToSelect('manufacturer')
 ->addAttributeToSelect('msrp')
 ->addAttributeToFilter('visibility', $visibility)
 ->addAttributeToFilter('status', '1')
 ->addAttributeToFilter('type_id', array('eq' => 'simple'))
 ->addFinalPrice()
 ->setPageSize(900000)
 ->setCurPage(1);