Understanding CORS and Pre-Flight

Ok so this isn’t a new thing, but most developers have been thwarted by this at some point for anyone finding this I’m going to explain it in plain english as I’ve lost count of the times I need to explain¬†these issues to¬†someone, or yourself ūüôā

cross-over

  1. CORS is a browser thing, that means using Postman, Curl whatever other means your doing a http request your only going to experience the problem in a browser as other tools do not respect CORS.
  2. Simply put it’s a set of rules that are checked by the browser in the response headers of a request, these rules determine which websites are allowed access to the¬†resource.
  3. It does not care what framework (Angular,React,Jquery) or Vanilla JS your using to make your request, CORS issues are generally down to how it’s configured on the resource (the server) your querying.

So when making a request the browser checks what headers the server returns, this can happen before the actual request is sent in something called a pre-flight request (more on that later).

For example I make a POST request from website.co.uk to domain.co.uk and the server returns

Access-Control-Allow-Origin: http://www.domain.co.uk

In this case only domain.co.uk is allowed to access this resource.
Another example would be that it returns:

Access-Control-Allow-Origin: *

This means anyone can access this resource. Read more below for info on setting these headers and what a pre-flight request it.

The other typical CORS issue is related to the methods

response

Here only the POST method is allowed, trying to PUT or DELETE will not work, you need to configure your server (see below) on how to enable this.

What exactly is CORS for?

CORS protects the end user the client YOU from a third party making requests on your behalf that originated from a different website than the one your on it does this by checking the Origin of the request to validate where it came from, if it’s still not making sense then, there’s some more detailed examples on Stackoverflow

What is a Pre-flight request?

So many posts i’ve seen where Angular, Jquery etc describe this as a feature of the library, wrong again it’s the browser doing this not your JS library!

preflght

Pre-flight is when the browser sends an OPTIONS request before your actual request gets sent, which the server typically responds with the options available e.g POST, PUT etc.. along with that in the headers will be Access Control rules.

Here’s an example

pre-flight

As you can see the response details what methods, headers and origins are allowed in the response headers.

If your website is not in the Access-Control-Allow-Origin: part or it’s not * that means your not allowed and it will not make your actual request and instead will return an error like:

Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource.

You might be asking, why don’t you see pre-flight on my request? Well this only happens for what is deemed a non-standard request.

The first part of the spec explains this https://www.w3.org/TR/cors/

It gets slightly more complicated if the resource author wants to be able to handle cross-origin requests using methods other than simple methods. In that case the author needs to reply to a preflight request that uses the OPTIONS method and then needs to handle the actual request that uses the desired method

Simple is GET, HEAD, POST without additional headers so any header modification will result in a pre-flight request.

Configuring your server to allow CORS

By default most server software e.g Apache or similar will block cross site requests, and you will need to modify the rules in your .htaccess or .conf

Apache example using .htaccess

<IfModule mod_headers.c>
    Header add Access-Control-Allow-Origin "*"
    Header add Access-Control-Allow-Headers "accept, origin, x-requested-with, content-type"
    Header add Access-Control-Allow-Methods "PUT, GET, POST, DELETE, OPTIONS"
</IfModule>

Typically the above will resolve CORS issues whilst using JS ajax requests, be careful as to what methods you need to allow.

But what are the consequences of doing this?, well there’s a good reason it’s not allowed by default¬†see above “What is exactly is CORS for?”. Short answer is that you would generally have other protections in place¬†for example a custom header that is submitted along with the request that only the client and the server know and you would also most likely wan’t to allow access to only a select few e.g

Access-Control-Allow-Origin: http://www.webapp.com, http://www.webclient.com

Using the above safegaurds it’s generally ok¬†to enable it.

Migrating Angular1.x to 2 with TypeScript

Whichever release candidate your using, or the final release. This a quick heads up guide on how to port over some simple¬†tasks, something the Hero example app doesn’t show in full.

Upgrade path

I tried using the upgrader but in reality if your strapped for time I think using that is a very lengthy time consuming process and only appeals to projects that are running continuous delivery (that’s if you can get it set up in the first place).

As your adding the overhead of creating the upgrade/dowgrade code and then after that having to refactor.

I opted for a re-build of the current app refactoring service by service etc, it also allows you to make some architectural changes to the api of your app along the way, how much dependant on the time you have.

The biggest challenges are where the previous app uses $q, $watch and¬†where you’ve used angular’s built in methods like angular.extend some can be fairly quickly swapped for lodash alternatives though or another framework, but overall it’s the sheer amount of typing you will need to do that gets tedious especially for Typescript.

Loading config

In Angular 1 you might have had some app wide config going on that sat in the main app.js e.g

angular.module('myApp')

    .config(function (appConfig) {
//configure stuff here

And also through a constant

angular.module('myApp').constant('appConfig', {
    "version": "1.0"
})

In Angular2 there’s two ways to solve this problem one is to provide the value on the root level app component like so:

providers: [
        { provide: VERSION, useValue: '1.0' }
    ],

You can then get that value by doing

var version = injectorValue.get(VERSION);

That’s great for small one liner configurations but a better way would be to load it from a local file. We can do that with a simple service.

import { Injectable } from [email protected]/core';
var config = require('./env.json');

@Injectable()
export class AppConfigService {

    private _config: Object;

    constructor() {
        this._config = config;
    }

    get(key: any) {
        return this._config[key];
    };
    getAll() {
        return this._config;
    }
}

In this case the env.json is generated externally from a gulp task. If you wanted to load an external file over http you could do something like below, here we would need to provide the endPoint value as we did above to the main app component.

providers: [
        { provide: endPoint, useValue: 'http://127.0.0.1/api' }
    ],
import {Http} from [email protected]/http';
import {Injectable, Inject} from [email protected]/core';

@Injectable()
export class AppConfig {

    constructor(private http: Http, 
                @Inject('AppConfig') private endPoint: string) { }

    getConfig(): {
        return this.http.get(`${this.endPoint}/`)
            .then((response) => {
                //do something with the response
            });
    } 
    // the rest of the code...
}

2. Promises

I won’t go into too much detail on this as there are some excellent guides around but where your code uses $q in Angular1 you will need to convert that to using an ES6 promise.

var promise = $q.when([]);
promise = promise.then(function() {
                        return getSomeData();
                    });
promise = promise.then(function() {
// do something else
});

In ES6 we could convert it like this

return new Promise((resolve,reject) => {
    getSomeData().then((result) => {
       //do something with result
       resolve(result);
}).catch((err) => {
       reject(err);
}
)};

or chain them together

let promise = new Promise((resolve,reject) => {
    getSomeData().then((result) => {
       //do something with result
       resolve(result);
}).catch((err) => {
       reject(err);
}
)};
promise.then((res) => {
   //do another thing
});

3. using $this _self etc

In most cases you should avoid doing that anyways either using .bind(this) although there is the odd case you need to, but with ES6 it makes it much easier to avoid through using arrows. In simple terms it’s shorthand for a function() {}, but the added feature of keeping the context of this to the parent. E.g

var $this = this;
Catalogue.getChildren(catId)
            .then(function(categories) {
                availableFilters.subCategories = categories;
            }).then(function(categories) {
                return CatalogueRepository.getFilters($this.getCriteria());
            })

Using arrows we no longer need $this

Catalogue.getChildren(catId)
            .then((categories) => {
                availableFilters.subCategories = categories;
            }).then(function(categories) => {
                return CatalogueRepository.getFilters(this.getCriteria());
            });

4. Angular forEach,copy etc..

for(var data in key) {
   key //key
   data[key] //value
}

Whilst there is vanilla js alternatives for these the quickest solution is to just import lodash or other library, lodash has _.deepClone _.forEach I use it like so after the imports.

var _ = require('lodash');

5. Window

For RC4 Provide window in your root component

providers: [
        { provide: Window, useValue: window }
    ],

In your component constructor use

constructor(
     @Inject(Window) private _window: Window
)

There may be a better way of doing the above, please let me know if you do it a different way.

6. Using pipe inside a service

Import the pipe and then call it like below.

import {TranslatePipe} from "../pipes/translate.pipe";
constructor(private translate: TranslatePipe) {
}
myFunction() {
this.translate.transform('some text');
}

7. Global variables

If for example your using another framework or something else in the global namespace you can use this to avoid TS errors, e.g using Ionic with Cordova.

declare var cordova: any;

@Injectable()
export class MyService {}

8. Code organisation

The sheer amount of imports can become a bit bewildering, to overcome this use barrel files, you have a bunch of services in /services then create a file inside there called index.ts

Inside this put:

export {appService} from "./appService";
export {configService} from "./configService";
export {dbService} from "./dbService";
export {UserDb} from "./userDb";

You can import it into your components like so:

import * as coreServices from "./services/";

You can provide or bootstrap them using [coreServices.UserDb,coreServices.dbService] etc..

One disadvantage to this is that you can’t quickly see the list of services upfront, so it’s best to group them for example /db /file /network and have the corresponding index files.

import * as network from "./network/";
import * as db from "./db/";
import * as files from "./file/";

This is much more readable.

9. Data in templates

If you need to load some data before it’s available to the template in Angular2 and want to avoid the undefined errors, there are a few ways of solving this.

1. ngOnInit( this.state = getMyState(); );
2. “The Elvis Operator” before your data e.g {{state?.name}}
3 ngIf e.g *ngIf=”state”
4. construct your model which will give you an empty object. this.state = new myModel();

10. Global services

In angular1 every service was a singleton, v2 has the same concept if you inject the service at the app level, but you can also provide it separately to to individual areas of the app through the providers:[], read more about that here

That’s it for now, let me know if you have any other simple things.

 

Hiding metabox from backend in WordPress

In my case hiding themify and yoast seo metabox from the backend for pages (this only works for pages, you will need to change that to whatever type of post it is see here for more) , use below snippet.

function hide_meta() {
    global $post;
    if ( get_page_template_slug($post->id) == 'services-new.php') {
        remove_meta_box('wpseo_meta', 'page', 'normal');
        remove_meta_box('themify-meta-boxes', 'page', 'normal');
    };
}
add_action( 'admin_notices', 'hide_meta' );

Reset npm/bower permissions

Useful snippet for resetting bower/npm permissions if you get errors like this

Error: EACCES: permission denied, open '/Users/jdavey/.config/configstore/bower-github.json'
You don't have access to this file.

sudo chown -R $USER:$GROUP ~/.npm
sudo chown -R $USER:$GROUP ~/.config

You can also run this on whatever folder is causing the permissions error. This resets them to the current user.

Upgrading to Magento 2

Having had the opportunity to work on a new build using Magento 2 the architecture of the application has changed significantly, to me it has definitely increased the difficulty of getting a Magento project up and running but in return offers a more granular and organised application to work with.

With this view I think it’s unlikely that you would upgrade your site to v2. Magento have not given an upgrade path to 2.0 and I doubt they will, so it’s probably worth waiting for the opportunity to re-design the site before moving over to the new platform.

Magento have committed to 3 years from general release of 2.0 so that gives you till around 2019 to move over. In the meantime that means they will continue to provide security patches and support for the 1.x platform.

Setting different meta title,description per social network

The only way to get different title,meta responses per social media channel is to have the response set on the backend, in this case when Facebook, Pinterest and Google Plus hit the page check the user agent and display the relevant info, Twitter you can still force the text/image that’s shown, the rest rely on the open graph tags or whatever the title/description is set to. From my experience it’s impossible to set this via plugins like addthis etc.. although it may be possible through some of the API’s it’s easier to check the user agent though.

jstl example

<c:choose>
<c:when test="${fn:contains(header['User-Agent'],'Facebot') || fn:contains(header['User-Agent'],'facebookexternalhit/1.1') || fn:contains(header['User-Agent'],'facebookexternalhit/1.0')}">
<c:if test="${not empty facebookTitle}">
<c:set var="ogTitle" value="${facebookTitle}" />
</c:if>
<c:if test="${not empty facebookDesc}">
<c:set var="ogDescription" value="${facebookDesc}" />
</c:if>
<c:if test="${not empty facebookImage}">
<c:set var="ogImage" value="${facebookImage}" />
</c:if>
</c:when>
<c:when test="${fn:contains(header['User-Agent'],'pinterest/0.1 +http://pinterest.com/')}">
<c:if test="${not empty pinterestTitle}">
<c:set var="ogTitle" value="${pinterestTitle}" />
</c:if>
<c:if test="${not empty pinterestDescription}">
<c:set var="ogDescription" value="${pinterestDescription}" />
</c:if>
<c:if test="${not empty pinterestImage}">
<c:set var="ogImage" value="${pinterestImage}" />
</c:if>
</c:when>
<c:when test="${fn:contains(header['User-Agent'],'Google (+https://developers.google.com/+/web/snippet/)')}">
<c:if test="${not empty gplusTitle}">
<c:set var="ogTitle" value="${gplusTitle}" />
</c:if>
<c:if test="${not empty gplusDesc}">
<c:set var="ogDescription" value="${gplusDesc}" />
</c:if>
<c:if test="${not empty gplusImage}">
<c:set var="ogImage" value="${gplusImage}" />
</c:if>
</c:when>
<c:otherwise>
<!-- use default title value -->
</c:otherwise>
</c:choose>

PHP example

function get_meta()
{
switch (true) {
case stristr($_SERVER['HTTP_USER_AGENT'], 'Facebot') || stristr($_SERVER['HTTP_USER_AGENT'],'facebookexternalhit'):
$title = 'Facebook Title';
$description = 'Facebook Description';
break;
case stristr($_SERVER['HTTP_USER_AGENT'], 'developers.google.com'):
$title = 'Google Title';
$description = 'Google Description';
break;
case stristr($_SERVER['HTTP_USER_AGENT'], 'pinterest'):
$title = 'Pinterest Title';
$description = 'Pinterest Description';
break;
default:
$title = 'Default title';
$description = 'Default Description';
}
}

Cq5 formatting string to a number with decimal places

How to format from a string to a number, the date coming from a request param. The only thing not covered here is to catch exceptions if the parameter passed was not a number. In all other cases falls back safely.

// e.g ?myStringNum=2.44433 in request URL
String myStringNum= request.getParameter("myStringNum");
if (myStringNum!= null) {
// cast as number first and decode the string if required
    Number formattedNumber=Float.valueOf(URLDecoder.decode(request.getParameter("myStringNum"),"UTF-8"));
// formatting template , to one decimal place
	DecimalFormat df = new DecimalFormat("#.#");
// set the attribute for cq5
     request.setAttribute("formattedNumber",df.format(formattedNumber));
}

//Then in your CQ5 template add
${formattedNumber}

Enhancing Magento Security – Best practices

After watching the recent panorama documentary on the recent TalkTalk hack, it made me wonder how vulnerable many of the Magento sites where and what can be done to tighten Magento’s security.

Many opensource platforms are often hacked for example WordPress is a regular culprit , though it’s not as widespread that the core code has vulnerabilities it’s usually a rogue plugin or one that hasn’t been updated, with WordPress’s built in automatic updating it has a bit more protection.

Magento doesn’t have such frequent easy updating but recently the Magento team have released a flurry of patches, in what seems to be either that they have woken up to the fact that Magento in it’s previous incarnation had many exploits or that retailers have fed back to Magento. Whatever the case like any opensource platform continual updates are required especially in an ecommerce environment.

Magento and other ecommerce systems will be a prime target for hackers, thousands of un-encrypted customer details and in most cases it’s in-practical to encrypt them so your left with the only option of tightening up server security and the application¬†itself.

So what steps can you take to secure Magento?

Patch/Upgrade

Install the most up to date patches
http://magento.com/security-patch

As of writing there are around 15 patches for community edition, depending on which version you have installed.

https://www.magentocommerce.com/download

Alternatively upgrade Magento to the most current release, as it includes all current patches 1.9.2.2

Upgrade to the next version 2.0
Probably not viable for most retailers just yet

PCI Compliance

Any ecommerce should be PCI compliant even if your taking payments offsite it’s worth having a PCI scanner in place. For that I would recommend using Trustwave some PayPal integrations require it anyways.

This will scan your site for common exploits, you might be surprised by what it picks up.

Trustwave

https://www.trustwave.com

Strong SSL

Ensure your using a strong cipher suite and your webserver is set up to use the correct protocols.

SSL Rating

https://www.ssllabs.com/ssltest/

Run your site through SSL Labs for any recommendations, A or above rating is recommended.

Change the admin path

A simple yet effective change to move the /admin to somewhere else, edit your config xml including the following.

<admin>
<routers>
<adminhtml>
<args>
<frontName><![CDATA[admin]]></frontName>
</args>
</adminhtml>
</routers>
</admin>

If this isn’t an option as it can break some plugins, then request your server admin add (if not already installed) a rule to fail2ban to prevent attacks on the /admin folder.

Prevent access

To magento’s directories, see a complete list in the bottom example.

directly in the vhost (Apache):

LocationMatch ^/(app/|var/) >
    Require all denied

Nginx:

location ^~ /app/ { deny all; }
location ^~ /var/ { deny all; }

Alternatively you could return a 404

location /app/                { return 404; }
location /downloader/         { return 404; }
location /errors/             { return 404; }
location /media/              { return 404; }
location /assets/             { return 404; }
location /images/             { return 404; }
location /skin/               { return 404; }
location /includes/           { return 404; }
location /lib/                { return 404; }
location /media/downloadable/ { return 404; }
location /pkginfo/            { return 404; }
location /report/config.xml   { return 404; }
location /shell/              { return 404; }
location /var/                { return 404; }

Also make sure to include any external import/export tools e.g Magmi

Turn on SSL for the admin

secure

Ideally you should have your entire site as SSL, this can affect performance on lower end servers.

Use strong passwords

Simple but obvious, this is probably the single biggest threat. Make it easy to remember but hard to guess, a password generator might not be useful here unless you use a password manager. Ideally use two-factor authentication

Two factor auth

Through this extension: http://www.xtento.com/magento-extensions/two-factor-authentication-enhanced-admin-security.html

Restrict admin by IP

Ideally this would be better sitting in the apache/nginx config, but if you don’t have that kind of access or want GUI¬†control there is a free module.

http://www.magentocommerce.com/magento-connect/et-ip-security.html

Use a more granular admin permission module

This allows you to control in a more detailed way what each admin user can do.

http://www.aitoc.com/en/magentomods_advanced_permissions.html

Even if it’s just you managing your site, you could have a master login and editor role, to limit the use of full access.

Advanced server side

More advanced server side implementations should include:

Install fail2ban

As mentioned above you can set it up to restrict the login attempts on /admin but more globally attacks on SSH and other services.

Use of hard/soft firewall

Any server should have a software firewall in place locking down ports that should not be open, ideally a hardware firewall gives more concrete protection.

Correct permissions on folders

Configuring which folders have read/write access ideally all should be read-only accept where required.

Anti virus installation

ClamAv or any other virus install a personal favourite on windows is ESET, it’s a worthy note you should have this on your personal/work machine’s too.

Code monitors

Use of products like CodeGaurd that alert you of code changes, there’s also sucuri, this would help spot simple iframe injections or other code injection attacks where the site is kept running with an unobtrusive line of malware injected into your index.php files or other index files.

DNS level protection

Use CloudFlare for DNS level blocking and DDoS protection.

Use a load balancer

Whilst not essential this can help prevent DDoS attacks and hide your main server IP, it also gives you lots of flexibility in changing/upgrading your server.

Change SSH port

This does cut out a lot of the attacks, there’s arguments for and against this as a hacker could easily find the port, but this will prevent the thousands of automated attacks.

Disable non secure FTP

It’s not enabled by default on most linux distro’s but if you have something like cPanel installed it may be.

Jail SFTP users & Jail Apache/Nginx

Effective in locking down what users can access, and possibly preventing further access to other parts of the system.

Remove .htaccess / disable AllowOverride and put the configuration directly in the vhost file. (Apache Only)

Moving the config further up the chain makes it harder for hackers to change the site configuration in combination with the jail it would be difficult for someone to make a change to the apache configuration for the site.

Disable Postfix and other mail services and use Mailgun/Mandrill instead

Recently a server I look after was compromised and was sending out spam emails via postfix. It’s best to disable these services unless you understand properly how to secure them. Mailgun and Mandrill both have limits in place that would prevent this kind of attack.

 

Frontend Development in CQ5

I’m new to using CQ5 in this instance version 5.5, as a frontend developer I found the software a little confusing at first, mainly in working in a VM development environment like below:

dev

 

As CQ5 is platform independent it can easily be set up on Ubuntu with Apache acting as a reverse proxy to your local content repository allowing you to preview the content through your shared folder. The above can be extended to include https and the author instance.

In short Apache passes any requests to etc/project* to your localhost e.g 8080 which is serving the shared folder and so serves up your local repo.

If you want to find out more about how the repository works in terms of folder structure etc there is some useful documentation here¬†on Adobe’s site.

Once this is set up you can get on with developing again there’s some useful guides

Component Properties and structure

Component best practices

Developing on CQ 5.6

Widget docs

All available Xtype’s

Any starting point should be copying an existing component and playing around with the attributes and settings.

JSP is server side so there’s not really the prospect of trying out your jsp without simply adding it to your component, although I did come across¬†http://zkfiddle.org/ which does work albeit a bit slow.

If you do get an error and your not in the component.jsp you need to undo your changes and then modify/save component.js to clear any cache.

Another niggle is the VM not shutting down properly, something I’m not aware of there being a fix, which gives you a nice blank page when you fire up the author or publisher. The only solution being copying over a new instance of your VM. I did read about the Lucene indexes getting corrupt but these take hours to re-compile so it’s not worth the time unless you where really stuck (there’s a good article here on that¬†http://www.wemblog.com/2011/12/how-to-rebuild-index-in-cq5-wem.html) and that’s if that is indeed the error.

If you have any suggestion for the above please leave a comment.