Open Source Guilt

For the past few years I've "maintained" two very popular PHP libraries; an OAuth 2 Server and an OAuth 2 Client.

I've willingly and happily poured hours of my life into both projects. This was easily done when I worked for the University of Lincoln as both libraries were outputs of a project that I worked on. After leaving the university I moved to London and my life "got flipped-turned upside down" (as Will Smith once put it) which naturally resulted in a reduction in the number of commits that went into the projects.

This didn't concern me too much though; as far as I was concerned at the time the code worked, it was documented and I could point to a number of blog posts with solutions to common problems. I received numerous emails from other developers thanking me for my work on the projects and a number of developers in the PHP community that I respected promoted the library, invited me to talk on podcasts and offered me freelance opportunities to implement the libraries into their codebases. I was happy.

The second half of 2013 I reworked some of the server code which was quickly becoming the more popular of the two libraries and I released version 3 which improved the library's compliance with the OAuth 2.0 specification and the code itself was much cleaner. I tagged version 3, and I was again happy.

But I was lazy. I didn't update the documentation and I didn't update my blog posts which had example code implementations. Github issues started to appear asking for help, but some very kind developers who were interested in the project helped me answer them. The emails changed from thanks and congratulations to calls for help; and at one point I was receiving upwards of 30 emails a week.

Late 2013 I started working on version 4 to correct some pet-peeves I had with the code. I had become a benevolent dictator of the project and rejected numerous improvements to the existing codebase. I did my best with the emails piling up in my inbox but I also ignored many. My focus was only on the new version, because to me I knew it was better and I was able to easily implement it and I became frustrated by people who asked for help suggesting they just read the code and work it out.

During this time (from about November to March this year) I feel that on reflection I was fundamentally depressed and fed up. I felt terribly burnt out and couldn't even comprehend looking at a computer screen once I got home from work, I was unhappy with where I was living and I felt immense pressure because so many other developers were depending on me for help working with my code. Some people started to call me out publicly - and rightly so, others were rude and hurtful towards to me and I'm ever grateful to Phil Sturgeon for shooting them down for being arseholes. I'd hit a rut and I needed to get out of it.

Change started earlier this year when I moved house, and at the end of May I left my old job. I took a break between starting a new job, and I started exercising. I improved my diet, made some new friends and found enjoyment in coding again.

However I still had the issue of my projects. Others had worked on the client library and improved it, but my server library had got to about 35 open issues (most of which concerned the lack of documentation) and I'd not commented on them in weeks. I'd missed my promises to others of getting some examples out and actually documenting the current stable version. Another OAuth 2 server project also overtook mine in popularity (and rightly so).

About a fortnight ago I received an email from a developer which immensely upset me; he wasn't rude or angry but expressed his immense disappointment in me. I knew at that point that I had to get my act into gear and stop avoiding this elephant in the room.

And I've really tried. I've polished off version 4, merged it into the develop branch and finally written some goddamn documentation for it. As soon as that is finished (including an upgrade guide from version 3) I'm going to merge everything into master. Then I'm going to sort out the client library and bring that up to spec.

"With great power comes great responsibility"

Releasing open source projects is a great feeling however there are a number of considerations one should bear in mind:

  1. Actions have consequences - consciously making the decision to put something you've made on the web with an open source license with the expectation that others may find it useful has a side effect; other developers may come to you with questions, problems, and security concerns. You have a responsibility to do your best to help them out if you care about the integrity, security and reputation of your project.
  2. People want to help - others may come to you with suggestions to improve or change the codebase. Sometimes these changes can be great and you'll learn from them (I certainly have). Other times the change might be a good one but it may not fit in with your coding style, your future plans of the project or the code layout, and you should help the person making the change adapt it as necessary. Finally some suggestions may not be appropriate and you should politely explain why and offer suggestions as to how they can make more useful improvement.
  3. Your personal reputation is on the line - your name and email address will generally be embedded somewhere in the codebase, in git commit logs and in pretty graph visualisations on Github. This includes when you interact with others in bug trackers, and emailing others directly (nothing can stop them taking a screenshot and posting it on Twitter). If you act like an arsehole people will find it and call you out.
  4. Popular open source projects work well when the authors are using the project regularly themselves - one of the reasons why I have continued to work on the OAuth Server project more than the others is because I'm regularly implementing it in my personal projects or as part of my day to day job.

Based on my experiences in recent months I'm going to try the following to make my life easier and happier:

  1. I'm going to set out clear roadmaps for the project and more openly invite others to contribute to features.
  2. Document answers to the most common questions and problems so that I can point others to them in the future.
  3. Encourage people to ask questions - whilst this might seem counterproductive to my bulging inbox problem, I'm going to ask people to contact The PHP League's mailing list rather than contact me directly; this way multiple people can contribute answers.
  4. Be more honest about my availability to work on new features.

Deployment at Pollen

During my first few weeks at Yuza I've been working on one of our latest projects called Pollen which advances app developers' sales income every 7 days (rather than waiting up to 60+ days for Apple and Google to pay out).

The platform is hosted on AWS and I've spent some time perfecting our initial deployment process so that we can deploy safely and securely throughout the day.

This is our current setup:

  1. A developer works on a project locally, using provisioned Vagrant boxes which emulate the live environment.
  2. Code is merged into the develop branch using pull requests on Github.
  3. Our Jenkins box runs unit and behavioural tests against the develop branch when a change occurs.
  4. If everything is sound the code is merged into the master branch by Jenkins.
  5. Another Jenkins job watches for changes to the master branch. This job runs futher sanity checks, then pulls down dependencies, minifies assets and creates an archive which is pushed to an S3 bucket.
  6. Jenkins then tells AWS to spin up a new instance in a auto-scaling group (specific to the application type). The auto-scaling group uses a launch configuration which has the following "user-data" script which is executed when the instance spins up:The script installs Ansible and some other basic system utitilies, and automagically sets the hostname of the box using the AWS API. It also downloads pre-made SSH credentials which have specifc permissions to read certain Github repositories, one of which is our main devops repository. In this codebase there are shell scripts which will be executed depending on the hostname of the instance. These shell scripts will finalise the provisioning of the instance.
  7. When the instance is provisioned it downloads the latest code from the archive on S3, and is finally added to the load balancer. Jenkins will one by one terminate any other instances in the auto-scaling group which will in turn spin up new instances and eventually all instances in the group are running the latest codebase and setup.

These processes allows deploys to happen in a little under 5 minutes, and we'be designed all applications to be stateless so that they can be elastically scaled up and down as required.

For now this setup works well for now however there are some things we'd like to change.

Currently each new instance has to be provisioned from scratch which can be really slow if there is lots to install and setup; my plan is to get to grips with Packer. This will allow us to bake AWS AMI images seperately with all the appropriate software installed and configured. New instances will use these AMIs when they spin up, and so all that is required is to download the latest code archive from S3. Hopefully this should reduce the deploy time to no more than a minute or two.

I'm also going to keep my eye on Terraform which landed last week and is basically a way of describing a provider-agnositc cloud setup. This will then allow us to keep our code, provisioning scripts and cloud setup all in version control.

It's worth spending time examining and refining your deployment setup; it can significantly reduce onboarding time for new developers joining your team, and automating your deployment process improves productivity by reducing the number of repetative tasks required to ship your code.

Using Ansible with a bastion SSH host

A common network security design pattern is to prevent any connections to your application servers from outside of their private subnet, and then using a bastion host hosted in a DMZ to selectively whitelist traffic to the servers.

We have such a setup for one of our server pools through which we only allow SSH traffic from specific IP addresses. These servers are also provisioned via Ansible which programatically configures servers via SSH.

Due to our bastion host setup Ansible is unable to talk directly to our application servers so I had to find a way to proxy SSH connections through the bastion host.

I love using Ansible for creating simple tasks to run, for example flushing memcache to clear caches. Running with this example this is my Ansible structure:


The tasks/ script looks like this:

ssh-add ${DEPLOY_KEYS_DIR}/memcache-servers.pem

ansible-playbook -i ansible/hosts.ini -u ansible ansible/tasks/restart-memcache.yml -v

It is called from the root devops folder and first adds the servers SSH key too the SSH agent and then calls the restart-memcache.yml playbook which in turn includes the memcache role restart.yml playbook (as well as other performing other tasks).

In the ssh.config file I have the following SSH configuration set:

Host bastion
    User                   ec2-user
    HostName               ###.###.###.###
    ProxyCommand           none
    IdentityFile           /path/to/ssh/key.pem
    BatchMode              yes
    PasswordAuthentication no

Host *
    ServerAliveInterval    60
    TCPKeepAlive           yes
    ProxyCommand           ssh -q -A ec2-user@###.###.###.### nc %h %p
    ControlMaster          auto
    ControlPath            ~/.ssh/mux-%r@%h:%p
    ControlPersist         8h
    User                   ansible
    IdentityFile           /path/to/ssh/key.pem

First the configuration for connecting to the bastion host is declared. Beneath that is a catch-all for all other hosts which in the ProxyCommand says to first connect to the bastion host then use netcat (nc) to pass the Ansible commands to the application server.

From the devops folder I can run ssh bastion -F ssh.config and I will be immediately connected to the bastion server.

Next Ansible needs to be told to use this custom SSH config when connecting to the application servers.

In the ansible.cfg file there is the following configuration:

ssh_args = -o ControlPersist=15m -F ssh.config -q
scp_if_ssh = True
control_path = ~/.ssh/mux-%%r@%%h:%%p

When Ansible runs from the devops folder it will automatically pick up this ansible.cfg file and use the defined config when playbooks are run.

One issue with this setup is that Ansible's output as it runs is very verbose as it includes the SSH debug connection information as it passes through the bastion host to connect to the application servers; I've not yet found a way to supress this.

Changing Times

For the past year I've been relatively quiet about my day to day work owing to the "stealth" mode we have been operating with the project at Videogamer. After many a rewrite the initial MVP of the project is complete and I'm very proud of what we have designed, coded and architected. There is still some work to be done on the business side before the project is revealed but I'm really excited for the future of the site and the team.

What I can say is though is that I've had the opportunity to play with some great technologies such as Elasticsearch, Neo4J and Ansible and I truly feel excited to be a developer at the moment. In my spare time I'm playing around with Docker, and I've been teaching myself Ruby and RubyMotion and I've written about 5000 words of my book.

At the end of May my contract was up at Videogamer and it was time to move on, I've thoroughly enjoyed my time with the team and I wish them all the best for the future; I really do believe the platform we've been building is the foundation of something great and I'm excited to see it in the wild.

Tomorrow I'm joining Yuza, a mobile business incubator in central London as their new software engineer. Yuza have recently launched Pollen Velocity Capital, a service which pays developers their app store revenues on a weekly basis (instead of the 30+ days that it can take for Apple and Google to pay); this money can then be one-click reinvested into social media campaigns or paid directly into the developer's bank account.

I've been relatively quiet on the blogging front this year but I intend to get back to regular blogging soon - I've already a post about OpenID 2.0 in my drafts folder, and I've been working on a big update to my OAuth 2.0 Server library.

Until then, adieu.

Notification overload


Someone sends me a tweet, email, iMessage, or I get a calendar alert


  • My Macbook alerts me
  • My iPad alerts me
  • My iPhone alerts me
  • And now my wrist vibrates as the alert is forwarded from my iPhone to my Pebble.


It would be great if Apple could solve the problem of sending notifications to whichever device is most relevant to me at the time - here are my suggestions:

  • If I'm sat casually browsing on my Macbook then alert me on there (but also send the notifications to my iPhone incase I suddenly get up and go).
  • If I'm working on my Macbook then send all notifications to my iPhone.
  • If I'm actively using my iPhone send all notifications there.
  • If I'm walking and my iPhone is sleeping in my pocket then buzz my Pebble.