composer – what you should know

Last year I wrote a piece called “a few thoughts about composer and how people use it“. In that post I had a list of things which are problematic about how composer is used.

That post got widely recognized, linked an visited, but in general those issues still exist.

However lately I’ve had even more people asking questions (either on related forums, irc or even irl) about problems that stem from issue number 2: people are using composer as an installer (and sometimes Number 3 because of Number 2).

In that Post I already gave a quick opinion on how workflows with composer should look like, In this post I’ll try to give a few more pointers on how to use composer without creating a mess.

starting a project vs installing

I’ve seen create-project abused as an installer quite a few times, and it seems convenient and simple to do that – as create-project + a hand full of scripts can easy do an initial install of a project. That is not what it is made for.

Now you might be wondering “why not? its only stupid if it doesn’t work!” – the thing is that this solution is one that usually leads to the question “is there a way to have composer update the create-project stuff??” very shortly after. No there is not and there shouldn’t be one.

So what is create-project for, if not to install your awesome web app on someones server? It is made so you can kickstart new projects by using skeletons that set up a basic project for you, usually adding all the things that might need updating at some point as dependencies, minus the code for your own project.

I wrote an a post on building project skeletons for binpress a few years ago, so if you are a framework builder, or if you have the need to build a custom project skeleton for you, your company, check it out.

You might find ways to make create-project work for you as an installer besides the no-real-update-path (I’ve seen people build quite elaborate scripts, and only have files that overwrite files from dependencies in their project-package), but the truth is, you will have a lot of effort, with not too much gain.

I mentioned it in the toughts-about post: composer is not a tool to run on live machines, (despite that environment checking thing that might make you think it is), and in professional environments, where servers are configured and or behind a firewall, so they are not allowed to pull data from everywhere your install simply won’t work. (The same goes for shared hosting where users might not even have access to a command line).

adding packages to a project

Now this is no starter-tutorial for composer, so I won’t go into how semver restrictions should look (rule of thumb: as tight as necessary, as lose as possible).

What I’d like to get of my chest here is: dear php devs, start thinking about what you add as composer dependencies. Your common day to day tools (like phpcs, phpunit and so on are no dependencies of your project. If you really really must (that is if you have a good reason)) add them as a dev-depepency (require-dev), but even that in a lot of cases doesn’t make sense (and this is a sin I’ve committed too). Believe me when I say there is no joy when you end up having version constraint conflicts because several of your dependencies insist on installing phpunit in different versions.

Another rule of thumb: the only thing that you should require is dependencies that are necessary to be there when you will run the app in the live environment – things that are only necessary on your dev environment (mockups, debugbars etc) belong in your require-dev, tools that you use over multiple projects (such as phpunit) should be installed on your dev (and ci) environment globally, not with the project.

globally installed composer packages

You might not know this, but composer allows you to install packages globally. STOP don’t get excited about this, whatever it is you are planing to use this for, if its not a composer plugin you are most likely wrong.

Take for example Laravel: Its tutorial tells you to do composer global require laravel/installer which will install 8(!) packages, including such commonly used ones as symfony/process or guzzlehttp/psr7. Why is this a bad thing? Well, in my stack I have quite a few tools, and many of them use quite common packages, which would lead to regular occurrences of having to use old tools because one tool is blocking another tools update. In laravels case is even more of a sin as that installer is only there to make a shortcut, so you can use laravel new blog`` instead ofcomposer create-project –prefer-dist laravel/laravel blog“`, the later being what you should use instead.

Now, as before i said “don’t put your tools into the projects”, and now I’m saying “don’t install them globally (btw, its only global for the user)”, how should you take care of your tools?

Tools, just like your webapp are applications by themselves, which means there should be builds of those tool-apps that you can install. A great way to do so is using phars. Some tools, like for example composer, come with ready to download an use phars, where the phar file contains everything needed to run the tool, and in composers example even better: with self-update functionality – for other tools its not too hard to build a phar, and for many you can use a tool I’ve recommended in another post before: phar-composer by clue.

• If you are looking for making your own tool self-update’able, have a look at phar-updater
• I mentioned that composer global is good for plugins, and i mentioned my thoughts-about post earlier. Number 1 in that post was pointing out how slow composer is and how many resources its using up – at least for the slow part there is a plugin which can help you a bit, someone came up with a cache-prewarmer, which will download packages in parallel before composer tries to install them, usually speeding up the process by a bit, check it out here: Prestissimo

install vs update

I’ve heard of a few people who always run composer update, and never do composer install. Worse, I’ve heard of people who run composer update in automated build processes: Don’t do that.

Running composer update should be a conscious decision made by a developer, who will also:

• run tests to ensure everything is still working after the update
• ensure that the packages installed do actually what they are supposed to do
• take responsibility for introducing this foreign code to the project (yes, that means review, especially of changes)

The composer.lock file written during this composer update run, is basically the recipe for composer install what dependencies to install during a build. Whenever no one is looking (or you don’t want to update), you should run composer install and be done with it.

tagging and building

Now lets talk about building for a moment. In many OSS projects I’ve seen the simple “I’ll tag it for github and be done with it”-approach (which then is often combined with the composer as installer mistake, or worse than that – with the dependencies being checked in to the git repository).

Now just tagging it and having github taking care of providing the build files (the automatically created .tar.gz) is a viable strategy for libraries which usually don’t require a build.

However for your application its not good enough: a build of an application should create an artifact (this might be a phar for a tool, a phar or a .tar.gz for an webapp) which on its own should be deployable on any system that meets the applications system requirements. Usually that is: the application itself, the dependencies of the application (according to the composer.lock file), and quite often the applications documentation. In corporate environments you often have templates for config files distributed as well, which contain variable that the install system used will replace, in products that are given to customers (or oss projects that are given to a non-dev community), its good practice to have config files with a -dist ending that can be used as templates to create the right config files, or an setup tool (thats included in the distribution), which will guide the user through the setup.

I feel this post has already grown very long, so i won’t put in a complete example on how to setup for builds, but let me mention a hand full of tools that with some research on the net allow you to get the setup right:

1. The first thing I do when I do a release is tag the release in my git repository – this allows me to checkout the exact same version when i need to work on it (for example for hotfixes). Usually before I do a tag i want to run the unittests a last time, check if there are any known security issues with dependencies, make sure everything is committed and so on. To do that I use a tool called RMT (Recommendation: install with phar-composer), which allows me to automate a few of the checks, which also creates a changelogs for me.

2. once I’ve the version tagged, i want to create the actual build – for that there is many different tools that you can use – if you are not familiar with any, Phing is one written in PHP, which is easy enough to learn – also there is a Phing Plugin for Jenkins, which is a contineous-integration server that allows you to automate builds.

3. Phing then can run composer, tell it to only install non-dev dependencies, and optimize the autoloader: composer install --prefer-dist --no-dev -o

4. Once i have the build artifact thats the deployment archive, i can then deploy and install this artifact. There are many solutions for that, if you use jenkins for the build automation you can have it run a script that copies the archive to the stage or live machines – or you could have it trigger ansible if you need a bigger solution. (This would really kill the scope of this post, and I’m not an admin).

5. If you don’t want to deploy directly, because you want to offer the build of your product (your oss software) to the world you can use githubs release stuff, if you use an automated system as mentioned before even through githubs release api..

6. when distributing software, consider signing it for example with gnupg, or provide it through keybase.io kbfs/public.

bottom line

as you can see there is a lot more things todo for a complete project to release, than what composer can (or should) handle by itself. So don’t try to do that.