A few thoughts about composer and how people use it

composer has changed the PHP ecosystem like no other tool introduced – almost everyone is using it today. Now, I have written about composer before, and have always been a big proponent of using it. However, as I have spend some time with looking more closely on a few things, there is a few problems (some with composer, some with how people (ab)use composer) that I would like to write about.

I’m pretty certain that my point of view is not the only valid one, and that some of you will disagree with what I say – use the comments if you want to add something or have questions, and if you blog a repsonse, please use a pingback so I’ll notice your post.

Please keep in mind: this is not a rant against composer. You should be using it.

Number 1 – composer gets slow and resource hungry

composer builds up quite huge dependency graphs to find out which packages to install, which means projects that depend on packages that have a lot of dependencies with history, and lax to no restraints on those 2nd level dependencies, will cause composer update to eat tons of ram, and take quite a while.

One option to “fix” that is to add your 2nd, 3rd etc. dependencies to your root composer.json, with tighter version contraints. While this actually limits the problem to some extend, its not a real fix though, as it means sooner or later you end up doing composers job manually yourself.

Another option is to turn of your memory limit, and keep supplying more memory to your machines, however I’ve been told that this breaks for windows users. Also it gets quite cumbersome when a composer update starts taking longer than a cigarette/coffee break.

Thats not the only thing that keeps it slow – composer is retrieving meta-data (and the contents) in a blocking manner, which means if one request is slow the whole process will be held up. Using something like reactphp could actually decrease the time composer needs for those tasks.

Number 2 – people are using composer as an installer

Something I did wrong myself when I started working with composer. Not only does composer make it easy to be abused in that manner, having the option to have composer check your php and your php extensions makes it seem like a logical step.

However, that is problematic for various reasons:

• composer is a developer tool, as such it shouldn’t run on live machines
• composer will fetch dependencies from a lot of sources, which means that your live systems would need to be able to fetch from there (ask your admin what he thinks about that), and to not get rate limited on github for example, you would need to put credentials on your live machine…
• if you don’t backup your dependencies (and when using composer as an installer you don’t have an explicit step for that) you might get a bad surprise when someone decides to remove his project. Packagist only stores meta-data, if someone removes the git or the dist zip from github (or any other repository) you can only pray that you have cached it somewhere. I’ve seen a hand full of packages on packagist that are not available on github anymore.
• your deployment becomes dependend on systems that are not under your control – if github gets ddosed again, or packagists provider fucks up routing again during deployment, you might end up with a much longer maintenance period that anticipated.

Let me from my perspective mention how I think composer should be used:

1. developer initializes a new projekt using composer create-project
2. developer adds requires as needed, he runs composer update to create the composer.lock file, which is added to the scm. this developer at this time is responsible to verify that the version(s) he updated to are working as required.
3. when pulling a changed composer.lock from scm, each developer working on it will run composer install to have the correct setup.
4. once a release is necessary the person who holds the release manager role will create a build of the project (it does not matter if he runs a build script or elevates a build in your build system, in the end someone is responsible to decide when something should be released). Most likely within this build there will be done something like composer install --prefer-dist --no-dev -o the result of this step should be a package (directory, zip, tarball, .deb, or whatever else your deployment system needs).
5. the admin uses the result of 4. to install the release on a live machine

now obviously this does not cover all variants, for example if your release is a library that you want to publish on github, then step 4 is the process where you tag your version number and step 5 is not relevant. Or you have more processes inbetween (staging/QA etc) but this should illustrate how responsibilites should be separated, and it should prevent you from getting in a half-installed-live-system-situation.

Number 3 – people use their own paths

Now, I’m pretty sure this is one that quite a few people will disagree with me.

Composer offers the ability to use custom installers, and even provies the composer/installers package, which provides an easy way to install packages into other folders.

Quite a few Frameworks and CMS make use of this, usually for one or two of three reasons

1. to install assets somewhere
2. to keep an own directory structure (for legacy reasons)
3. to distinguish various types of packages (regular depedencies, modules, themes, plugins…you name it)

where 3. seems to be the most common usecase.
Obviously its conveniant, by slapping those dependencies into different directories, you can have your module loaders etc, very easy detect a new module, or a new plugin.

So why am i listing this as a problem?
because:

• own folders lower the interoperatbility of packages (ok, granted, this is bean counting, your blogs theme wouldn’t work with my forum anyways)
• own folders lower the transparency (vendor code is in vendors, own code in the project),
• and raise the learning curve (ok, if you only use one framework constantly, you probably can remember 5 paths)

as you see, the argument against it is actually a bit weak, however to me those are still valid arguments, and there are better solutions (puli, assetic (for assets), your installer managing a configuration file vs. using a dir to find all modules) available.

Number 4 – people don’t adhere semver

Semantic Versioning actually makes a lot of sense – the version contraints in composer follow the assumption that semantic versioning is used, still people choose not to. Why? i don’t know. But if you release bc breaking changes in a PATCH release, then there is a special place in hell for you.

Number 5 – people don’t tag their releases / don’t release

This goes hand in hand with Number 6 – I’ve seen packages that are on packagist, without ever having a version tagged. Thing is, the moment you put it on packagist, in theory, you should have a first version. If you don’t tag it, people can’t properly set their dependencies, meaning they shouldn’t be using it.

Don’t be scared of releasing, release often, and Majors if needed.

Also, as i’ve seen this lately, there are people who use branches instead of tags – well at least thats how it looks to me (maybe someone wants to enlighten me for the reason), which means you end up with versions like “2.3.x-dev, 2.2.x-dev, 2.1.x-dev”, which means you will always be on unstable versions.

Number 6 – people release packages with dependencies to unstable versions

Do I even need to say something about this? To my surprise it is not exactly uncommon, and quite often a result of number 5. Oh the joys on you requiring a package that has a stable version, but relies on a dev-package, that itself relies on a dev-package from a -develop branch.

Those who know composer a bit better might already have spotted the hidden evil in this combination. Your root packages composer.lock is the only composer.lock that is relevant. When you run composer update on your root package you might end up with a completely different version of dev-randompackage-develop than the developer who build the dev-otherrandompackage, and yet another one than the one who build stablepackage – which makes for nasty heisenbugs.

So, what now? / TL;DR

As you can see the things i listed are quite different aspects.

Number 1: to tackle this problem it probably requires a major rewrite of composer, and maybe even rethinking of how to use it

Number 2: one can get around a part of this problem using toran proxy or a local satis install but in the end the real solution might be if there was more tutorials on how to get the installing part right.

Number 3: i named some better options already

Number 4,5,6: Thats us, all of us who publish packages, work on opensource projects etc. We have to show more responsibility and discipline here – and clean up the chaos we are having right now.