Composer & virtual packages

Composer & virtual packages

Composer has been a blessing for the PHP community, and many many people use it today. However most people don’t know all it can do – i for certain every now and then learn something new. A few days ago i stumbled over a “virtual package” on packagist – and found it to be a feature that i was actually missing in composer. Turns out, composer can do it, its just not so well documented.

So what is this about? Virtual packages allow you to have a more loose dependency. Rather than depending on a specific package, you depend on a virtual one, which can be fulfilled by all packages that provide the virtual one. Sounds confusing? Lets have a look at an example.

Lets assume you are building a library, lets call it example/mylib, and this library makes use of a PSR-3 compatible logger. Now obviously your package will depend on “psr/log” in version 1.0.0, so you have the psr/log interfaces available. Now you don’t want to depend on a specific implementation, because you don’t want a user of your lib to be forced to use that one. But since you are building your lib so it needs a log provider injected (and if it is a NullLog), you want your dependencies to reflect that. So, what you do is: you require “psr/log-implementation” in version 1.0.0, just the same way you would require a regular package. “psr/log-implementation” doesn’t exist as a package though, but there are several packages which provide this virtual package so if someone who depends on your library depends on one of those as well, all of his depedencies will be met.

How does a library provide a virtual package? Simple, by using the provide keyword in its composer.json.

Lets look at a hand full of composer.json examples:

Example 1: example/mylib composer.json – our own project that requires a logger

{
  "name": "example/mylib",
  "description": "...",
  ...
  "require": {
    "psr/log": "1.0.0",
    "psr/log-implementation": "1.0.0"
  }
  ...
}

In Example 1 we define our example/mylib to require the (existing) package “psr/log”, and a virtual package “psr/log-implementation”, where the way how require those is exactly the same. To be used our lib now requires that a “psr/log” 1.0.0 and a “psr/log-implementation” 1.0.0 package are available when we install it in any application.

Example 2: somelog/logger composer.json – a random psr/log implementation

{
  "name": "somelog/logger",
  "description": "...",
  ...
  "require": {
    "psr/log": "1.0.0"
  }
  "provide": {
    "psr/log-implementation": "1.0.0"
  }
  ...
}

In Example 2 we have a random psr-compliant logger, and it defines that it provides a “psr/log-implementation” 1.0.0 package (besides providing “somelog/logger”), so if we require this package anywhere, we automatically get an “psr/log-implementation” on 1.0.0 requirement fulfilled. Packages can also have more than one virtual package that they fulfill in their “provide” section, allowing one package to fulfill more than one requirement.

Example 3: myapp/myapp composer.json – an application using our library

{
  "name": "myapp/myapp",
  "description": "...",
  ...
  "require": {
    "example/mylib": "1.0.0",
    "somelog/logger": "1.0.0"
  }
  ...
}

Example 3 shows an application requiring the library and fulfilling the libraries derived requirement for a psr/log-implementation by requiring “somelog/logger”.

This whole thing can do a few very nifty tricks, where you can have the users of your lib customize their stack and still work with your library – basically you raise the interoperability while still hinting on what you need. I wish more projects would make use of this.

EDIT: Matthias Noback wrote a post as a reply to this one which you can read here. Please read it, as he has some valid points.

As disqus swallowed my comment on his blog directly (not sure if it is pending approval or if loging in just made it forget), I’ve decided to address Matthias concerns here.

4. psr/log-implementation, or any virtual package for that matter, is very problematic as a package. There is no definition of this virtual package. It is merely a phenomenon arising from the fact that some package has the name of the “virtual package” in its provide section. In the case of the psr/log-implementation package, this lack of a proper definition or rules for virtual packages means that there can a) be packages that contain a class that implements LoggerInterface (from psr/log), but don’t have "provide": { "psr/log-implementation": ..." } in their composer.json and b) that packages might say they provide, while they don’t. Which makes the concept unreliable.
5. Some day, someone may decide to introduce another virtual package, called the-real-psr/log-implementation (they can easily do that, right?). Such packages may be completely exchangeable with existing psr/log-implementation packages, but in order to be useful, every existing PSR-3 compliant logger package needs to mention that virtual package too in their provide section. And so on, and so on. The underlying conceptual problem is: there is no such thing as a canonical virtual package.

Basically I agree with points 4 & 5 of what he wrote. However I’d like to point out that with composer and package naming there are a few issues anyways, and one option to deal with it would be a PSR on how such packages should look – or any other form of convention.

Besides that as a developer you are always responsible to check what packages you include in your application. Just because composer tells you “you need an implementation of package a” you can’t pick a random implementation and hope it will be doing everything correctly – however it is still nice if your dependency management tells you that you need an implementation

1. Strictly speaking (as in, would the code compile), the code from the library itself doesn’t need a package that provides psr/log-implementation. It just needs the LoggerInterface (which happens to be in the psr/log package).
2. Of course, in order to actually run the code from the library you will need an instance of LoggerInterface, which means you need a class that implements said interface. But that doesn’t mean you actually need a package that contains such a class. That class can be located anywhere, in the current project, in a globally installed PEAR package, in a PHP extension, it may even be shipped with PHP. If you want to communicate that your library needs a working logger implementation, just using the LoggerInterface – and thus requiring just psr/log – is quite enough.
3. By depending on an implementation package, you basically undo any effort you made to depend on abstractions and not on concretions. Since a “PSR logger” implementation is by definition a concrete implementation of the LoggerInterface from psr/log. In other words, you have pointed your previously inverted dependency arrow back to concrete packages (although you leave it undecided which concrete package that will be).

1. is correct, the code only needs the interface, but as a package is a bit more than just the code, 2. comes into play, and thats where I disagree, if you use a dependency management system, then you shouldn’t try to fullfill parts with stuff outside of that dependency management – meaning, it shouldn’t be a globally installed PEAR package, or if you can’t do without should wrap it.
Also, just using the interface does not “communicate”, infact just using the interface the dependency management will be fine even if no implementation (thats still required from the application) is included.
I make that distinction between packages and code, so point 3 is not really something that bothers me in this case, infact i’d say a virtual package is the most abstract package you can require, where as not requiring one at all is just missing something. It really is just giving an application that uses (well the dev of it) a hint about what he needs to take care of, while leaving him the choice how to do it.

Bonus: if I’m not completely wrong the App dev can even decide to add a provide to his own app, and include the implementation there.. comment if you tried it.

6. The notion of an “implementation package” is really vague. What does it mean for a package to be an implementation package. Is it sufficient for it to implement just one interface? What if the “interface package” contains multiple interfaces, which one should the “implementation package” implement? All, one?
7. The final argument against psr/log-implementation packages is that psr/log (the interface package) itself contains a NullLogger class, which is an implementation of its own LoggerInterface, and therefore this package itself also qualifies as a psr/log-implementation package!

6. is really a no-issue for me, as if I provide a package(s implemenation) I have to provide the complete package. Which brings me to 7, and thats an issue I’ve already thought about posting a rant over. Interface packages such as psr/log should not contain logic. That nulllogger should be in an own package. But that issue in that case is small as psr/log does not “provide”. I’ll write more about that when I find the time to write up on why I don’t like stackphp.

The DoctrinePHPCRBundle Matthias picked is a good example for what I suggested in this post, it requires the virtual phpcr/phpcr-implementation, and both implemenentations of that require phpcr/phpcr which contains the shared interface. Maybe that is a bit of a better example than psr/log, as it is less abstract (example wise) – I did pick psr/log, as it is a very commonly used package, and as I’m a big proponent of the work that the PHP-FIG guys are doing by standardizing interfaces.

Advertisement