Why is PHP Composer so slow?

Question

Why is PHP Composer so slow when all I do is init a project with zero dependencies? Here are the commands I run:

composer init

Answer 1

Because Composer is implemented by file_get_contents(). That has no TCP optimizations, no Keep-Alive, no multiplexing, etc.

I created a Composer plugin to download packages in parallel: https://packagist.org/packages/hirak/prestissimo

$ composer global require hirak/prestissimo

Please try it. In my environment, composer install becomes 10 times faster.

Answer 2

step 1

$ composer self-update

step 2

$ composer clear-cache

step 3

$ composer update

Answer 3

I was running into this issue, and it was throwing me for a curve since I don't have Xdebug installed anywhere on my machine. It turns out that it was IPv6 addressing mode failures. So to test I ran

curl --ipv4 'https://packagist.org/packages.json'
curl --ipv6 'https://packagist.org/packages.json'

IPv4 went through, but IPv6 failed. In the end, you should look to find out why you're networking stack doesn't support it, but in my case, I decided to just give preference to IPv4 traffic until I can resolve that. On CentOS I created/modified the file /etc/gai.conf and put in the following:

label       ::1/128        0
label       ::/0           1
label       2002::/16      2
label       ::/96          3
label       ::ffff:0:0/96  4
precedence  ::1/128        50
precedence  ::/0           40
precedence  2002::/16      30
precedence  ::/96          20
precedence  ::ffff:0:0/96  100

On Ubuntu you can also edit that file and uncomment the line

precedence ::ffff:0:0/96  100

Source on Rackspace Community Hub

Answer 4

Edit: Conclusion: no solution listed here or generally seems to be bulletproof. What ultimately helps is usually using composer incrementally, requiring or removing only a single package at a time using composer require, composer remove command, also regards adding any "non-dependency specific" setting, any library can be added/removed for the sake of it of updating/refreshing the composer.json lock cache configuration.

Grouping my experience with answers from here and from the linked in question.

Prior I tried every single solution mention in this question to this time. However, it still did not help me, though I tried/noticed installing any other single library was considerably fast, so I went on investigating libraries one by one which could cause the bottle-neck.

1) YES And I located it to be codeception, "codeception/codeception": "2.4.0",

It took almost 200 seconds alone, though running it together with other libraries seemed at some times running almost for 12 minutes. Furthermore composer does not seem to reflect real-time when after initiating for 20 seconds it outputs it run 0.35s and so on later.

[374.9MiB/173.14s]     39/45:   https://codeload.github.com/sebastianbergmann/php-text-template/legacy.zip/31f8b717e51d9a2afca6c9f046f5d69fc27c8686
[374.9MiB/173.16s]     40/45:   https://codeload.github.com/sebastianbergmann/diff/legacy.zip/720fcc7e9b5cf384ea68d9d930d480907a0c1a29
[374.9MiB/173.17s]     41/45:   https://codeload.github.com/sebastianbergmann/exporter/legacy.zip/68609e1261d215ea5b21b7987539cbfbe156ec3e
[374.9MiB/173.19s]     42/45:   https://codeload.github.com/sebastianbergmann/phpunit-mock-objects/legacy.zip/f9756fd4f43f014cb2dca98deeaaa8ce5500a36e
[374.9MiB/173.20s]     43/45:   https://codeload.github.com/sebastianbergmann/recursion-context/legacy.zip/5b0cd723502bac3b006cbf3dbf7a1e3fcefe4fa8
[374.9MiB/173.23s]     44/45:   https://codeload.github.com/sebastianbergmann/php-code-coverage/legacy.zip/4cab20a326d14de7575a8e235c70d879b569a57a
[374.9MiB/173.24s]     45/45:   https://codeload.github.com/doctrine/instantiator/legacy.zip/ae466f726242e637cebdd526a7d991b9433bacf1
[374.9MiB/173.24s]     Finished: success: 45, skipped: 0, failure: 0, total: 45
[374.8MiB/173.25s] Analyzed 6622 packages to resolve dependencies
[374.8MiB/173.25s] Analyzed 435842 rules to resolve dependencies
[374.8MiB/173.25s] Package operations: 54 installs, 0 updates, 0 removals
[374.8MiB/173.26s] Installs: sebastian/recursion-context:3.0.0, sebastian/exporter:3.1.2, doctrine/instantiator:1.3.0, phpunit/php-text-template:1.2.1, phpunit/phpunit-mock-objects:6.1.2, codeception/stub:1.0.4, sebastian/diff:3.0.2, sebastian/comparator:3.0.2, theseer/tokenizer:1.1.3, sebastian/version:2.0.1, sebastian/environment:3.1.0, sebastian/code-unit-reverse-lookup:1.0.1, phpunit/php-token-stream:3.1.1, phpunit/php-file-iterator:1.4.5, phpunit/php-code-coverage:6.0.5, sebastian/object-reflector:1.1.1, sebastian/object-enumerator:3.0.3, sebastian/global-state:2.0.0, phpunit/php-timer:2.1.2, symfony/polyfill-ctype:v1.17.0, webmozart/assert:1.8.0, phpdocumentor/reflection-common:2.1.0, phpdocumentor/type-resolver:1.1.0, phpdocumentor/reflection-docblock:5.1.0, phpspec/prophecy:v1.10.3, myclabs/deep-copy:1.9.5, sebastian/resource-operations:1.0.0, phar-io/version:1.0.1, phar-io/manifest:1.0.1, phpunit/phpunit:7.1.5, codeception/phpunit-wrapper:7.6.1, symfony/yaml:v4.4.8, behat/gherkin:v4.6.2, symfony/polyfill-mbstring:v1.17.0, symfony/dom-crawler:v4.4.8, symfony/css-selector:v4.4.8, symfony/browser-kit:v4.4.8, symfony/event-dispatcher-contracts:v1.1.7, psr/container:1.0.0, symfony/event-dispatcher:v4.4.8, symfony/service-contracts:v2.0.1, symfony/polyfill-php73:v1.17.0, symfony/console:v4.4.8, symfony/finder:v4.4.8, ralouphie/getallheaders:3.0.3, psr/http-message:1.0.1, guzzlehttp/psr7:1.6.1, guzzlehttp/promises:v1.3.1, symfony/polyfill-php72:v1.17.0, symfony/polyfill-intl-idn:v1.17.0, guzzlehttp/guzzle:6.5.3, symfony/process:v4.4.8, facebook/webdriver:1.7.1, codeception/codeception:2.4.0
[374.8MiB/173.28s]   - Installing sebastian/recursion-context (3.0.0): [374.9MiB/173.28s] Loading from cache[374.9MiB/173.28s]
[374.9MiB/173.28s]  Extracting archive[374.9MiB/173.51s]   - Installing sebastian/exporter (3.1.2): [374.9MiB/173.52s] Loading from cache[374.9MiB/173.52s]
[374.9MiB/173.52s]  Extracting archive[374.9MiB/173.75s]   - Installing doctrine/instantiator (1.3.0): [374.9MiB/173.76s] Loading from cache[374.9MiB/173.76s]
[374.9MiB/173.76s]  Extracting archive[374.9MiB/174.00s]   - Installing phpunit/php-text-template (1.2.1): [374.9MiB/174.00s] Loading from cache[374.9MiB/174.00s]
[374.9MiB/174.00s]  Extracting archive[374.9MiB/174.23s]   - Installing phpunit/phpunit-mock-objects (6.1.2): [374.9MiB/174.23s] Loading from cache[374.9MiB/174.23s]
[374.9MiB/174.23s]  Extracting archive[374.9MiB/174.55s]   - Installing codeception/stub (1.0.4): [374.9MiB/174.56s] Loading from cache[374.9MiB/174.56s]
[374.9MiB/174.56s]  Extracting archive[374.9MiB/174.80s]   - Installing sebastian/diff (3.0.2): [374.9MiB/174.80s] Loading from cache[374.9MiB/174.80s]
[374.9MiB/174.80s]  Extracting archive[374.9MiB/175.07s]   - Installing sebastian/comparator (3.0.2): [374.9MiB/175.07s] Loading from cache[374.9MiB/175.07s]
[374.9MiB/175.07s]  Extracting archive[374.9MiB/175.33s]   - Installing theseer/tokenizer (1.1.3): [374.9MiB/175.33s] Loading from cache[374.9MiB/175.33s]

The run before

 Problem 1
    - Installation request for codeception/codeception 2.4.0 -> satisfiable by codeception/codeception[2.4.0].
    - Conclusion: remove symfony/finder v5.0.8
    - Conclusion: don't install symfony/finder v5.0.8
    - codeception/codeception 2.4.0 requires symfony/finder >=2.7 <5.0 -> satisfiable by symfony/finder[v2.7.0, v2.7.1, v2.7.10, v2.7.11, v2.7.12, v2.7.13, v2.7.14, v2.7.15, v2.7.16, v2.7.17, v2.7.18, v2.7.19, v2.7.2, v2.7.20, v2.7.21, v2.7.22, v2.7.23, v2.7.24, v2.7.25, v2.7.26, v2.7.27, v2.7.28, v2.7.29, v2.7.3, v2.7.30, v2.7.31, v2.7.32, v2.7.33, v2.7.34, v2.7.35, v2.7.36, v2.7.37, v2.7.38, v2.7.39, v2.7.4, v2.7.40, v2.7.41, v2.7.42, v2.7.43, v2.7.44, v2.7.45, v2.7.46, v2.7.47, v2.7.48, v2.7.49, v2.7.5, v2.7.50, v2.7.51, v2.7.6, v2.7.7, v2.7.8, v2.7.9, v2.8.0, v2.8.1, v2.8.10, v2.8.11, v2.8.12, v2.8.13, v2.8.14, v2.8.15, v2.8.16, v2.8.17, v2.8.18, v2.8.19, v2.8.2, v2.8.20, v2.8.21, v2.8.22, v2.8.23, v2.8.24, v2.8.25, v2.8.26, v2.8.27, v2.8.28, v2.8.29, v2.8.3, v2.8.30, v2.8.31, v2.8.32, v2.8.33, v2.8.34, v2.8.35, v2.8.36, v2.8.37, v2.8.38, v2.8.39, v2.8.4, v2.8.40, v2.8.41, v2.8.42, v2.8.43, v2.8.44, v2.8.45, v2.8.46, v2.8.47, v2.8.48, v2.8.49, v2.8.5, v2.8.50, v2.8.52, v2.8.6, v2.8.7, v2.8.8, v2.8.9, v3.0.0, v3.0.1, v3.0.2, v3.0.3, v3.0.4, v3.0.5, v3.0.6, v3.0.7, v3.0.8, v3.0.9, v3.1.0, v3.1.1, v3.1.10, v3.1.2, v3.1.3, v3.1.4, v3.1.5, v3.1.6, v3.1.7, v3.1.8, v3.1.9, v3.2.0, v3.2.1, v3.2.10, v3.2.11, v3.2.12, v3.2.13, v3.2.14, v3.2.2, v3.2.3, v3.2.4, v3.2.5, v3.2.6, v3.2.7, v3.2.8, v3.2.9, v3.3.0, v3.3.1, v3.3.10, v3.3.11, v3.3.12, v3.3.13, v3.3.14, v3.3.15, v3.3.16, v3.3.17, v3.3.18, v3.3.2, v3.3.3, v3.3.4, v3.3.5, v3.3.6, v3.3.7, v3.3.8, v3.3.9, v3.4.0, v3.4.1, v3.4.10, v3.4.11, v3.4.12, v3.4.13, v3.4.14, v3.4.15, v3.4.16, v3.4.17, v3.4.18, v3.4.19, v3.4.2, v3.4.20, v3.4.21, v3.4.22, v3.4.23, v3.4.24, v3.4.25, v3.4.26, v3.4.27, v3.4.28, v3.4.29, v3.4.3, v3.4.30, v3.4.31, v3.4.32, v3.4.33, v3.4.34, v3.4.35, v3.4.36, v3.4.37, v3.4.38, v3.4.39, v3.4.4, v3.4.40, v3.4.5, v3.4.6, v3.4.7, v3.4.8, v3.4.9, v4.0.0, v4.0.1, v4.0.10, v4.0.11, v4.0.12, v4.0.13, v4.0.14, v4.0.15, v4.0.2, v4.0.3, v4.0.4, v4.0.5, v4.0.6, v4.0.7, v4.0.8, v4.0.9, v4.1.0, v4.1.1, v4.1.10, v4.1.11, v4.1.12, v4.1.2, v4.1.3, v4.1.4, v4.1.5, v4.1.6, v4.1.7, v4.1.8, v4.1.9, v4.2.0, v4.2.1, v4.2.10, v4.2.11, v4.2.12, v4.2.2, v4.2.3, v4.2.4, v4.2.5, v4.2.6, v4.2.7, v4.2.8, v4.2.9, v4.3.0, v4.3.1, v4.3.10, v4.3.11, v4.3.2, v4.3.3, v4.3.4, v4.3.5, v4.3.6, v4.3.7, v4.3.8, v4.3.9, v4.4.0, v4.4.1, v4.4.2, v4.4.3, v4.4.4, v4.4.5, v4.4.6, v4.4.7, v4.4.8].
    - Can only install one of: symfony/finder[v2.7.0, v5.0.8].
    - Can only install one of: symfony/finder[v2.7.1, v5.0.8].
    - Can only install one of: symfony/finder[v2.7.10, v5.0.8].
    - Can only install one of: symfony/finder[v2.7.11, v5.0.8].

2) YES Btw, prestissimo as I mentioned was already applied and it seemed to help, as on another very fast computer which has much faster internet altogether without codeception, installation took 5 times more, 518 seconds, while perhaps due to prestissimo or other advices here, it took 110 seconds.

composer global require hirak/prestissimo

3) YES Windows is slow, and so are virtualized folders from Windows to Linux. Running outside the shared folder inside Linux seemed to speed it up about twice. Though it could be due cache.

4) NO

    "process-timeout": 1800,

is per single process like git checkout and did not seem to help at all even set to 10s, as it was already considerably huge. It perhaps helps only when there is a total outage of some service.

5) YES installing with a vendor folder removed or empty=freshly seems to be considerably faster (about 1 minute and more), than adding to the vendor files and other packages existing in it

6) YES seems to be slower on Ubuntu18 apache2 without xdebug own image, than on Debian10 php-fpm even with xdebug at Yii2-starter-kit , for some reason about twice slower , cat /etc/*-release to find out the Linux distribution

7) NO php -ddefault_socket_timeout=1 /usr/local/bin/composer --verbose --profile install mentioned in here did not seem to help Slow updating of composer dependencies, despite --prefer-dist flag nor there mentioned IP v6 v v4 protocol Though it could be due cache.

8) NO Installing zip doesn't seem to help, PHP installation had zip preinstalled, it still complained not being there until were installed as Linux packages but it made process in a single simulation slower by 100 seconds from 650 to 750 seconds. Same environmental conditions. As there is no 'unzip' command installed zip files are being unpacked using the PHP zip extension https://stackoverflow.com/a/51208804/3419535 testing on cacheless conditions --no-cache

9) YES remove "minimum-stability ": "dev",

10) YES use prefer-dist composer require robmorgan/phinx:@dev --prefer-dist -vvv --profile (using a require command as an example, as using partial require commands as mentioned very first above, instead of a full installation via an install command, seems to be one of the most significant speedups, plus requiring @dev- as along the previous points when minimum-stability: dev not applied generically to all dependencies seems to be harmless using on per case(dependency) basis)

Answer 5

In case, If you have deleted your composer.lock file, and then gone for composer install it gets too slow to download dependencies. If this is the case, use composer update command and then it may help you or others.

Answer 6

The same here. Get more details with "composer install --profile -vvv". In my case it takes a long time to download a few JSON files. They get cached on my server, but they are still downloaded with every Composer update/install call.

... 30 minutes later ...

It looks like some performance problem @packagist.org. Now the Composer install runs within 2 seconds! And downloaded JSON files are properly cached.