Why does Nexus download npm packages from sites other than the configured remote URL?

Problem

You have configured a Nexus npm proxy repository with remote URL of the official npm registry https://registry.npmjs.org. You notice that some npm tarballs downloaded outbound from Nexus contact other URLs for package tarballs, such as:

  • http://registry.npmjs.org
  • http://registry.npmjs.com
  • https://registry.npmjs.com

Cause

Originally the npm registry URL was advertised as http://registry.npmjs.org . In a blog post on April 1, 2016, it was announced the official registry URL changed to https://registry.npmjs.org.

Current to July 19, 2016, the official registry is still documented as https://registry.npmjs.org and this remains the default publish registry value the npm client uses.

Unfortunately using a different root tarball URL in the official registry metadata was allowed briefly, so some package deployers will have deployed metadata references to the previously mentioned URL variations.

When Nexus is asked to download an npm package from a remote, it first requests that package's metadata from the remote URL configured in your proxy repository configuration. The metadata contains specific tar ball URLs - these are the locations where Nexus or an npm client should download the specific versioned tarballs your build is requesting. The URLs are not enforced to be matching the official registry URL, so Nexus must defer to these locations and contact whatever URL is specified.

Example

Let's see an example of this in action.

Examine the metadata for a specific version of event-stream package.

https://registry.npmjs.org/event-stream/3.3.3

Notice the tarball URL points at http://registry.npmjs.org:

"dist": {
    "shasum": "f7d437a98f9c0715f4b643a97b66425549479b9b",
    "tarball": "http://registry.npmjs.org/event-stream/-/event-stream-3.3.3.tgz"
  },

For the next version from https://registry.npmjs.org/event-stream/3.3.4, the tarball URL points at a different host http://registry.npmjs.com:

"dist": {
    "shasum": "4ab4c9a0f5a54db9338b4c34d86bfce8f4b35571",
    "tarball": "http://registry.npmjs.com/event-stream/-/event-stream-3.3.4.tgz"
  },

Finally, request the main event-stream metadata from https://registry.npmjs.org/event-stream - all the tarball URLs there refer to https://registry.npmjs.org/ except the one for version 3.3.4 which refers to https://registry.npmjs.com/.

In summary, the request for metadata was over https and the tarball URLs vary between protocols and host names.

Now suppose the firewall Nexus goes through for outbound requests is allowed to access https://registry.npmjs.org but not registry.npmjs.com, and 3.3.4 is now the latest version of this package, and your build has a dependency version on the latest version, Nexus could return 404 Not Found for 3.3.4 ( it was blocked by your firewall) and mysteriously the previous version 3.3.3 was found and worked fine.

You might be left wondering what happened?

Work Around

This is a good example of how strict organizational firewalls and repository managers proxying a variety of repo formats have conflicts.

The only solution at this time is to verify by checking the metadata as above the exact cause of the problem, and have your network team open access to the required domains needed by Nexus to download packages.

Also make sure that both secure URLs https://registry.npmjs.org and https://registry.npmjs.com are accessible through the firewall Nexus uses and the certificates are trusted by Nexus if they are rewritten by a HTTP proxy server.

References

[NEXUS-10493] NPM tarballs proxied from registry.npmjs.org may be fetched by Nexus at registry.npmjs.com

[NEXUS-10037] NPM tarballs proxied over http may be fetched remotely over https

Have more questions? Submit a request

0 Comments

Article is closed for comments.
Powered by Zendesk