ied

Travis npm js-standard-style Join the chat at https://gitter.im/migme/beachball

An alternative package manager for Node.

Internals

Under the hood, ied maintains an "object database", similar to git. Instead of storing packages by some arbitrary name, a SHA1-checksum is being generated to approximate their contents. The checksums can not only be used for guaranteeing a certain level of trust and consistency, but they also simplify the algorithm through which dependencies are being managed.

The algorithm through which packages are being installed guarantees consistency through atomic installs. The installation of a package either fails or succeeds, but at no point in time can a dependency itself be required without having its own sub-dependencies installed (with the exception of shared circular dependencies).

The checksum of a package is based on the contents of the package itself, not of its sub-dependencies. Therefore the validity of a package can be verified by hashing the package itself. Subsequent dependency updates have no effect of the generated checksum.

Since node_modules is essentially a file-system based content addressable storage, multiple versions of the same package can co-exist in the same project. In order to expose dependencies via CommonJS, symbolic links are being created that reference a specific version of the package. This has multiple advantages:

  1. Undeclared dependencies that have been installed as sub-dependencies of "direct" dependencies are unlikely to be required "accidentally".

  2. There is no need to "manually" (as in additionally to the installation procedure itself) de-duplicate the dependency graph. As long as the uniqueness of filenames itself can be guaranteed on an OS-level, it is impossible to install the same package twice. This does not prevent users from installing different versions of the same dependency as long as the content is different (whereas a different version declared in the package.json counts as different contents).

  3. Shorter pathnames and less problems due to OS-level limitations (as in Windows where the maximum path length is limited).

  4. Additional application-level startup performance improvements. require needs to traverse less directories. A limited number of symbolic links need to be followed. This performance improvement is primarily useful for continuously running tests, where startup time is actually noticeable for larger test suits.

Directory Structure

The used directory structure is primarily optimized for reducing the amount of IO interaction with the file system during subsequent installations and guaranteeing the consistency of installed packages.

A consequence of the require.resolve algorithm used by Node, all packages need to be stored in a project-level node_modules directory. This directory is completely flat on a package-level, meaning that there are no nested packages inside it.

Instead each package is being stored in its content-addressed directory. Such a directory has two sub-directories:

On a project level, the node_modules directory contains the fetched packages, installed dependencies and links that expose the packages to user-land via require.

A comparison of sample directory structures produced by ied, npm 2 and npm 3 is available as a GitHub Gist.

Why?

The original idea was to implement npm's pre-v3 install algorithm in as few lines as possible. This goal was achieved in c4ba56f.

Currently the main goal of this project is to provide a more performant alternative to npm.

Installation

The easiest way to install ied is using npm:

  npm i -g ied

Alternatively you can also "bootstrap" ied. After an initial installation via npm, ied will install its own dependencies:

  git clone https://github.com/alexanderGugel/ied ied && cd $_ && make install

Usage

The goal of ied is to support ~ 80 per cent of the npm commands that one uses on a daily basis. Feature parity with npm other than with its installation process itself is not an immediate goal. Raw performance is the primary concern during the development process.

A global configuration can be supplied via environment variables. NODE_DEBUG can be used in order to debug specific sub-systems. The progress bar will be disabled in that case.

Although run-script is supported, lifecycle scripts are not.

At this point in time, the majority of the command API is self-documenting. More extensive documentation will be available once the API is stabilized.

A high-level USAGE help is also supplied. The main goal is to keep the API predictable for regular npm-users. This means certain flags, such as for example --save, --save-dev, --only, are supported.

  ied is a package manager for Node.

  Usage:

    ied command [arguments]

  The commands are:

    install     fetch packages and dependencies
    run         run a package.json script
    test        run the test-suite of the current package
    shell       enter a sub-shell with augmented PATH
    ping        check if the registry is up
    ls          print the dependency graph
    expose      make a sub-dependency `require`able
    config      print the used config
    init        initialize a new package
    link        link the current package or into it
    unlink      unlink the current package or from it

  Flags:
    -h, --help      show usage information
    -v, --version   print the current version
    -S, --save      update package.json dependencies
    -D, --save-dev  update package.json devDependencies
    -o, --only      install a subset of the dependencies
    -r, --registry  use a custom registry (default: http://registry.npmjs.org/)

  Example:
    ied install
    ied install <pkg>
    ied install <pkg>@<version>
    ied install <pkg>@<version range>

    Can specify one or more: ied install semver@^5.0.1 tape
    If no argument is supplied, installs dependencies from package.json.
    Sub-commands can also be called via their shorthand aliases.

  README:  https://github.com/alexanderGugel/ied
  ISSUES:  https://github.com/alexanderGugel/ied/issues

Credits

Some ideas and (upcoming) features of ied are heavily inspired by Nix, a purely functional package manager.

FAQ

License

Licensed under the MIT license. See LICENSE.