Extension Example

In this chapter, we'll go over a simple real-world example of what a Phylum CLI extension might look like.

Our goal is writing an extension which can print out all dependencies with more than one version present in our dependency file.

The full example looks like this:

import { mapValues } from "https://deno.land/std@0.150.0/collections/map_values.ts";
import { distinct } from "https://deno.land/std@0.150.0/collections/distinct.ts";
import { groupBy } from "https://deno.land/std@0.150.0/collections/group_by.ts";

// Ensure dependency file argument is present.
if (Deno.args.length != 1) {
    console.error("Usage: phylum duplicates <DEPENDENCY_FILE>");
    Deno.exit(1);
}

// Parse dependency file using Phylum's API.
const depfile = await Phylum.parseDependencyFile(Deno.args[0]);

// Group all versions for the same dependency together.
const groupedDeps = groupBy(depfile.packages, dep => dep.name);

// Reduce each dependency to a list of its versions.
const reducedDeps = mapValues(groupedDeps, deps => deps.map(dep => dep.version));

for (const [dep, versions] of Object.entries(reducedDeps)) {
    // Deduplicate identical versions.
    const distinctVersions = distinct(versions);

    // Print all dependencies with more than one version.
    if (distinctVersions.length > 1) {
        console.log(`${dep}:`, distinctVersions);
    }
}

Now there's a lot to unpack here, so we'll go through things one by one:

Before we can start writing our extension code, we need to create our new extension:

phylum extension new duplicates

We can then start writing the extension by replacing ./duplicates/main.ts with our example code:

import { mapValues } from "https://deno.land/std@0.150.0/collections/map_values.ts";
import { distinct } from "https://deno.land/std@0.150.0/collections/distinct.ts";
import { groupBy } from "https://deno.land/std@0.150.0/collections/group_by.ts";

These are the Deno API imports. We use version 0.150.0 of Deno's STD here and import the required functions by loading them as remote ES modules. We'll go into more detail on what we need these for later.

// Ensure dependency file argument is present.
if (Deno.args.length != 1) {
    console.error("Usage: phylum duplicates <DEPENDENCY_FILE>");
    Deno.exit(1);
}

The Deno.args variable contains an array with all CLI arguments passed after our extension name, so for phylum my-extension one two that would be ["one", "two"].

Here we make sure that we get exactly one parameter and print a useful help message to the terminal if no parameter was provided.

The Deno.exit function will terminate the extension and return the provided error code.

// Parse dependency file using Phylum's API.
const depfile = await Phylum.parseDependencyFile(Deno.args[0]);

Phylum's API is exposed in the global Phylum object. Here we are using the parseDependencyFile method which reads the lockfile or manifest path passed as an argument and returns an object containing all dependencies and the package ecosystem. Since this function is asynchronous, we need to await it.

The returned object will look something like this:

{
  packages: [
    { type: "npm", name: "accepts", version: "1.3.8" },
    { type: "npm", name: "array-flatten", version: "1.1.1" },
    { type: "npm", name: "accepts", version: "1.0.0" }
  ],
  package_type: "npm",
  path: "package-lock.json"
}

// Group all versions for the same dependency together.
const groupedDeps = groupBy(depfile.packages, dep => dep.name);

Since our package list contains multiple instances of the same dependency, we need to group each instance together to find duplicate versions. Deno's convenient groupBy function does this for us automatically and we just need to tell it which field to group by using dep => dep.name.

This will transform our package list into the following:

{
  accepts: [
      { name: "accepts", version: "1.3.8" },
      { name: "accepts", version: "1.0.0" }
  ],
  "array-flatten": [ { name: "array-flatten", version: "1.1.1" } ]
}

// Reduce each dependency to a list of its versions.
const reducedDeps = mapValues(groupedDeps, deps => deps.map(dep => dep.version));

Since our dependency structure now contains useless information like name and type, we map each of these grouped values to contain only the version numbers for each dependency.

This results in a simple array with all dependencies and their versions:

{
  accepts: ["1.3.8", "1.0.0"],
  "array-flatten": ["1.1.1"]
}

for (const [dep, versions] of Object.entries(reducedDeps)) {

Since we now have an object containing all dependencies and the required versions, we can iterate over all fields in this object to check the number of versions it has.

    // Deduplicate identical versions.
    const distinctVersions = distinct(versions);

But before we can check the versions themselves, we need to make sure all the versions are actually unique. Some dependency files might specify the same version multiple times, so we need to ensure we filter duplicate versions.

    // Print all dependencies with more than one version.
    if (distinctVersions.length > 1) {
        console.log(`${dep}: `, distinctVersions);
    }
}

With all versions deduplicated, we can finally print out each dependency with more than one version in our dependency file.

For our example, the output looks like this:

accepts: [ "1.3.8", "1.0.0" ]

And that's all the code we need to check for duplicates. Now we can use the phylum extension run subcommand to test the extension without installing it:

phylum extension run ./duplicates ./package-lock.json

This should then print the following error:

Extension error: Uncaught (in promise) Error: Requires read access to "./package-lock.json"
    at async Function.parseDependencyFile (deno:phylum:201:16)
    at async file:///tmp/duplicates/main.ts:12:14

Phylum's extensions are executed in a sandbox with restricted access to operating system APIs. Since we want to read the lockfile from ./package-lock.json with the parseDependencyFile method, we need to request read access to this file ahead of time. All available permissions are documented in the extension manifest documentation.

While it would be possible to request read access to just ./package-lock.json, this would only work for package-lock.json files defeating the purpose of passing the dependency file as a parameter. Instead, we request read access to all files in the working directory:

[permissions]
read = ["./"]

Alternatively if you wanted to allow read access to any file, so dependency files outside of the working directory are supported, you could use read = true instead.

Now phylum extension run should prompt for these permissions and complete without any errors if they have been granted. Then we can install and run our extension:

phylum extension install ./duplicates
phylum duplicates ./package-lock.json