Processor API

Processor is a standalone node module that will be required/loaded by a Drovp thread.

It runs in its own raw Node.js process, and it's responsibility is to accept an operation payload, and return a promise that resolves when the operation has been processed.

The function has to be exported as a default export using CommonJS. ESModules are not supported yet (#2).

Example module
module.exports = async (payload, utils) => {
    const {input, options} = payload;

    // ... do what needs to be done to the input

    // Optionally emit outputs
    utils.output.file('/new/file/path.ext');

    // Use node's built-in `console.*` methods for debugging/logging
    console.time('log time');
    console.log('hey');
    console.timeEnd('log time');
};

Debugging

If you want to log some stuff for debugging, use node's built-in console.* methods, but maybe read up on the non-standard logging environment documented in the log utility below. If you want to prevent the issues described there, you can just use the utils.log() instead, but this is not necessary in most cases.

These logs can then be seen in the operations Logs section.

Processor

Type: (operation: Operation, utils: ProcessorUtils) => Promise<void>

Processor is a simple async function that accepts operation payload, some utilities, and resolves when everything is done.


operation

Type: Operation

The operation payload. See Operation below for more details.


utils

Type: ProcessorUtils

The operation utilities. You can use these to report progress, access Drovp dependencies, emit outputs, ...

See ProcessorUtils API below for more details.


Return

Promise that resolves when operation is done. Any resolved value is ignored.

If you want to emit an output item, use utils.output API.

If the promise throws, it's the same as calling utils.output.error(message) and resolving right afterwards.

Payload

interface Payload {
    id: string;
    options?: object;
    inputs: Item[];
    input: Item; // convenience getter for the 1st item in the `items` array
}

Operation payload with these properties:


id

Type: string

Operation ID.


options

Type: object

Processor's options schema generated and user configured options of the profile into which the item(s) were dropped into.

If processor doesn't have an options schema, this is undefined.

Example

For options schema that looks like this:

[
    {name: 'foo', type: 'boolean', default: false},
    {name: 'bar', type: 'number', default: 0},
],

And assuming the user has kept all of the profile options on their default values, you'll get this options object:

{foo: false, bar: 0}

input/inputs

Type: Item | Item[]

If processor config doesn't allow bulking (default behavior), operation will always have a single Item in it's inputs array. If bulking is enabled, there will be as many items in this array as user has dropped into the profile.

input is just a convenience getter for the 1st item in the inputs array. Useful if you only expect one input per operation.


{other}

Type: any

Processor can also configure an operation preparator, which can modify the operation payload however it sees fit, in which case the payload will also have other properties you decided to add to it.

Item

Type: ItemFile | ItemDirectory | ItemBlob | ItemString | ItemUrl

Depending on processor's accepts.* configuration, Item can be one of:

ItemFile

interface ItemFile {
    kind: 'file';
    type: string; // Lowercase extension without the dot
    path: string; // Path to the file
    size: number; // File size
}

ItemDirectory

interface ItemDirectory {
    kind: 'directory';
    path: string; // Path to the directory
}

ItemBlob

interface ItemBlob {
    kind: 'blob';
    mime: string; // Blob mime type
    contents: Buffer; // BLob contents as a Buffer
}

ItemString

interface ItemString {
    kind: 'string';
    type: string; // String type, e.g. 'text/plain'
    contents: string; // String contents
}

ItemURL

interface ItemUrl {
    kind: 'url';
    url: string; // URL path
}

ProcessorUtils

interface ProcessorUtils {
    dependencies: {[key: string]: any};
    output: OutputEmitters;
    progress: ProgressUtil;
    title: (value: string | undefined | null) => void;
    stage: (name: string) => void;
    log: (...args: any[]) => void;
    appVersion: string;
}

dependencies

Type: {[key: string]: any}

A map of dependency names and their exports. Usually the exports are paths to binaries, but it can be any serializable value. See each dependency's documentation for what is being exported.

Each dependency export can be accessed with its short, and full no-collision name.

Example

If you depend on @drovp/ffmpeg:ffprobe, your dependencies map will look like this:

{
    ffprobe: '/path/to/ffprobe/binary',
    '@drovp/ffmpeg:ffprobe': '/path/to/ffprobe/binary',
}

output

Type: OutputEmitters

Utilities for emitting output items. See OutputEmitters for details.

Quick example
output.file('/path/to/file');
output.directory('/path/to/directory');
output.url('https://example.com');
output.string('token_nd8S7f7aG87b');
output.warning('Something of interest.');
output.error(new Error('Something went wrong.'));
output.error('Something went wrong.');
// Attaching flair and/or badge
output.file('/path/to/file', {
    flair: {variant: 'warning', title: 'foo', description: 'This item food the bad'},
    badge: {variant: 'warning', icon: 'warning', title: 'This item food the bad'},
});

progress

Type: Progress

A progress reporting utility. See Progress.

Quick example
// using `progress()` method always sets all 3 values
// `total` and `indeterminate` parameters are always optional,
// and set to undefined when omitted
progress(completed, total, indeterminate);
progress(null); // reset all
progress({completed: 1, total: 10, indeterminate: true}); // object format

// using `progress.X` only updates concerned property
progress.completed = 1;
progress.total = 10;
progress.indeterminate = true;

title

Type: (value: string | undefined | null) => void

Sets or deletes operation title. By default, operation titles are constructed from its payload (file paths, domains, etc), but if you have something more descriptive or relevant, you can use this to overwrite it.


stage

Type: (name: string) => void

Used to name the stage the operation is currently in. This is just to provide an extra piece of information to the user.

The name should be 20 characters and less.

For example, if the operation downloads something and then extracts it, you'd set it to downloading, than extracting.

Setting a stage logs STAGE: ${name} to the operation log.


log

Type: (...args: any[]) => void

Logs one or multiple values into the current operations Logs section.

This is the actual utility responsible for sending operation logs. The global console.log is replaced with it for the duration of the operation. This is necessary because all messaging (utils.output.*, utils.stage, ...) is handled through IPC channel, while console.log outputs to stdout, is batched, and therefore not reliably in sync with other stuff. This would often cause operation logs to be out of sync/order with progress/stage/etc reporting, or even cause debugging issues where stuff that is logged just before the operation resolves is nowhere to be seen.

You can continue using console.log, as it's definitely more comfortable (you don't have to be passing the log utility to all dependencies), but just be aware that even thought it's replaced with the immediate IPC version, there is still an issue where if something in your operation reports AFTER the operation resolved (some hanging unresolved async calls you forgot to await for), it'll either be logged in the logs of the operation that came after it, or if the process is idle, logged in the Drovp's developer console as "Thread stdout with no job in progress" error. If preventing this is a high priority, you can of course just use this log() instead of console.log() directly, as it's scoped to the current operation.

Note: console.log is the only replaced console.* method. Everything else still reports through stdout/err, and has the aforementioned caveats.


appVersion

Type: string

Current Drovp app version.