Subscribe
Building_an_engine_with_the_veritone_engine_toolkit
04.8.19 —

Building an engine with the Veritone Engine Toolkit

Veritone’s aiWARE platform lets customers run AI solutions at a very large scale, and it does it by spinning up as many instances of the cognitive engines required for a given use case to meet the demand and handle the load. When they’re done, they shut down to free resources.

Engine developers don’t need to worry about this process (the platform handles all that for us), they just need to make sure the engine can receive work, process the files, and send the output.

In order to put an engine onto the platform, you must assemble a Docker container and build an integration with the Veritone aiWARE platform and we recommend using the Veritone Engine Toolkit to do so.

Project homepage: https://machinebox.io/veritone/engine-toolkit

The Veritone Engine Toolkit

The Veritone Engine Toolkit does the heavy lifting of integrating with the Veritone aiWARE platform, allowing you to focus on what makes your engine valuable.

The Engine Toolkit works by interfacing on your behalf with the aiWARE platform (connecting to the messaging queue to receive work and send results, pulling data via GraphQL, and more) and makes simple HTTP requests to your engine to process the files.

You just need to listen on an HTTP endpoint (by writing some kind of simple web server), and provide a webhook which the Engine Toolkit will call when the time is right.

The Process webhook is where your engine does its work

The Process webhook is called for each chunk (a chunk might be an image from a frame, an audio clip, or some other kind of file — depending on what your engine can do), and the response your webhook returns will be delivered by the toolkit to the platform.

Using whichever framework or mechanism makes sense for your language of choice, you can extract the chunk file from the HTTP request, along with a series of other data which is outlined in the documentation and previewed on the left.

The webhook will return a blob of data that describes the output for the work that was done during the processing.

For example, if our engine is finding faces, we might output some JSON that looks like this:

{
	"series": [{
		"startTimeMs": 1000,
		"stopTimeMs": 2000,
		"object": {
			"type": "face",
			"confidence": 0.95,
			"boundingPoly": [
				{"x":0.3,"y":0.1}, 
                                {"x":0.5,"y":0.1},
				{"x":0.5,"y":0.9},
                                {"x":0.3,"y":0.9}
			]
		}
	}, {
		"startTimeMs": 5000,
		"stopTimeMs": 6000,
		"object": {
			"type": "face",
			"confidence": 0.95,
			"boundingPoly": [
				{"x":0,"y":0}, {"x":1,"y":0},
				{"x":1,"y":1}, {"x":0,"y":1}
			]
		}
	}]
}

The Veritone Engine Toolkit will post this data via the message queue to the platform on your behalf, and moments later it will be available via the APIs, as well as the applications that run in the platform.

Here is a complete implementation of a sample Process webhook handler written in Go:

https://medium.com/media/967fa8e355c738d6862835f31bc0a492/href

  • Using just the standard library, notice how the HTTP request provides everything we need (FormValue, FormFile, etc)
  • The handler outputs data into the Vendor object, but real engines should return data conforming to the Veritone standard output in the documentation

Testing your engine

Before you deploy your engine to production, you’ll want to test it to ensure that it’s behaving as expected.

It is recommended that you write code that makes HTTP requests to the endpoints you have implemented. This approach allows you to run the code before deploying your engine to production, or even as part of a continuous integration environment.

You don’t need to include every field that the real requests might contain, instead just include the data that you are using in your engine.

Sometimes it’s handy to get your hands on the ins and outs of an API, and you can do this using the Engine Toolkit Test Console which is embedded in the engine binary.

The Engine Toolkit Test Console allows you to make real requests to your engine’s webhooks — a handy way to make sure they’re doing what you want them to

To activate the Engine Toolkit Test Console, we need to run our engine with some special parameters:

docker build -f Dockerfile -t your-engine .
docker run -e "VERITONE_TESTMODE=true" -p 9090:9090 -p 8080:8080 --name your-engine -t your-engine

The VERITONE_TESTMODE environment variable activates the Console, and since it is served on port 9090, we have to expose that with the -p 9090:9090 flag.

Once this is running, open your browser at https://localhost:9090/ to access the Console.

The Engine Toolkit Test Console automatically checks the engine’s webhooks and other required items (like the Manifest file) so you can be sure your engine is going to work when pushed to production.

What about more advanced cases?

If your engine needs to perform some more advanced interactions with the platform, you can still use the GraphQL API from within your Veritone Engine Toolkit powered engine.

The Veritone aiWARE platform documentation often mentions the Payload JSON, which contains details about the task you’re engine is being asked to do. It also contains a token, which is used to authenticate with the platform and it is this token that allows you to access the GraphQL services.

The payload comes into the Process webhook via the payload form variable. It is a JSON string, so you must unmarshal it before you can get at the data inside it.

In Go, the code to do this might look like this:

var payload struct {
    Token string
    // list the other fields you care about
    // from the payload here
}
err := json.Unmarshal([]byte(f.FormValue("payload"), &payload)
if err != nil {
    return errors.Wrap(err, "unmarshal payload")
}
// TODO: use payload.Token

Whichever language or tech stack you’re using, there will certainly be a library or package available that will parse HTTP requests for you, so consuming webhook requests and providing responses shouldn’t be hard.

If you need a little help or inspiration, you can check out the Sample engines from the Veritone Engine Toolkit GitHub repo.

Once you’ve extracted what you need from the payload, you can write your own code to interact with the APIs to fulfil the purpose of the engine.

More information?

If you’d like to learn more about these technologies, please check out the following links:

Building an engine with the Veritone Engine Toolkit was originally published in Machine Box on Medium, where people are continuing the conversation by highlighting and responding to this story.