pip install brewtils
Python Plugin Guide (Remote)
The goal of this section is to help you write a remote plugin from scratch. In order to write remote plugins there are a few things that you need.
Prerequisites
In order to build a plugin you’ll need a functioning Python environment. Setting that up is outside the scope of this tutorial; we recommend pyenv. You’ll also need access to a functioning Beer Garden. Check out the installation instructions for ways to stand up your own Beer Garden.
The simplest way to get started is to use the utility package we’ve created to make plugin building as easy as possible. The package is called brewtils
and lives on PyPi. You can find Brewtils API Documentation here. Add it to your environment like this:
Due to some bugs in how pip does special character substitution we recommend using version 9.0.0 or later. Use pip install -U pip to install the newest version.
|
Now you’re ready to start writing your plugin.
Don’t forget, if you’re writing a remote plugin, you’ll need to be able to access the Beer Garden REST Service |
Hello World!
This section will take you through setting up a simple Hello, World example.
Write the code
Plugins are basically just classes that you would like to expose over the network. So let’s create a simple, and classic "Hello, World!" example. Use your favorite text editor and open up a file called main.py
and add the following:
from brewtils.decorators import command, system (1)
@system (2)
class HelloWorldClient(object):
@command (3)
def hello_world(self):
print "Hello, World!"
return "Hello, World!"
1 | Imports from the brewtils.decorators module. These are all we need to "pluginify" our class. |
2 | The @system decorator marks this class as a Beer Garden plugin. |
3 | The @command decorator marks this method as a Command that’s part of the enclosing System. |
Once we have the client, we now need to create our entry point. Let’s open back up the previous file and add the following:
from brewtils.plugin import RemotePlugin
# Your implementation of the client goes here
def main():
plugin = RemotePlugin(HelloWorldClient(),
name='hello-world',
version='1.0.0',
description='My First Plugin',
bg_host="<HOST>",
bg_port="<PORT>",
ssl_enabled=<SSL_ENABLED>,
max_concurrent=5,
metadata={'foo': 'bar'})
plugin.run()
if __name__ == "__main__":
main()
To review what’s happening here, we add the import for RemotePlugin
to the top of our file, then create a main method that will simply create a HelloWorldClient
object. Then we pass the object into the RemotePlugin
which takes an object, a name of the plugin, plugin version, description and a few more things. That’s all the code you need to get started. Now we just need to move on to configuration.
A quick note on multi-threading. If you set max_concurrent=5 in the initializers, you are inherently saying that your functions are thread safe and can be processed in multiple threads (a maximum of 5 can run). It is up to the plugin developer to determine/create a plugin that can actually operate in a multi-threaded fashion.
|
Run the plugin
Run the plugin like this:
python main.py
It will start up and print some logging showing the plugin registering with Beer Garden. Congratulations! You’ve just deployed your first plugin!
Use the plugin
At this point you should see your plugin on the Available Systems
page in Brew View
. Click the big blue Explore hello_world commands
button to see a list of all commands available for the Plugin you made.
Since we only defined one command as part of this tutorial the only command listed should be the say_hello
command. Click the Make it Happen!
button to go to the page for that command.
The command page is where you can specify options and customization for individual executions of that command. Since we didn’t define any options (this command always prints 'Hello World!') the only customization available is the comment field. You can add free-form text here and it will be included as part of the request you’re about to generate.
Are you ready? Click the Make Request
button once you’re ready.
Making a request takes you to the Request page for the request you just generated. You can see the unique ID as part of the page title. You should see the status start as IN PROGRESS
and then change to SUCCESS
once the request completes. Also notice that the output changes when the request is finished.
If you didn’t catch those changes on the first try, don’t worry. Use the Pour it Again
button in the top-right corner to go back to the command screen you just left. From here you can use the Make Request
button to make another request.
This command doesn’t have any parameters, but for commands that do the Pour it Again button will default them to exactly how they were for the original request.
|
Stop
The best way to stop a plugin is to use the Systems Management page to send a stop message. In Brew View
find the Systems Management
option under the Administration
menu. Then click the stop icon next to the hello-world
listing.
You should see your plugin log that it has terminated and stop executing, and if you go to the main page in Brew View
you should see the hello-world
plugin is 'STOPPED'.
You can also use Ctrl-c to stop the plugin from the terminal. This works, but it doesn’t tell Beer Garden that the plugin is stopping. You’ll still see the plugin terminate and stop executing, but the status in Brew View will still be 'RUNNING'. After a short period the status will change to 'UNRESPONSIVE'.
|
Exception Handling
It is important to be able to tell Beer Garden when something on your system goes wrong. brewtils
takes advantage of Python’s exceptions in order to handle command malfunctions. So if you have a function:
def my_error(self):
raise ValueError("Something went wrong!")
This will result in the request status turning to "ERROR" and its output will be Something went wrong!
. It is expected that plugins will throw errors as a way to notify Beer Garden that something has gone wrong.
If you choose to handle errors and not throw, you will notice something that may be quite confusing to your plugins users. Let’s have an example:
def my_error(self, x):
if x is None:
# This is actually an error, but we will short-circuit
return "An error occurred."
return x
If x
being None
is an error, then you should throw an error. Otherwise, the request will be marked as SUCCESS
while the output will say An error occurred
If your command has a JSON
output type, then Beer Garden will attempt to format your exception as a JSON error message. Here’s an example:
@command(command_type="JSON")
def my_error(self):
raise ValueError("Error Message")
If you call this method, you’ll still notice a status of ERROR
but the output will be something like:
{
"message": "Error Message",
"attributes": {}
}
If you’re asking what the attributes
entry is supposed to represent, it will take the dict
of the exception and attempt to jsonify it. Let’s say you have a custom exception class like the following:
class MyError(Exception):
def __init__(self, *args, **kwargs):
self.foo = kwargs.pop("foo")
self.bar = kwargs.pop("bar")
Then you throw that error during a command, Beer Garden will modify the output as per the following:
{
"message": "Error Message"
"attributes": {"foo": "foo_value", "bar": "bar_value"}
}
If your attributes are not JSON serializable, then a string representation of the dictionary will be provided to the attributes. |
Making your plugin more descriptive
In order to help users of Beer Garden understand how to utilize your plugin, you’ll need to provide some basic information about your plugins. There are lots of ways to specify lots of different information that your plugin can do. So we’ll take it one at a time.
Adding a command description
There are two ways you can add a description to your command. One way is by using pydoc comments
class HelloWorldClient(object):
@command
def hello_world(self):
"""Prints Hello, World!"""
print("Hello, World!")
The description will only take the first line of your comment |
Or you can specify a description via the @command
decorator
class HelloWorldClient(object):
@command(description="Prints Hello, World!")
def hello_world(self):
print("Hello, World!")
Describing a parameter
Obviously, most methods will have required arguments. You can utilize the @parameter
decorator to describe the argument as in the following:
class HelloWorldClient(object):
@command(description="Echos the message you pass to it")
@parameter(key="message",
description="Message to print and return")
def echo(self, message):
print(message)
return message
@parameter
only requires the key
option be passed to it. The key
must match the parameter’s name in the method. There are lots of different options when considering the @parameter
. Check out the following table for a high- level summary:
Argument | Required? | Options | Default | Description |
---|---|---|---|---|
Y |
N/A |
N/A |
Specifies the Argument Name |
|
N |
[String, Integer, Float, Boolean, Any, Dictionary] |
Any |
Specifies the type of Parameter |
|
N |
[True, False] |
False |
Specifies if the parameter is a list |
|
N |
N/A |
key |
Specifies a Pretty way to refer to the key |
|
N |
[True, False] |
False |
Specifies if the parameter is required |
|
N |
N/A |
N/A |
The Default value of the Parameter |
|
N |
N/A |
N/A |
A short Description of the Parameter |
|
N |
N/A |
N/A |
A list of possible values |
|
N |
N/A |
N/A |
If parameter comes in as a kwarg |
|
N |
N/A |
N/A |
A python Object that has a parameters list |
|
N |
[True, False] |
False |
Specifies if this parameter can be null |
|
N |
Integer |
N/A |
Specifies maximum (See detailed for more info) |
|
N |
Integer |
N/A |
Specifies minimum (See detailed for more info) |
|
N |
N/A |
N/A |
Specifies regex to validate against this value |
|
N |
[textarea] |
N/A |
Specifies the form type to render for this plugin |
Key argument
The key
argument to @parameter
is the only required parameter. It must match the name of an argument in whatever method it is decorating. This is how users of your plugin will identify the parameter they would like to set.
@parameter(key="message")
def do_something(self, message):
print(message)
return message
Type argument
Setting the type
field for a Parameter will let Beer Garden do a couple of things.
First, it lets Beer Garden perform type validation on that parameter. If the value does not match the type (and can’t be converted sensibly) then the Request will be rejected.
Second, it allows the UI to use the best form element for that type of data. For example, you could use a string to represent a date, but setting type
to 'date' means the UI will use a nice datepicker form element.
Finally, several validation constraints can only be applied to specific types. The minimum and maximum constraints are good examples of this - they don’t make sense for booleans, but they definitely do for integers!
@parameter(key="number", type="Integer")
def multiply_by_zero(self, number):
return number * 0
This table outlines the valid type
options.
Type | Valid Constraints | Notes |
---|---|---|
String |
minimum, maximum, regex |
Minimum and Maximum refer to length |
Integer |
minimum, maximum |
|
Float |
minimum, maximum |
|
Boolean |
||
Date |
Form control will not have 'time' option |
|
DateTime |
||
Dictionary |
JSON Object |
|
Any |
Valid JSON (Object, Array, String), number, literal |
The default type is Any . This gives the most flexibility, but it’s a good idea to always specify a type to take advantage of the benefits described above.
|
Multi argument
The multi
field let’s Beer Garden know that the parameter should be a list. Most of the other fields stay the same and continue to describe the individual items in the list.
@parameter(key="list_of_strings", multi=True, type="String")
def do_something(self, list_of_strings):
for s in list_of_strings:
print(s)
Some of the fields do change meaning when you’ve specified that multi is true. See the below table for a more detailed description. |
Argument | How is it changed? |
---|---|
choices |
Choices specify the only valid values, no value can be repeated. |
maximum |
Specifies Maximum length of the list |
minimum |
Specifies Minimum length of the list |
Display name argument
The display_name
field allows you control over how Beer Garden renders the name of the field. This is useful if your argument has a less-than-useful name from the end-users perspective.
@parameter(key="foo", display_name="Name")
def do_something(self, foo):
print("Hi!, my name is: %s" % foo)
Optional argument
The optional
field allows you to specify whether or not the parameter is optional or required. The default depends on if there is a default value in the method definition.
@parameter(key="foo", optional=True, nullable=True))
def do_something(self, foo):
# By default, foo would not be optional but
# it is specified in the param so it's assumed
# the developer will handle the None case.
if foo is None:
print("foo is empty!")
else:
print(foo)
If a default is passed in, then optional will be set to True by default.
If you specify that something is optional, then it must also be nullable if no default is specified. |
Default argument
The default
field allows you to specify the default value for a parameter if it is not given by a user. If there is a default value in the method definition then it will use that.
@parameter(key="foo")
def do_something(self, foo="bar"):
print(foo)
In the above case, if someone utilizes this command but does not pass Beer Garden the foo
parameter, then Beer Garden will default it to bar
. Below is another example of how to use the default argument.
@parameter(key="foo", default="bar"))
def do_something(self, foo):
print(foo)
These are functionally equivalent for Beer Garden.
Description argument
The description
field adds a description to the plugin parameter you are defining.
@parameter(key="foo", description="Your first name")
def do_something(self, foo):
pass
Basic Usage
The easiest way to use choices
is to pass a list of values:
@parameter(key="output_type", choices=["json", "xml"])
def format(self, obj, output_type):
if output_type == "json":
return jsonify(obj)
elif output_type == "xml":
return xmlify(obj)
Sometimes it’s useful to have the display text (what shows up in the UI) be different from the 'real' value (what gets sent to the plugin). To do this, instead of a list of literal values just pass a list of objects with text
and value
keys:
@parameter(key="output_type", choices=[
{"text": "The best", "value": "json"},
{"text": "The worst", "value": "xml"}])
def format(self, output_type):
pass
Additional Configuration
There are some configuration options that control how choices
works. Beer Garden will pick sensible defaults, but to tweak them pass a dictionary to choices
:
@parameter(key="output_type",
choices={'type': 'static', 'value': ['json','xml']})
def format(self, output_type):
...
That way you can add additional key/value pairs to the choices dictionary.
Choices Type
You probably noticed the 'type': 'static'
entry above. Beer Garden is able to figure out exactly what to do when you pass a list of values to choices
, but it needs a hint when you use the dictionary configuration. There are a couple of other ways to populate the choices list (more on those in a bit) so you need to be explicit.
The example above is using the static
type, which tells Beer Garden to expect a list of values in the value
attribute. This is functionally identical to passing a list of values to choices
directly.
The other choice types will be explained in detail in the Choice Sources section.
Display
When you use choices
the UI form control can be a typeahead or a select. To specify which to use just set the display
key:
@parameter(key="output_type",
choices={'type': 'static', 'value': ['json','xml'],
'display': 'typeahead'})
def format(self, output_type):
...
Strictness
The strict
configuration controls whether values that aren’t explicitly listed are allowed. Setting strict
to False will result in a typeahead control that will use the choices
values but still allow any text to be submitted.
@parameter(key="output_type",
choices={'type': 'static', 'value': ['json','xml'],
'strict': False})
def format(self, output_type):
...
Setting strict to False for a select won’t affect the display, but the strict value also controls validation on the backend.
|
Choice Sources
In all the examples so far the list of choices has been a literal list of values. That’s useful, but it’s also useful to have values that can change at runtime. In order to do that you need to provide choices
with instructions on how to populate the choice list instead of the list itself.
In all cases the result of the choices operation must be a valid choices list.
|
URL
Specifying a URL will tell the browser to load choices using an HTTP GET request. You can use type 'url' if using dictionary configuration or just pass the URL as a string:
@parameter(key="p1", choices='https://test.com/p1.json')
@parameter(key="p2", choices={"type": "url",
"value": 'https://test.com/p2.json'})
def format(self, p1, p2):
...
Be aware that the user’s browser will be making this request. So if the Beer Garden UI is being accessed at a secure (https) address then a request to a non-secure (http) URL will likely fail due to mixed-content restrictions. |
Command
Specifying a command will load choices by making a request to the current system. You can use type 'command' if using dictionary configuration or just pass the command as a string. If you’re not using choice parameters (more on those in a minute) you can omit the parenthesis for brevity.
@parameter(key="p1", choices="get_choices()")
@parameter(key="p2", choices={"type": "command",
"value": "get_choices"})
def format(self, p1, p2):
...
@command
def get_choices(self):
return [
{"text": "The best", "value": "json"},
{"text": "The worst", "value": "xml"}
]
Currently you must use a command from the same system (this restriction will be removed in a future release - see issue 269). |
Choice parameters
It’s often useful to have the choices for one parameter depend on the current value of another. To do that you can use choice parameters.
To create a reference on another parameter enclose its key in ${}
. How the parameter is passed depends on what choice source is being used.
For 'command' types the parameter will be passed as an argument to the command. For example, suppose you have two parameters: day_type
and day_of_week
. You’d like the choices for day_of_week
to depend on what the user has selected for day_type
:
@command
def get_days(self, type):
if type == "Weekday":
return ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"]
elif type == "Weekend":
return ["Saturday", "Sunday"]
else:
raise Exception("Huh?")
@parameter(key="day_type", choices=["Weekday", "Weekend"])
@parameter(key="day_of_week", choices="get_days(type=${day_type})")
def my_command(self, day_type, day_of_week):
do_something(day_of_week)
return "All done!"
For 'url' types the choice parameter should be used as a query parameter:
@parameter(key="day_type", choices=["Weekday", "Weekend"])
@parameter(key="day_of_week",
choices="https://getthedays.com?type=${day_type}")
def my_command(self, day_type, day_of_week):
do_something(day_of_week)
return "All done!"
Choice parameters also enable using a static choices dictionary with one parameter used as the dictionary key. To do this use type static
and pass the dictionary as the value. Since we can construct the dictionary before defining the command
we can rework the day_of_week
example to look like this:
day_dict = {
"Weekday": ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"],
"Weekend": ["Saturday", "Sunday"]
}
@parameter(key="day_type", choices=["Weekday", "Weekend"])
@parameter(key="day_of_week", choices={'type': 'static',
'value': day_dict,
'key_reference': '${day_type}'})
def my_command(self, day_type, day_of_week):
do_something(day_of_week)
return "All done!"
When using a choices dictionary the None
key can be used to specify the allowed values when the reference key is null
. For example, if we wanted to modify the day_of_week
example to additionally allow any day to be selected if day_type
was null
we could do this:
day_dict = {
"Weekday": ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"],
"Weekend": ["Saturday", "Sunday"],
None: ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday",
"Saturday", "Sunday"]
}
@parameter(key="day_type", choices=["Weekday", "Weekend"],
nullable=True)
@parameter(key="day_of_week", choices={'type': 'static',
'value': day_dict,
'key_reference': '${day_type}'})
def my_command(self, day_type, day_of_week):
do_something(day_of_week)
return "All done!"
Is kwarg argument
The is_kwarg
argument allows you to name a keyword argument that is otherwise unspecified. This is useful if you take keyword args, but want to call out the normal use-case ones more explicitly while still being compatible to other python libraries calling you.
@parameter(key="foo", is_kwarg=True)
def do_something(self, **kwarg):
foo = kwarg.pop("foo")
print(foo)
Model argument
The model
argument allows you to specify some structure for a complicated object. Have a look at the following for an example of how to use the model
field.
from brewtils.models import Parameter
class Person(object):
name = Parameter(key="name",
type="String",
description="Person's name")
age = Parameter(key="age",
type="Integer",
description="Person's age")
class ExampleClient(object):
@parameter(key="person", model=Person)
def greet(self, person):
print("Hello %s" % person.name)
It is assumed that if you have a model, that the type is "Dictionary" |
Nullable argument
The nullable
argument allows you to specify if the parameter can be null. If the argument is allowed to be null, then you must tell us this is possible. The default is assuming that parameters cannot be null.
If there is a default value for a parameter, then nullable
is set to True.
@parameter(key="foo", nullable=True))
def do_something(self, foo):
if foo is None:
print("That's ok!")
else:
print("That's ok too!")
Maximum argument
The maximum
argument allows you to specify the maximum value for a parameter. This meaning changes based on the type
and whether or not the multi
flag is enabled. If the multi
flag is enabled, then maximum is referring to the list length maximum. Otherwise, if type
is integer, it will compare the value of the parameter to the maximum. Otherwise if the type
is a string, it will ensure the length of the string is within bounds.
@parameter(key="foo", type="String", maximum=1)
@parameter(key="bar", type="Integer", maximum=1)
@parameter(key="bazs", type="String", maximum=1)
def do_something(self, foo, bar, bazs):
# guarantees that foo is 1 character at most
# guarantees that bar is no more than 1
# guarantees that bazs is no more than 1 item long
print(foo)
print(bar)
print(bazs)
Minimum argument
The minimum
argument allows you to specify the minimum value for a parameter. This meaning changes based on the type
and whether or not the multi
flag is enabled. If the multi
flag is enabled, then minimum is referring to the list length minimum. Otherwise, if type
is integer, it will compare the value of the parameter to the minimum. Otherwise if the type
is a string, it will ensure the length of the string is within bounds.
@parameter(key="foo", type="String", minimum=1)
@parameter(key="bar", type="Integer", minimum=1)
@parameter(key="bazs", type="String", minimum=1)
def do_something(self, foo, bar, bazs):
# guarantees that foo is at least 1 character
# guarantees that bar is no less than 1
# guarantees that bazs is no less than 1 item long
print(foo)
print(bar)
print(bazs)
Regex argument
The regex
argument allows you to specify a regex that the parameter must pass in order to be considered valid.
@parameter(key="ip", regex=r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$')
def do_something(self, ipv4):
print("This is a valid IPv4: %s" % ipv4)
Form Input Type Argument
The form_input_type
field allows you control over how Beer Garden renders the text-field. This is useful if you have some string field that is actually quite long. Currently the only supported type is textarea
.
@parameter(key="comment", type="String", form_input_type="textarea")
def do_something(self, comment):
print("This is a long comment: %s" % comment)
Customizing Command Form
One of the benefits of Beer Garden is the web form that’s automatically created for each Command. We try hard to make sure every combination of parameters will result in a form that makes sense and to give Plugin developers the tools they need to express their data. However, there may be a bug in a specific corner case, or you may want to change the form directly to fit your application. The following sections can help in those cases.
Schema and/or Form Modification
The form is generated using a package called Angular Schema Form (ASF). A complete tutorial on ASF is outside the scope of this document, so if you’re unfamiliar with how ASF works please consult their documentation. At a very high level: ASF takes a JSON Schema (data model) and Form (presentation and styling) and uses those to construct the form that’s presented on the Command View page. Behind the scenes the current data state is stored in the Model, which is updated whenever the user makes changes. This Model is what’s shown in the Preview pane.
The normal Beer Garden behavior is to use the Command definition to construct a Schema and Form for a particular Command. However, Beer Garden allows overriding either of these on a per-command basis.
When overriding the Schema or Form definition, it is highly recommended to use the Beer Garden-generated Schema or Form as a starting point. Set the debug_mode option to true in the Brew-view configuration to enable extra displays on the Command View page.
|
For example, a Command defined like this:
@parameter(key="message", type="String", optional=False)
def echo(self, message):
return message
Will result in these definitions:
"message": {
"title": "message",
"optional": false,
"nullable": false,
"type": [
"string",
"null"
],
"required": true
}
{
"key": [
"parameters",
"message"
]
}
In order to make the field read-only you can add the readonly
key to the form definition:
@command(form={"key": ["parameters", "message"], "readonly": True})
@parameter(key="message", type="String", optional=False)
def echo(self, message):
return message
See the Angular Schema Form documentation for examples of ways to modify the Form.
Changing the Schema or Form does not change the data constraints of the Command. For this reason it’s recommended to not modify the Schema, as that can cause the Request to fail Beer Garden validation during creation. |
Don’t forget to include all parameters in the form definition. The form definition can be an array as well as a dictionary.
|
Command Page Replacement
Beer Garden also supports completely replacing the Command View page with custom HTML. This is intended to allow developers to implement functionality beyond what Angular Schema Form can provide.
This feature should be considered a Beta capability. If you’re interested in using this feature in production please contact the Beer Garden team. |
To use an alternate Command View page pass a path to the file as the template
argument to the @command
decorator:
@command(template='./resources/minimalist.html')
def echo_minimalist(self, message):
return message
The minimalist.html
page should define a way for the user to submit a POST request to /api/v1/requests
with the Request to be created in the body. The request content-type
can be either 'application/json' or 'application/x-www-form-urlencoded'.
Please note that the template HTML will still be rendered in the context of the Beer Garden Angular application. Since this HTML is outside Angular control it’s treated as untrusted, which causes certain elements to be removed. If you understand the risks associated with removing this restriction and want to allow these elements you can set the allow_unsafe_templates
option to true
in the Brew-view configuration.
Please make sure you understand the implications of allowing unsafe templates, especially in a production environment. Again, please reach out to the Beer Garden team if you’re interested in using this capability. |
External Logging
Starting in Beer Garden 2.1.0, it is possible to tell your plugins how to log from the Beer Garden application itself. Beer Garden provides a new api (/api/v1/config/logging
) which will respond with a logging configuration that plugins can optionally use. Checkout our Swagger documentation for the complete details of this new endpoint.
This allows you to do one-time configuration of a logger in Beer Garden configuration files and propagate that logging configuration to all of your plugins. To use it, you simply add the following to your entry point.
from brewtils.log import setup_logger
# Your client definition Here
if __name__ == "__main__":
setup_logger(
bg_host='localhost',
bg_port=2337,
system_name='my_name',
ca_cert=None,
client_cert=None,
ssl_enabled=None,
)
# Init your plugin as normal.
This tells Beer Garden to setup a root logger based on what is returned from the endpoint at api/v1/config/logging
. Checkout the configuration section for more information on how to configure this. This is mostly useful for remote plugins.