Remote Gardens

Parent and Child Gardens

Beer Garden has the ability to connect other Beer Gardens in a Parent-Child relationship. The approach Beer Garden takes for command and control is Parents can only send Operations to Children, and Children can only send Events to Parents.

Another design approach is that Children have no true concept of Parents, only a location to publish events. Child Beer Gardens do not know what plugins are deployed on the Parent and can not create a System Client connection to Parents.

Another design approach is that Parents have no concept of Grand Children. Beer Garden promotes all Systems that it can execute against, not what systems are connected to itself and children. So if a Child has 3 Beer Gardens connected, the child tells the Parent it has the ability to route to all of the systems. The Parent will trust the Child Beer Garden to the routing appropriately.

Internal Routing

The Router requires all objects to utilize the new standard Operation class.

Operations that can be forwarded include:

INSTANCE_START
INSTANCE_STOP
REQUEST_CREATE
SYSTEM_DELETE
GARDENS_SYNC

How to determine where to route

Before any Operation is executed, Beer Garden has to determine if this is an Operation that it can fulfill. For all requests the Targeted Garden must be identified. If the Operation is not be forwarded, and is not for the local Beer Garden, then the Operation can not be routed. Each route-able Brewtils class has a different approach to determining the Target Beer Garden.

Local Only Operations

There are a handful of Operations that will always be handled by the Local Beer Garden. The criteria utilized to determine this is:

  • Is the Operation a READ from the database?

  • Is the Operation managing scheduled jobs?

  • Is the Operation managing Gardens?

  • Is the Operation kicking off Local Garden Actions?

  • Is the Operation managing local only resources?

Systems and Instances

The combination of Namespace/System/Version is tied to a hosting Beer Garden. This information is stored within the Garden object. For System and Instance based Operations, this is a quick look up against the cached Garden information in the Router class.

Requests

Operations to CREATE requests must first identify the Namespace/System/Version. Then that information is utilized to determine the Target Beer Garden, same as the Systems and Instances is looked up.

What does a Child emit to the Parent?

Children forward all internal events to their Parents. If there are events that should not be forwarded, then that can be configured in the configuration of Beer Garden.

How does a Parent connect to a Child?

Child configuration is done through the User Interface. If an administrator does not configure the child connection class, then the Systems connected through that Child are inaccessible. These Systems will appear on the UI, and a user can create Requests against them, but will be prompted with an error message stating the invalid connection.

How does a Child connect to a Parent?

Parent configuration is down through the Child’s config.yaml file. This configuration sets a destination in which to fowward the internal events. Then the parent can receive/process those events to maintain synchronization with the child.

Things to highlight in the configuration of the Child.

  • The enabled flag must be set to True for the events to be forwarded

  • SSL Configuration is not inherited from other sections on the config, so set it if it is required.

  • If SSL is required for the Parent, and not for the Child, setting SSL on the Parent config does not enable SSL for the Childs API

  • STOMP only requires the SEND destination to be populated.

  • STOMP HEADERS field will populate all outbound messages with those header properties.

Filtering Events

The default operating procedure is to forward all events generated by the Child to the Parent. That way the events can be utilized to recreate the transactions that occurred. If you are running in an environment that has limited bandwidth, the Events can be filtered down in the Parent Configuration.

config.yaml
parent:
  http:
    skip_events:
    - INSTANCE_INITIALIZED
    - REQUEST_STARTED

Here is a partial list of the events that are generated and the impact if filtered.

Critical = Blocks expected behaviors

Awareness = Utilized for informational events

None = Not utilized by Parent

Event Skip Event Impact Justification

REQUEST_COMPLETED

Critical

If request is spawned by Parent, the parent will never receive Completion event to return to System Client or User Interface

REQUEST_CANCELED

Critical

If request is spawned by Parent, the parent will never receive Cancellation event to return to System Client or User Interface

GARDEN_UPDATED

Critical

These updates are utilized for Routing decisions

GARDEN_SYNC

Critical

These updates are utilized for Routing decisions

REQUEST_CREATED

Awareness

If skipped, Request will appear during REQUEST_STARTED, REQUEST_UPDATED, or REQUEST_COMPLETED

REQUEST_STARTED

Awareness

If skipped, Request will appear during REQUEST_UPDATED, or REQUEST_COMPLETED

REQUEST_UPDATED

Awareness

If skipped, Request will appear during REQUEST_COMPLETED

INSTANCE_UPDATED

Awareness

If skipped, status changes will not be updated on Parent

SYSTEM_CREATED

Awareness

If skipped, new deployed systems will not appear on the Parent. Will require Garden Sync to get the latest values

SYSTEM_UPDATED

Awareness

If skipped, status and instance changes will not be updated on Parent. Will require Garden Sync to get the latest values

SYSTEM_REMOVED

Awareness

If skipped, deleted systems will not appear on the Parent. Will require Garden Sync to get the latest values

GARDEN_STARTED

Awareness

These are utilized to update the status of the Garden, but does not impact routing

GARDEN_STOPPED

Awareness

These are utilized to update the status of the Garden, but does not impact routing

INSTANCE_INITIALIZED

None

Utilized for internal processes only

INSTANCE_STARTED

None

Utilized for internal processes only

INSTANCE_STOPPED

None

Utilized for internal processes only

QUEUE_CLEARED

None

Utilized for internal processes only

ALL_QUEUES_CLEARED

None

Utilized for internal processes only

GARDEN_CREATED

None

Utilized for internal processes only

GARDEN_REMOVED

None

Utilized for internal processes only

GARDEN_UNREACHABLE

None

Utilized for internal processes only

GARDEN_ERROR

None

Utilized for internal processes only

GARDEN_NOT_CONFIGURED

None

Utilized for internal processes only

ENTRY_STARTED

None

Utilized for internal processes only

ENTRY_STOPPED

None

Utilized for internal processes only

JOB_CREATED

None

Utilized for internal processes only

JOB_DELETED

None

Utilized for internal processes only

JOB_PAUSED

None

Utilized for internal processes only

JOB_RESUMED

None

Utilized for internal processes only

PLUGIN_LOGGER_FILE_CHANGE

None

Utilized for internal processes only

RUNNER_STARTED

None

Utilized for internal processes only

RUNNER_STOPPED

None

Utilized for internal processes only

RUNNER_REMOVED

None

Utilized for internal processes only

Filtering events is at your own risk. Beer Garden reserves the right to utilize these events in the future for critical operations between Parent and Children. If you are running into bandwidth issues or want to limit the number of events being processed. Below is the current list of events that are safe to skip in Beer Garden V3.1.

config.yaml
parent:
  http:
    skip_events:
    - INSTANCE_INITIALIZED
    - INSTANCE_STARTED
    - INSTANCE_STOPPED
    - QUEUE_CLEARED
    - ALL_QUEUES_CLEARED
    - GARDEN_CREATED
    - GARDEN_REMOVED
    - GARDEN_UNREACHABLE
    - GARDEN_ERROR
    - GARDEN_NOT_CONFIGURED
    - ENTRY_STARTED
    - ENTRY_STOPPED
    - JOB_CREATED
    - JOB_DELETED
    - JOB_PAUSED
    - JOB_RESUMED
    - PLUGIN_LOGGER_FILE_CHANGE
    - RUNNER_STARTED
    - RUNNER_STOPPED
    - RUNNER_REMOVED

Filtering Events Based on Commands

There is a Command Publishing Blocklist that allows commands to be added to the list through the UI found in the admin section of the navigation bar under Command Publishing Blocklist. Any events involving requests for commands in the blocklist will be filtered out and not sent to the parent.

Once a command has been added to the blocklist, tasking status will no longer be available on the parent garden. If a blocked command is tasked through the parent garden, the request will still be forwarded to the child and executed. However, the parent will always show a status of CREATED, as the child will no longer be publishing task updates back up to the parent.