PuppetDB

PuppetDB backend.

cumin.backends.puppetdb.CATEGORIES = ('C', 'F', 'O', 'P', 'R')

tuple() -- available categories in the grammar.

• C: shortcut for querying resources of type Class, equivalent of R:Class = class_path`.
• F: for querying facts.
• O: shortcut for querying resources of type Class that starts with Role::.
• P: shortcut for querying resources of type Class that starts with Profile::.
• R: for querying generic resources.

cumin.backends.puppetdb.GRAMMAR_PREFIX = 'P'

str -- the prefix associate to this grammar, to register this backend into the general grammar. Required by the backend auto-loader in cumin.grammar.get_registered_backends().

cumin.backends.puppetdb.OPERATORS = ('=', '>=', '<=', '<', '>', '~')

tuple() -- available operators in the grammar, the same available in PuppetDB API.

The ~ one is used for regex matching.

class cumin.backends.puppetdb.PuppetDBQuery(config, logger=None)

Bases: cumin.backends.BaseQuery

PuppetDB query builder.

The puppetdb backend allow to use an existing PuppetDB instance for the hosts selection. At the moment only PuppetDB v3 API are implemented.

• Each query part can be composed with the others using boolean operators (and, or, not)
• Multiple query parts can be grouped together with parentheses ((, )).
• A query part can be of two different types:
  • Hostname matching: this is a simple string that be used to match directly the hostname of the hosts in the selected backend. It allows for glob expansion (*) and the use of the powerful ClusterShell.NodeSet.NodeSet.
  • Category matching: an identifier composed by a category, a colon and a key, followed by a comparison operator and a value, as in F:key = value.

Some query examples:

• All hosts: *
• Hosts globbing: host10*
• ClusterShell.NodeSet.NodeSet syntax for hosts expansion: host10[10-42].domain
• Category based key-value selection:
  • R:Resource::Name: query all the hosts that have a resource of type Resource::Name.
  • R:Resource::Name = 'resource-title': query all the hosts that have a resource of type Resource::Name whose title is resource-title. For example R:Class = MyModule::MyClass.
  • R:Resource::Name@field = 'some-value': query all the hosts that have a resource of type Resource::Name whose field field has the value some-value. The valid fields are: tag, certname, type, title, exported, file, line. The previous syntax is a shortcut for this one with the field title.
  • R:Resource::Name%param = 'some-value': query all the hosts that have a resource of type Resource::Name whose parameter param has the value some-value.
  • C:Class::Name: special shortcut to query all the hosts that have a resource of type Class whose name is Class::Name. The Class::Name part is completely arbitrary and depends on the puppet hierarchy chosen. It's equivalent to R:Class = Class::Name, with the addition that the param and field selectors described above can be used directly without the need to add another condition.
  • O:Module::Name: special shortcut to query all the hosts that have a resource of type Class whose name is Role::Module::Name. The Module::Name part is completely arbitrary and depends on the puppet hierarchy chosen. It's equivalent to R:Class = Role::Module::Name, with the addition that the param and field selectors described above can be used directly without the need to add another condition, although usually roles should not have parameters in the role/profile Puppet paradigm.
  • P:Module::Name: special shortcut to query all the hosts that have a resource of type Class whose name is Profile::Module::Name. The Module::Name part is completely arbitrary and depends on the puppet hierarchy chosen. It's equivalent to R:Class = Profile::Module::Name, with the addition that the param and field selectors described above can be used directly without the need to add another condition.
  • F:FactName = value: query all the hosts that have a fact FactName, as reported by facter, with the value value.
  • Mixed facts/resources queries are not supported, but the same result can be achieved using the main grammar with multiple subqueries for the PuppetDB backend.
• A complex selection for facts: host10[10-42].*.domain or (not F:key1 = value1 and host10*) or (F:key2 > value2 and F:key3 ~ '^value[0-9]+')

__init__(config, logger=None)

Query constructor for the PuppetDB backend.

Parameters:according to parent cumin.backends.BaseQuery.__init__().

_add_bool(bool_op)

Add a boolean AND or OR query block to the query and validate logic.

Parameters:bool_op (str) -- the boolean operator to add to the query: and, or. Raises:cumin.backends.InvalidQueryError -- if an invalid boolean operator was found.

_add_category(category, key, value=None, operator='=', neg=False)

Add a category token to the query 'F:key = value'.

Parameters:

• category (str) -- the category of the token, one of CATEGORIES.
• key (str) -- the key for this category.
• value (str, optional) -- the value to match, if not specified the key itself will be matched.
• operator (str, optional) -- the comparison operator to use, one of OPERATORS.
• neg (bool, optional) -- whether the token must be negated.

Raises:

cumin.backends.InvalidQueryError -- on internal parsing error.

_add_hosts(hosts, neg=False)

Add a list of hosts to the query.

Parameters:

• hosts (ClusterShell.NodeSet.NodeSet) -- with the list of hosts to search.
• neg (bool, optional) -- whether the token must be negated.

_api_call(query)

Execute a query to PuppetDB API and return the parsed JSON.

Parameters:query (str) -- the query parameter to send to the PuppetDB API. Raises:requests.HTTPError -- if the PuppetDB API call fails.

_build(query_string)

Override parent class _build method to reset tokens and add logging.

Parameters:according to parent cumin.backends.BaseQuery._build().

_close_subgroup()

Handle subgroup closing.

_execute()

Concrete implementation of parent abstract method.

Parameters:according to parent cumin.backends.BaseQuery._execute(). Returns:with the FQDNs of the matching hosts. Return type:ClusterShell.NodeSet.NodeSet

static _get_grouped_tokens()

Return an empty grouped tokens structure.

Returns:the dictionary with the empty grouped tokens structure. Return type:dict

_get_query_string(group)

Recursively build and return the PuppetDB query string.

Parameters:group (dict) -- a dictionary with the grouped tokens. Returns:the query string for the PuppetDB API. Return type:str

_get_resource_query(key, value=None, operator='=')

Build a resource query based on the parameters, resolving the special cases for %params and @field.

Parameters:

• key (str) -- the key of the resource.
• value (str, optional) -- the value to match, if not specified the key itself will be matched.
• operator (str, optional) -- the comparison operator to use, one of OPERATORS.

Returns:

the resource query.

Return type:

str

Raises:

cumin.backends.InvalidQueryError -- on invalid combinations of parameters.

_get_special_resource_query(category, key, value, operator)

Build a query for Roles and Profiles, resolving the special cases for %params and @field.

Parameters:

• category (str) -- the category of the token, one of category_prefixes keys.
• key (str) -- the key of the resource to use as a suffix for the Class title matching.
• value (str, optional) -- the value to match in case %params or @field is specified.
• operator (str, optional) -- the comparison operator to use if there is a value, one of OPERATORS.

Returns:

the resource query.

Return type:

str

Raises:

cumin.backends.InvalidQueryError -- on invalid combinations of parameters.

_open_subgroup()

Handle subgroup opening.

_parse_token(token)

Concrete implementation of parent abstract method.

Parameters:according to parent cumin.backends.BaseQuery._parse_token(). Raises:cumin.backends.InvalidQueryError -- on internal parsing error.

base_url_template = 'https://{host}:{port}/v3/'

str -- string template in the str.format() style used to generate the base URL of the PuppetDB server.

category_prefixes = {'P': 'Profile', 'C': '', 'O': 'Role'}

dict -- dictionary with the mapping of special categories to title prefixes.

endpoint

Endpoint in the PuppetDB API for the current query.

Getter:Returns the current endpoint or a default value if not set. Setter:str: the value to set the endpoint to. Raises:cumin.backends.InvalidQueryError -- if trying to set it to an invalid endpoint or mixing endpoints in a single query.

endpoints = {'P': 'resources', 'C': 'resources', 'R': 'resources', 'O': 'resources', 'F': 'nodes'}

dict -- dictionary with the mapping of the available categories in the grammar to the PuppetDB API endpoints.

execute(query_string)

inherited Build and execute the query, return the NodeSet of FQDN hostnames that matches.

Parameters:query_string (str) -- the query string to be parsed and executed. Returns:with the FQDNs of the matching hosts. Return type:ClusterShell.NodeSet.NodeSet

grammar = Forward: ...

pyparsing.ParserElement -- load the grammar parser only once in a singleton-like way.

hosts_keys = {'nodes': 'name', 'resources': 'certname'}

dict -- dictionary with the mapping of the available endpoints of the PuppetDB API to the field to query to get the hostname.

cumin.backends.puppetdb.grammar()

Define the query grammar.

Backus-Naur form (BNF) of the grammar:

    <grammar> ::= <item> | <item> <and_or> <grammar>
       <item> ::= [<neg>] <query-token> | [<neg>] "(" <grammar> ")"
<query-token> ::= <token> | <hosts>
      <token> ::= <category>:<key> [<operator> <value>]

Given that the pyparsing library defines the grammar in a BNF-like style, for the details of the tokens not specified above check directly the source code.

Returns:the grammar parser. Return type:pyparsing.ParserElement

cumin.backends.puppetdb.query_class

Required by the backend auto-loader in cumin.grammar.get_registered_backends().

alias of PuppetDBQuery