PuppetDB

PuppetDB backend.

cumin.backends.puppetdb.CATEGORIES = ('C', 'F', 'O', 'P', 'R')

available categories in the grammar.

  • C: shortcut for querying resources of type Class, equivalent of R:Class = class_path`.

  • F: for querying facts.

  • O: shortcut for querying resources of type Class that starts with Role::.

  • P: shortcut for querying resources of type Class that starts with Profile::.

  • R: for querying generic resources.

Type

tuple()

cumin.backends.puppetdb.OPERATORS = ('=', '>=', '<=', '<', '>', '~')

available operators in the grammar, the same available in PuppetDB API.

The ~ one is used for regex matching.

Type

tuple()

class cumin.backends.puppetdb.ParsedString(string, is_quoted)[source]

Bases: object

Simple string wrapper which can communicate if a string should be enquoted downstream.

Constructor for ParsedString.

Parameters
  • string (str) -- The string to store in this object.

  • is_quoted (bool) -- Whether the output should be quoted when this is converted to a string.

capwords(sep)[source]

Perform capwords operation on internal value and return a new ParsedString.

Parameters

according to string.capwords().

replace(old, new, count=- 1)[source]

Perform replace operation on internal value and return a new ParsedString.

Parameters

according to str.replace().

cumin.backends.puppetdb.grammar()[source]

Define the query grammar.

Backus-Naur form (BNF) of the grammar:

    <grammar> ::= <item> | <item> <and_or> <grammar>
       <item> ::= [<neg>] <query-token> | [<neg>] "(" <grammar> ")"
<query-token> ::= <token> | <hosts>
      <token> ::= <category>:<key> [<operator> <value>]
      <value> ::= <numeric> | <bareword> | <quoted_string> | <unquoted_string>

Given that the pyparsing library defines the grammar in a BNF-like style, for the details of the tokens not specified above check directly the source code.

Returns

the grammar parser.

Return type

pyparsing.ParserElement

class cumin.backends.puppetdb.PuppetDBQuery(config)[source]

Bases: cumin.backends.BaseQuery

PuppetDB query builder.

The puppetdb backend allow to use an existing PuppetDB instance for the hosts selection. The supported PuppetDB API versions are 3 and 4. It can be specified via the api_version configuration key, if not configured, the v4 will be used.

  • Each query part can be composed with the others using boolean operators (and, or, not)

  • Multiple query parts can be grouped together with parentheses ((, )).

  • A query part can be of two different types:

    • Hostname matching: this is a simple string that be used to match directly the hostname of the hosts in the selected backend. It allows for glob expansion (*) and the use of the powerful ClusterShell.NodeSet.NodeSet.

    • Category matching: an identifier composed by a category, a colon and a key, followed by a comparison operator and a value, as in F:key = value.

  • Values may be of various types supported by PuppetDB (numerics, boolean, and strings) for example:

    • Booleans: true, false

    • Strings: 'a string', and unquoted single words that aren't true or false and do not start with an integer.

    • Numeric values: 15, 23.5, 0, 0xfa Note: hexadecimal and octal numbers are supported by cumin but converted into normal integers. Some fields in PuppetDB may have hex or octal stored as strings, and should be quoted such as '0xfa'.

    Note: PuppetDB may or may not support a particular value type for a particular resource.

Some query examples:

  • All hosts: *

  • Hosts globbing: host10*

  • ClusterShell.NodeSet.NodeSet syntax for hosts expansion: host10[10-42].domain

  • Category based key-value selection:

    • R:Resource::Name: query all the hosts that have a resource of type Resource::Name.

    • R:Resource::Name = 'resource-title': query all the hosts that have a resource of type Resource::Name whose title is resource-title. For example R:Class = MyModule::MyClass.

    • R:Resource::Name@field = 'some-value': query all the hosts that have a resource of type Resource::Name whose field field has the value some-value. The valid fields are: tag, certname, type, title, exported, file, line. The previous syntax is a shortcut for this one with the field title.

    • R:Resource::Name%param = 'some-value': query all the hosts that have a resource of type Resource::Name whose parameter param has the value some-value.

    • C:Class::Name: special shortcut to query all the hosts that have a resource of type Class whose name is Class::Name. The Class::Name part is completely arbitrary and depends on the puppet hierarchy chosen. It's equivalent to R:Class = Class::Name, with the addition that the param and field selectors described above can be used directly without the need to add another condition.

    • O:Module::Name: special shortcut to query all the hosts that have a resource of type Class whose name is Role::Module::Name. The Module::Name part is completely arbitrary and depends on the puppet hierarchy chosen. It's equivalent to R:Class = Role::Module::Name, with the addition that the param and field selectors described above can be used directly without the need to add another condition, although usually roles should not have parameters in the role/profile Puppet paradigm.

    • P:Module::Name: special shortcut to query all the hosts that have a resource of type Class whose name is Profile::Module::Name. The Module::Name part is completely arbitrary and depends on the puppet hierarchy chosen. It's equivalent to R:Class = Profile::Module::Name, with the addition that the param and field selectors described above can be used directly without the need to add another condition.

    • F:FactName = value: query all the hosts that have a fact FactName, as reported by facter, with the value value.

    • Mixed facts/resources queries are not supported, but the same result can be achieved using the main grammar with multiple subqueries for the PuppetDB backend.

  • All hosts with physicalcorecount fact greater than 2: F:physicalcorecount > 2

  • A complex selection for facts: host10[10-42].*.domain or (not F:key1 = value1 and host10*) or (F:key2 > value2 and F:key3 ~ '^value[0-9]+')

Query constructor for the PuppetDB backend.

Parameters

according to parent cumin.backends.BaseQuery.__init__().

base_url_template = '{scheme}://{host}:{port}'

string template in the str.format() style used to generate the base URL of the PuppetDB server.

Type

str

endpoints = {'C': 'resources', 'F': 'nodes', 'O': 'resources', 'P': 'resources', 'R': 'resources'}

dictionary with the mapping of the available categories in the grammar to the PuppetDB API endpoints.

Type

dict

category_prefixes = {'C': '', 'O': 'Role', 'P': 'Profile'}

dictionary with the mapping of special categories to title prefixes.

Type

dict

grammar = Forward: ...

load the grammar parser only once in a singleton-like way.

Type

pyparsing.ParserElement

property endpoint

Endpoint in the PuppetDB API for the current query.

Getter

Returns the current endpoint or a default value if not set.

Setter

str: the value to set the endpoint to.

Raises

cumin.backends.InvalidQueryError -- if trying to set it to an invalid endpoint or mixing endpoints in a single query.

_open_subgroup()[source]

Handle subgroup opening.

_close_subgroup()[source]

Handle subgroup closing.

static _get_grouped_tokens()[source]

Return an empty grouped tokens structure.

Returns

the dictionary with the empty grouped tokens structure.

Return type

dict

_build(query_string)[source]

Override parent class _build method to reset tokens and add logging.

Parameters

according to parent cumin.backends.BaseQuery._build().

_execute()[source]

Concrete implementation of parent abstract method.

Parameters

according to parent cumin.backends.BaseQuery._execute().

Returns

with the FQDNs of the matching hosts.

Return type

ClusterShell.NodeSet.NodeSet

_add_category(category, key, value=None, operator='=', neg=False)[source]

Add a category token to the query 'F:key = value'.

Parameters
  • category (str) -- the category of the token, one of CATEGORIES.

  • key (str) -- the key for this category.

  • value (str, optional) -- the value to match, if not specified the key itself will be matched.

  • operator (str, optional) -- the comparison operator to use, one of OPERATORS.

  • neg (bool, optional) -- whether the token must be negated.

Raises

cumin.backends.InvalidQueryError -- on internal parsing error.

_add_hosts(hosts, neg=False)[source]

Add a list of hosts to the query.

Parameters
_parse_token(token)[source]

Concrete implementation of parent abstract method.

Parameters

according to parent cumin.backends.BaseQuery._parse_token().

Raises

cumin.backends.InvalidQueryError -- on internal parsing error.

_get_resource_query(key, value=None, operator='=')[source]

Build a resource query based on the parameters, resolving the special cases for %params and @field.

Parameters
  • key (str) -- the key of the resource.

  • value (str, optional) -- the value to match, if not specified the key itself will be matched.

  • operator (str, optional) -- the comparison operator to use, one of OPERATORS.

Returns

the resource query.

Return type

str

Raises

cumin.backends.InvalidQueryError -- on invalid combinations of parameters.

_get_special_resource_query(category, key, value, operator)[source]

Build a query for Roles and Profiles, resolving the special cases for %params and @field.

Parameters
  • category (str) -- the category of the token, one of category_prefixes keys.

  • key (str) -- the key of the resource to use as a suffix for the Class title matching.

  • value (str, optional) -- the value to match in case %params or @field is specified.

  • operator (str, optional) -- the comparison operator to use if there is a value, one of OPERATORS.

Returns

the resource query.

Return type

str

Raises

cumin.backends.InvalidQueryError -- on invalid combinations of parameters.

_get_query_string(group)[source]

Recursively build and return the PuppetDB query string.

Parameters

group (dict) -- a dictionary with the grouped tokens.

Returns

the query string for the PuppetDB API.

Return type

str

_add_bool(bool_op)[source]

Add a boolean AND or OR query block to the query and validate logic.

Parameters

bool_op (str) -- the boolean operator to add to the query: and, or.

Raises

cumin.backends.InvalidQueryError -- if an invalid boolean operator was found.

_api_call(query)[source]

Execute a query to PuppetDB API and return the parsed JSON.

Parameters

query (str) -- the query parameter to send to the PuppetDB API.

Raises

requests.HTTPError -- if the PuppetDB API call fails.

cumin.backends.puppetdb.GRAMMAR_PREFIX = 'P'

the prefix associate to this grammar, to register this backend into the general grammar. Required by the backend auto-loader in cumin.grammar.get_registered_backends().

Type

str

cumin.backends.puppetdb.query_class

Required by the backend auto-loader in cumin.grammar.get_registered_backends().

alias of cumin.backends.puppetdb.PuppetDBQuery