altimeter.core.graph package

Subpackages

• altimeter.core.graph.field package
  • Submodules
  • altimeter.core.graph.field.base module
  • altimeter.core.graph.field.dict_field module
  • altimeter.core.graph.field.exceptions module
  • altimeter.core.graph.field.list_field module
  • altimeter.core.graph.field.resource_link_field module
  • altimeter.core.graph.field.scalar_field module
  • altimeter.core.graph.field.tags_field module
  • altimeter.core.graph.field.util module
  • Module contents

Submodules

altimeter.core.graph.exceptions module

Exceptions related ton Graphing

exception DuplicateResourceIdsFoundException

Bases: altimeter.core.exceptions.AltimeterException

Duplicate resource ids were found

exception GraphSetOrphanedReferencesException

Bases: altimeter.core.exceptions.AltimeterException

A GraphSet contained orphaned references, for instance a ResourceLink referring to an id not present in the GraphSet.

exception LinkParseException

Bases: altimeter.core.exceptions.AltimeterException

Error parsing Links from JSON.

exception ListFieldSourceKeyNotFoundException

Bases: altimeter.core.exceptions.AltimeterException

The source_key of a ListField was not found

exception ListFieldValueNotAListException

Bases: altimeter.core.exceptions.AltimeterException

A ListField does not contain a list

exception SchemaParseException

Bases: altimeter.core.exceptions.AltimeterException

Schema.parse error

exception UnmergableDuplicateResourceIdsFoundException

Bases: altimeter.core.exceptions.AltimeterException

Duplicate unmergable resource ids were found

exception UnmergableGraphSetsException

Bases: altimeter.core.exceptions.AltimeterException

GraphSets are unable to be merged.

altimeter.core.graph.graph_set module

A GraphSet represents the contents of a Graph.

class GraphSet(**data)

Bases: altimeter.core.base_model.BaseImmutableModel

A GraphSet represents the contents of a Graph. It contains a list of Resource objects, a graph name and version and a few metadata fields describing the scan. Generally GraphSets are created by running GraphSpec.scan.

Parameters

• name – graph name

• version – graph version

• start_time – epoch scan start time

• end_time – epoch scan end time

• resources – Resource objects

• errors – Errors encountered during scan

end_time: int

errors: List[str]

classmethod from_graph_sets(graph_sets)

Create a new GraphSet from a list of GraphSets

Return type

GraphSet

classmethod from_json_file(path)

Return type

GraphSet

name: str

resources: Tuple[altimeter.core.resource.resource.Resource, …]

start_time: int

version: str

class ValidatedGraphSet(**data)

Bases: altimeter.core.graph.graph_set.GraphSet

A GraphSet which has been validated - duplicate resources have been merged and required resource links have been verified

classmethod dedupe_and_validate_resources(resources)

Return type

Tuple[Resource, …]

end_time: int

errors: List[str]

classmethod from_graph_set(graph_set)

Create a ValidatedGraphSet from a GraphSet

Return type

ValidatedGraphSet

name: str

resources: Tuple[altimeter.core.resource.resource.Resource, …]

start_time: int

to_neptune_lpg(scan_id)

Return type

Dict

to_rdf()

Generate an rdflib.Graph from this ValidatedGraphSet.

Return type

Graph

Returns

rdf.Graph object representing this ValidatedGraphSet.

version: str

dedupe_resources(resources)

Resolve any duplicate resource ids. In general duplicate resource ids can have their Resource objects merged if they are of the same type and all fields are identical or additive only across the resources or if one of the Resources allows a special merge via its ResourceSpec class’ allow_clobber attribute.

Return type

Tuple[Resource, …]

validate_resources(resources)

Validate that all inter-resource relationships in a tuple of Resources are valid and that there are no duplicate resource ids.

Raises

• GraphSetOrphanedReferencesException if references to Resources are found in – the graph which do not exist. For example, if an EC2 instance has a VPC and references it as a ResourceLinkField if that VPC is not found as a Resource in the graph this exception is raised.

• DuplicateResourceIdsFoundException if duplicate resource ids are found –

Return type

None

altimeter.core.graph.graph_spec module

A GraphSpec contains a specification to scan and create a graph.

class GraphSpec(name, version, resource_spec_classes, scan_accessor)

Bases: object

A GraphSpec contains a specification to scan and create a graph. It contains a set of ResourceSpec classes defining what to scan and a scan_accessor object defining how to scan.

Parameters

• name (str) – graph name

• version (str) – graph version

• resource_spec_classes (Tuple[Type[ResourceSpec], …]) – tuple of ResourceSpec classes

• scan_accessor (Any) – object which is passed to ResourceSpec.scan for each ResourceSpec in resource_spec_classes. It should provide methods which ResourceSpec.scan can use to access whatever API the ResourceSpec needs to access.

scan()

Perform a scan on all of the resource classes in this GraphSpec and return a list of Resource objects.

Return type

List[Resource]

Returns

List of Resource objects

altimeter.core.graph.links module

A Link represents the predicate-object portion of a triple.

class BaseLink(**data)

Bases: altimeter.core.base_model.BaseImmutableModel, abc.ABC

A link represents the predicate-object portion of a triple. BaseLink is an abstract base class for Link subclasses.

to_lpg(parent, vertices, edges, prefix)

Graph this link on a BNode in a Graph using a given Namespace to create the full predicate.

Parameters

• parent (Dict) – a dictionary og the parent

• vertices (List[Dict]) – a list of dictionaries of the vertices for a labeled property graph

• edges (List[Dict]) – a list of dictionaries of the edges for a labeled property graph

• prefix (str) – a prefix to add to the attribute name

Return type

None

to_rdf(subj, namespace, graph, node_cache)

Graph this link on a BNode in a Graph using a given Namespace to create the full predicate.

Parameters

• subj (BNode) – subject portion of triple - graph this link’s pred, obj against it.

• namespace (Namespace) – RDF namespace to use for this triple’s predicate

• graph (Graph) – RDF graph

• node_cache (NodeCache) – NodeCache to use to find cached nodes.

Return type

None

class LinkCollection(**data)

Bases: altimeter.core.base_model.BaseImmutableModel

classmethod from_links(links)

Return type

LinkCollection

get_links()

Return type

Tuple[Union[SimpleLink, MultiLink, TagLink, ResourceLink, TransientResourceLink], …]

multi_links: Optional[Tuple[altimeter.core.graph.links.MultiLink, …]]

resource_links: Optional[Tuple[altimeter.core.graph.links.ResourceLink, …]]

simple_links: Optional[Tuple[altimeter.core.graph.links.SimpleLink, …]]

tag_links: Optional[Tuple[altimeter.core.graph.links.TagLink, …]]

to_lpg(vertex, vertices, edges, prefix='')

Graph this LinkCollection as a labelled property graph

Return type

None

to_rdf(subj, namespace, graph, node_cache)

Graph this LinkCollection on an RDF graph

Return type

None

transient_resource_links: Optional[Tuple[altimeter.core.graph.links.TransientResourceLink, …]]

class MultiLink(**data)

Bases: altimeter.core.graph.links.BaseLink

Represents a named set of sublinks. For example an ‘EBSVolumeAttachemnt’ MultiLink could exist which specifies sublinks Volume, AttachTime

obj: altimeter.core.graph.links.LinkCollection

pred: str

to_lpg(parent, vertices, edges, prefix='')

Convert this link to the appropriate vertices, edges, and properties

:param : parent: the parent dictionary vertex :type vertices: List[Dict] :param vertices: the list of all vertex dictionaries :type edges: List[Dict] :param edges: the list of all edge dictionaries :type prefix: str :param prefix: A string to prefix the property name with

Return type

None

to_rdf(subj, namespace, graph, node_cache)

Graph this link on a BNode in a Graph using a given Namespace to create the full predicate.

Parameters

• subj (BNode) – subject portion of triple - graph this link’s pred, obj against it.

• namespace (Namespace) – RDF namespace to use for this triple’s predicate

• graph (Graph) – RDF graph

• node_cache (NodeCache) – NodeCache to use to find cached nodes.

Return type

None

class ResourceLink(**data)

Bases: altimeter.core.graph.links.BaseLink

Represents a link to another resource which must exist in the graph.

obj: str

pred: str

to_lpg(parent, vertices, edges, prefix='')

Convert this link to the appropriate vertices, edges, and properties

:param : parent: the parent dictionary vertex :type vertices: List[Dict] :param vertices: the list of all vertex dictionaries :type edges: List[Dict] :param edges: the list of all edge dictionaries :type prefix: str :param prefix: string to prefix the property name with

Return type

None

to_rdf(subj, namespace, graph, node_cache)

Graph this link on a BNode in a Graph using a given Namespace to create the full predicate.

Parameters

• subj (BNode) – subject portion of triple - graph this link’s pred, obj against it.

• namespace (Namespace) – RDF namespace to use for this triple’s predicate

• graph (Graph) – RDF graph

• node_cache (NodeCache) – NodeCache to use to find cached nodes.

Return type

None

class SimpleLink(**data)

Bases: altimeter.core.graph.links.BaseLink

A SimpleLink represents a scalar value. In RDF terms a SimpleLink creates a Literal in the graph.

obj: Any

classmethod obj_is_scalar(val)

Return type

Any

pred: str

to_lpg(parent, vertices, edges, prefix='')

Convert this link to the appropriate vertices, edges, and properties

:param : parent: the parent dictionary vertex :param : param vertices: the list of all vertex dictionaries :param : param edges: the list of all edge dictionaries :param : param prefix: the prefix assigned to the key :param : type parent: Dict

Return type

None

to_rdf(subj, namespace, graph, node_cache)

Graph this link on a BNode in a Graph using a given Namespace to create the full predicate.

Parameters

• subj (BNode) – subject portion of triple - graph this link’s pred, obj against it.

• namespace (Namespace) – RDF namespace to use for this triple’s predicate

• graph (Graph) – RDF graph

• node_cache (NodeCache) – NodeCache to use to find cached nodes.

Return type

None

class TagLink(**data)

Bases: altimeter.core.graph.links.BaseLink

Represents a AWS-style Tag attached to a node.

obj: str

pred: str

to_lpg(parent, vertices, edges, prefix='')

Convert this link to the appropriate vertices, edges, and properties

:param : parent:git the parent dictionary vertex :type vertices: List[Dict] :param vertices: the list of all vertex dictionaries :type edges: List[Dict] :param edges: the list of all edge dictionaries :type prefix: str :param prefix: string to prefix the property name with

Return type

None

to_rdf(subj, namespace, graph, node_cache)

Graph this link on a BNode in a Graph using a given Namespace to create the full predicate.

Parameters

• subj (BNode) – subject portion of triple - graph this link’s pred, obj against it.

• namespace (Namespace) – RDF namespace to use for this triple’s predicate

• graph (Graph) – RDF graph

• node_cache (NodeCache) – NodeCache to use to find cached nodes.

Return type

None

class TransientResourceLink(**data)

Bases: altimeter.core.graph.links.BaseLink

Represents a link to another resource which may or may not exist in the graph.

obj: str

pred: str

to_lpg(parent, vertices, edges, prefix='')

Convert this link to the appropriate vertices, edges, and properties

:param : parent: the parent dictionary vertex :type vertices: List[Dict] :param vertices: the list of all vertex dictionaries :type edges: List[Dict] :param edges: the list of all edge dictionaries :type prefix: str :param prefix: string to prefix the property name with

Return type

None

to_rdf(subj, namespace, graph, node_cache)

Graph this link on a BNode in a Graph using a given Namespace to create the full predicate.

Parameters

• subj (BNode) – subject portion of triple - graph this link’s pred, obj against it.

• namespace (Namespace) – RDF namespace to use for this triple’s predicate

• graph (Graph) – RDF graph

• node_cache (NodeCache) – NodeCache to use to find cached nodes.

Return type

None

altimeter.core.graph.node_cache module

A NodeCache is a simple cache for graph nodes.

class NodeCache

Bases: dict

A NodeCache is a simple cache for graph nodes. Unlike a standard dict it does not allow overwriting entries.

altimeter.core.graph.schema module

A Schema consists of a list of Fields which define how to parse an arbitrary dictionary into a list of Links.

class Schema(*fields)

Bases: object

A Schema consists of a list of Fields which define how to parse an arbitrary dictionary into a altimeter.core.graph.links.LinkCollection.

Parameters

fields (Field) – fields for this Schema.

parse(data, context)

Parse this schema into a list of Links

Parameters

• data (Dict[str, Any]) – raw data to parse

• context (Dict[str, Any]) – contains auxiliary information which can be passed through the parse process.

Return type

LinkCollection

Returns

LinkCollection

Module contents