altimeter.core package

Subpackages

• altimeter.core.artifact_io package
  • Submodules
  • altimeter.core.artifact_io.exceptions module
  • altimeter.core.artifact_io.reader module
  • altimeter.core.artifact_io.writer module
  • Module contents
• altimeter.core.graph package
  • Subpackages
    • altimeter.core.graph.field package
      • Submodules
      • altimeter.core.graph.field.base module
      • altimeter.core.graph.field.dict_field module
      • altimeter.core.graph.field.exceptions module
      • altimeter.core.graph.field.list_field module
      • altimeter.core.graph.field.resource_link_field module
      • altimeter.core.graph.field.scalar_field module
      • altimeter.core.graph.field.tags_field module
      • altimeter.core.graph.field.util module
      • Module contents
  • Submodules
  • altimeter.core.graph.exceptions module
  • altimeter.core.graph.graph_set module
  • altimeter.core.graph.graph_spec module
  • altimeter.core.graph.links module
  • altimeter.core.graph.node_cache module
  • altimeter.core.graph.schema module
  • Module contents
• altimeter.core.neptune package
  • Submodules
  • altimeter.core.neptune.client module
  • altimeter.core.neptune.exceptions module
  • altimeter.core.neptune.results module
  • altimeter.core.neptune.sparql module
  • Module contents
• altimeter.core.resource package
  • Submodules
  • altimeter.core.resource.exceptions module
  • altimeter.core.resource.resource module
  • altimeter.core.resource.resource_spec module
  • Module contents

Submodules

altimeter.core.base_model module

Base pydantic altimeter model classes

class BaseImmutableModel(**data)

Bases: pydantic.main.BaseModel

Base immutable pydantic altimeter model

class Config

Bases: object

Pydantic config

allow_mutation = False

arbitrary_types_allowed = True

extra = 'forbid'

class BaseMutableModel(**data)

Bases: pydantic.main.BaseModel

Base mutable pydantic altimeter model

class Config

Bases: object

Pydantic config

arbitrary_types_allowed = True

extra = 'forbid'

altimeter.core.config module

Configuration classes

class ConcurrencyConfig(**data)

Bases: altimeter.core.base_model.BaseImmutableModel

Concurrency configuration class

max_account_scan_threads: int

max_svc_scan_threads: int

class Config(**data)

Bases: altimeter.core.base_model.BaseImmutableModel

Top level configuration class

accessor: altimeter.aws.auth.accessor.Accessor

artifact_path: str

concurrency: altimeter.core.config.ConcurrencyConfig

classmethod from_file(filepath)

Load a Config from a file

Return type

Config

classmethod from_path(path)

Load a Config from an s3 uri or a file

Return type

Config

classmethod from_s3(s3_uri)

Load a Config from an s3 object

Return type

Config

graph_name: str

neptune: Optional[altimeter.core.config.NeptuneConfig]

pruner_max_age_min: int

scan: altimeter.core.config.ScanConfig

write_master_json: bool

exception InvalidConfigException

Bases: Exception

Indicates an invalid configuration

class NeptuneConfig(**data)

Bases: altimeter.core.base_model.BaseImmutableModel

Neptune configuration class

auth_mode: Optional[str]

graph_load_sns_topic_arn: Optional[str]

host: str

iam_credentials_provider_type: Optional[str]

iam_role_arn: Optional[str]

port: int

region: str

ssl: Optional[bool]

use_lpg: Optional[bool]

class ScanConfig(**data)

Bases: altimeter.core.base_model.BaseImmutableModel

Scan configuration class

accounts: Tuple[str, …]

preferred_account_scan_regions: Tuple[str, …]

regions: Tuple[str, …]

scan_sub_accounts: bool

altimeter.core.exceptions module

Base Exceptions.

exception AltimeterException

Bases: Exception

An error occurred.

altimeter.core.json_encoder module

Function for encoding JSON with datetimes.

json_encoder(obj)

json encoder function supporting datetime serialization.

Parameters

obj (Any) – object to encode to JSON

Return type

Any

Returns

json encoded data

altimeter.core.log module

Provides a class Logger with methods for logging.

class BaseLogEvent

Bases: object

Base class for LogEvent classes

class BaseLogger(log_tid=True, pretty_output=False)

Bases: object

Provides contextmanager ‘bind’ which can be use to bind keys to the logger using ‘with’ syntax, they will be removed from the logger in subsequent calls. In general use Logger, not BaseLogger directly.

bind(**bindings)

Context manager to bind a set of k/vs to the logger. The k/vs will be removed when the with block exits.

Return type

Generator[None, None, None]

debug(event, **kwargs)

Create DEBUG level log entry.

Parameters

• event (EventName) – EventName object for this event

• kwargs (Any) – log event k/vs

Return type

None

error(event, **kwargs)

Create ERROR level log entry.

Parameters

• event (EventName) – EventName object for this event

• kwargs (Any) – log event k/vs

Return type

None

info(event, **kwargs)

Create INFO level log entry.

Parameters

• event (EventName) – EventName object for this event

• kwargs (Any) – log event k/vs

Return type

None

warn(event, **kwargs)

Create WARN level log entry.

Parameters

• event (EventName) – EventName object for this event

• kwargs (Any) – log event k/vs

Return type

None

warning(event, **kwargs)

Create WARN level log entry.

Parameters

• event (EventName) – EventName object for this event

• kwargs (Any) – log event k/vs

Return type

None

class EventName(name)

Bases: object

Dataclass for log event names.

Parameters

name (str) – name of this event

name: str

class LogEventMeta(name: str, bases: Tuple[Type, …], namespace: Dict[str, Any])

Bases: type

Metaclass for LogEvents. This allows EventNames to specified in subclasses of BaseLogEvent as empty typed variables e.g.

AuthToAccountStart: EventName

Rather than requiring

AuthToAccountStart: EventName(“AuthToAccountStart”)

class Logger(log_tid=True, pretty_output=False)

Bases: altimeter.core.log.BaseLogger

Singleton logger class

class Singleton

Bases: type

Singleton Metaclass

altimeter.core.log_events module

class LogEvent(AuthToAccountStart=EventName(name='AuthToAccountStart'), AuthToAccountEnd=EventName(name='AuthToAccountEnd'), AuthToAccountFailure=EventName(name='AuthToAccountFailure'), GraphLoadedSNSNotificationStart=EventName(name='GraphLoadedSNSNotificationStart'), GraphLoadedSNSNotificationEnd=EventName(name='GraphLoadedSNSNotificationEnd'), MetadataGraphUpdateStart=EventName(name='MetadataGraphUpdateStart'), MetadataGraphUpdateEnd=EventName(name='MetadataGraphUpdateEnd'), NeptuneLoadStart=EventName(name='NeptuneLoadStart'), NeptuneLoadEnd=EventName(name='NeptuneLoadEnd'), NeptuneLoadPolling=EventName(name='NeptuneLoadPolling'), NeptuneLoadError=EventName(name='NeptuneLoadError'), NeptuneRDFWriteStart=EventName(name='NeptuneRDFWriteStart'), NeptuneRDFWriteEnd=EventName(name='NeptuneRDFWriteEnd'), NeptuneGremlinWriteStart=EventName(name='NeptuneGremlinWriteStart'), NeptuneGremlinWriteEnd=EventName(name='NeptuneGremlinWriteEnd'), NeptunePeriodicWrite=EventName(name='NeptunePeriodicWrite'), PruneNeptuneGraphStart=EventName(name='PruneNeptuneGraphStart'), PruneNeptuneGraphEnd=EventName(name='PruneNeptuneGraphEnd'), PruneNeptuneGraphError=EventName(name='PruneNeptuneGraphError'), PruneNeptuneGraphSkip=EventName(name='PruneNeptuneGraphSkip'), PruneOrphanedNeptuneGraphStart=EventName(name='PruneOrphanedNeptuneGraphStart'), PruneOrphanedNeptuneGraphEnd=EventName(name='PruneOrphanedNeptuneGraphEnd'), PruneNeptuneGraphsStart=EventName(name='PruneNeptuneGraphsStart'), PruneNeptuneGraphsEnd=EventName(name='PruneNeptuneGraphsEnd'), PruneNeptuneGraphsError=EventName(name='PruneNeptuneGraphsError'), ReadFromFSStart=EventName(name='ReadFromFSStart'), ReadFromFSEnd=EventName(name='ReadFromFSEnd'), ReadFromS3Start=EventName(name='ReadFromS3Start'), ReadFromS3End=EventName(name='ReadFromS3End'), ScanResourceTypeStart=EventName(name='ScanResourceTypeStart'), ScanResourceTypeEnd=EventName(name='ScanResourceTypeEnd'), WriteToFSStart=EventName(name='WriteToFSStart'), WriteToFSEnd=EventName(name='WriteToFSEnd'), WriteToS3Start=EventName(name='WriteToS3Start'), WriteToS3End=EventName(name='WriteToS3End'))

Bases: altimeter.core.log.BaseLogEvent

Contains EventNames for logging.

AuthToAccountEnd: altimeter.core.log.EventName = EventName(name='AuthToAccountEnd')

AuthToAccountFailure: altimeter.core.log.EventName = EventName(name='AuthToAccountFailure')

AuthToAccountStart: altimeter.core.log.EventName = EventName(name='AuthToAccountStart')

GraphLoadedSNSNotificationEnd: altimeter.core.log.EventName = EventName(name='GraphLoadedSNSNotificationEnd')

GraphLoadedSNSNotificationStart: altimeter.core.log.EventName = EventName(name='GraphLoadedSNSNotificationStart')

MetadataGraphUpdateEnd: altimeter.core.log.EventName = EventName(name='MetadataGraphUpdateEnd')

MetadataGraphUpdateStart: altimeter.core.log.EventName = EventName(name='MetadataGraphUpdateStart')

NeptuneGremlinWriteEnd: altimeter.core.log.EventName = EventName(name='NeptuneGremlinWriteEnd')

NeptuneGremlinWriteStart: altimeter.core.log.EventName = EventName(name='NeptuneGremlinWriteStart')

NeptuneLoadEnd: altimeter.core.log.EventName = EventName(name='NeptuneLoadEnd')

NeptuneLoadError: altimeter.core.log.EventName = EventName(name='NeptuneLoadError')

NeptuneLoadPolling: altimeter.core.log.EventName = EventName(name='NeptuneLoadPolling')

NeptuneLoadStart: altimeter.core.log.EventName = EventName(name='NeptuneLoadStart')

NeptunePeriodicWrite: altimeter.core.log.EventName = EventName(name='NeptunePeriodicWrite')

NeptuneRDFWriteEnd: altimeter.core.log.EventName = EventName(name='NeptuneRDFWriteEnd')

NeptuneRDFWriteStart: altimeter.core.log.EventName = EventName(name='NeptuneRDFWriteStart')

PruneNeptuneGraphEnd: altimeter.core.log.EventName = EventName(name='PruneNeptuneGraphEnd')

PruneNeptuneGraphError: altimeter.core.log.EventName = EventName(name='PruneNeptuneGraphError')

PruneNeptuneGraphSkip: altimeter.core.log.EventName = EventName(name='PruneNeptuneGraphSkip')

PruneNeptuneGraphStart: altimeter.core.log.EventName = EventName(name='PruneNeptuneGraphStart')

PruneNeptuneGraphsEnd: altimeter.core.log.EventName = EventName(name='PruneNeptuneGraphsEnd')

PruneNeptuneGraphsError: altimeter.core.log.EventName = EventName(name='PruneNeptuneGraphsError')

PruneNeptuneGraphsStart: altimeter.core.log.EventName = EventName(name='PruneNeptuneGraphsStart')

PruneOrphanedNeptuneGraphEnd: altimeter.core.log.EventName = EventName(name='PruneOrphanedNeptuneGraphEnd')

PruneOrphanedNeptuneGraphStart: altimeter.core.log.EventName = EventName(name='PruneOrphanedNeptuneGraphStart')

ReadFromFSEnd: altimeter.core.log.EventName = EventName(name='ReadFromFSEnd')

ReadFromFSStart: altimeter.core.log.EventName = EventName(name='ReadFromFSStart')

ReadFromS3End: altimeter.core.log.EventName = EventName(name='ReadFromS3End')

ReadFromS3Start: altimeter.core.log.EventName = EventName(name='ReadFromS3Start')

ScanResourceTypeEnd: altimeter.core.log.EventName = EventName(name='ScanResourceTypeEnd')

ScanResourceTypeStart: altimeter.core.log.EventName = EventName(name='ScanResourceTypeStart')

WriteToFSEnd: altimeter.core.log.EventName = EventName(name='WriteToFSEnd')

WriteToFSStart: altimeter.core.log.EventName = EventName(name='WriteToFSStart')

WriteToS3End: altimeter.core.log.EventName = EventName(name='WriteToS3End')

WriteToS3Start: altimeter.core.log.EventName = EventName(name='WriteToS3Start')

Module contents