ai.analysis package

Subpackages

Submodules

ai.analysis.cost_analyzer_decorator module

ai.analysis.cost_analyzer_decorator.cost_analyzer(warning_limit: ~ai.analysis.money.money.Money = $0.001, error_limit: ~ai.analysis.money.money.Money = $0.01, dataset_usage_analyzer: ~ai.analysis.dataset_usage_analyzer.DatasetUsageAnalyzer = None) → Callable[[type[Assistant]], type[Assistant]][source]

Factory for a class decorator that instruments Assistant subclasses with cost logging.

The decorated Assistant will log a warning if a single run’s cost exceeds warning_limit, and an error if it exceeds error_limit. It will also record each run via a DatasetUsageAnalyzer.

Parameters:

warning_limit (Money) – Cost threshold above which a warning is logged. Defaults to 0.001 USD.
error_limit (Money) – Cost threshold above which an error is logged. Defaults to 0.01 USD.
dataset_usage_analyzer (DatasetUsageAnalyzer, optional) – Analyzer to collect usage data. If not provided here, must be injected when instantiating the Assistant.

Returns:

A decorator that produces a cost-instrumented Assistant class.

Return type:

Callable[[Type[Assistant]], Type[Assistant]]

ai.analysis.cost_analyzer_decorator_test module

class ai.analysis.cost_analyzer_decorator_test.MockedAssistant(**data)[source]

Bases: Assistant

A minimal Assistant implementation that returns a mocked ChatCompletion.

async run_openai(prompt: str) → ChatCompletion[source]

Simulate an OpenAI API call returning preset usage counts.

Parameters:: prompt (str) – The input prompt (ignored in this mock).
Returns:: A mock completion with .model and .usage set.
Return type:: ChatCompletion

async ai.analysis.cost_analyzer_decorator_test.test_cost_analyzer()[source]

Verify that cost_analyzer logs and records an AssistantRun correctly.

Decorates MockedAssistant with cost_analyzer.
Executes run_openai to trigger cost logging.
Asserts that DatasetUsageAnalyzer.add_run was called once with an AssistantRun containing the expected assistant name and token counts.

Raises:: AssertionError – If any of the expected calls or attributes are missing.

ai.analysis.dataset_usage_analyzer module

class ai.analysis.dataset_usage_analyzer.FormattedAnalysis(assistant: str, model: str, prompt_cost: str, completions_cost: str, total_cost: str, share: str)[source]

Bases: object

Holds a formatted cost analysis entry for presentation.

assistant

Name of the assistant.

Type:: str

model

Model identifier used.

Type:: str

prompt_cost

Formatted cost of prompt tokens.

Type:: str

completions_cost

Formatted cost of completion tokens.

Type:: str

total_cost

Formatted total cost.

Type:: str

share

Percentage share of total cost.

Type:: str

assistant: str

model: str

prompt_cost: str

completions_cost: str

total_cost: str

share: str

__init__(assistant: str, model: str, prompt_cost: str, completions_cost: str, total_cost: str, share: str) → None

ai.analysis.dataset_usage_analyzer.transform_field_names(cls) → List[str][source]

Generate human-readable column names from a dataclass.

Replaces underscores with spaces and capitalizes each field name.

Parameters:: cls (Type) – Dataclass whose fields will be transformed.
Returns:: List of formatted field names.
Return type:: List[str]

class ai.analysis.dataset_usage_analyzer.DatasetUsageAnalyzer(currency: Currency)[source]

Bases: AbstractAnalyzer

Aggregates and summarizes cost analyses across multiple assistant runs.

_currency

Target currency for formatting costs.

Type:: Currency

_logger

Logger for warnings and debug messages.

Type:: logging.Logger

_runs_by_assistant

Mapping of assistants to their runs.

Type:: dict[Assistant, list[AssistantRun]]

__init__(currency: Currency)[source]

Initialize the analyzer with a currency.

Parameters:: currency (Currency) – Currency in which to report costs.

get_cost_analysis() → AnalysisResult[source]

Compute the combined AnalysisResult for all recorded runs.

Returns:: Aggregated cost analysis across all assistants.
Return type:: AnalysisResult

add_run(assistant_run: AssistantRun) → None[source]

Record a new AssistantRun under its assistant.

Parameters:: assistant_run (AssistantRun) – The run to add.

generate_cost_summary() → List[FormattedAnalysis][source]

Generate a summary table of costs per assistant, plus a total row.

Returns:: Formatted cost entries for each assistant and the grand total.
Return type:: List[FormattedAnalysis]