ai.analysis package¶

Subpackages¶

Submodules¶

ai.analysis.cost_analyzer_decorator module¶

ai.analysis.cost_analyzer_decorator.cost_analyzer(warning_limit: ~ai.analysis.money.money.Money = $0.001, error_limit: ~ai.analysis.money.money.Money = $0.01, dataset_usage_analyzer: ~ai.analysis.dataset_usage_analyzer.DatasetUsageAnalyzer = None) → Callable[[type[Assistant]], type[Assistant]][source]¶

Factory for a class decorator that instruments Assistant subclasses with cost logging.

The decorated Assistant will log a warning if a single run’s cost exceeds warning_limit, and an error if it exceeds error_limit. It will also record each run via a DatasetUsageAnalyzer.

Parameters:

warning_limit (Money) – Cost threshold above which a warning is logged. Defaults to 0.001 USD.
error_limit (Money) – Cost threshold above which an error is logged. Defaults to 0.01 USD.
dataset_usage_analyzer (DatasetUsageAnalyzer, optional) – Analyzer to collect usage data. If not provided here, must be injected when instantiating the Assistant.

Returns:

A decorator that produces a cost-instrumented Assistant class.

Return type:

Callable[[Type[Assistant]], Type[Assistant]]

ai.analysis.cost_analyzer_decorator_test module¶

class ai.analysis.cost_analyzer_decorator_test.MockedAssistant(*, name: AssistantName, ai_model: AssistantChatModel, base_prompt: str, ai_model_provider: ModelProvider, temperature: float = 1.0)[source]¶

Bases: Assistant

A minimal Assistant implementation that returns a mocked ChatCompletion.

async run_openai(prompt: str) → ChatCompletion[source]¶

Simulate an OpenAI API call returning preset usage counts.

Parameters:: prompt (str) – The input prompt (ignored in this mock).
Returns:: A mock completion with .model and .usage set.
Return type:: ChatCompletion

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}¶: A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'ai_model': FieldInfo(annotation=AssistantChatModel, required=True), 'ai_model_provider': FieldInfo(annotation=ModelProvider, required=True), 'base_prompt': FieldInfo(annotation=str, required=True), 'name': FieldInfo(annotation=AssistantName, required=True), 'temperature': FieldInfo(annotation=float, required=False, default=1.0)}¶

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

async ai.analysis.cost_analyzer_decorator_test.test_cost_analyzer()[source]¶

Verify that cost_analyzer logs and records an AssistantRun correctly.

Decorates MockedAssistant with cost_analyzer.
Executes run_openai to trigger cost logging.
Asserts that DatasetUsageAnalyzer.add_run was called once with an AssistantRun containing the expected assistant name and token counts.

Raises:: AssertionError – If any of the expected calls or attributes are missing.

ai.analysis.dataset_usage_analyzer module¶

class ai.analysis.dataset_usage_analyzer.FormattedAnalysis(assistant: str, model: str, prompt_cost: str, completions_cost: str, total_cost: str, share: str)[source]¶

Bases: object

Holds a formatted cost analysis entry for presentation.

assistant¶

Name of the assistant.

Type:: str

model¶

Model identifier used.

Type:: str

prompt_cost¶

Formatted cost of prompt tokens.

Type:: str

completions_cost¶

Formatted cost of completion tokens.

Type:: str

total_cost¶

Formatted total cost.

Type:: str

share¶

Percentage share of total cost.

Type:: str

assistant: str¶

model: str¶

prompt_cost: str¶

completions_cost: str¶

total_cost: str¶

share: str¶

__init__(assistant: str, model: str, prompt_cost: str, completions_cost: str, total_cost: str, share: str) → None¶

ai.analysis.dataset_usage_analyzer.transform_field_names(cls) → List[str][source]¶

Generate human-readable column names from a dataclass.

Replaces underscores with spaces and capitalizes each field name.

Parameters:: cls (Type) – Dataclass whose fields will be transformed.
Returns:: List of formatted field names.
Return type:: List[str]

class ai.analysis.dataset_usage_analyzer.DatasetUsageAnalyzer(currency: Currency)[source]¶

Bases: AbstractAnalyzer

Aggregates and summarizes cost analyses across multiple assistant runs.

_currency¶

Target currency for formatting costs.

Type:: Currency

_logger¶

Logger for warnings and debug messages.

Type:: logging.Logger

_runs_by_assistant¶

Mapping of assistants to their runs.

Type:: dict[Assistant, list[AssistantRun]]

__init__(currency: Currency)[source]¶

Initialize the analyzer with a currency.

Parameters:: currency (Currency) – Currency in which to report costs.

get_cost_analysis() → AnalysisResult[source]¶

Compute the combined AnalysisResult for all recorded runs.

Returns:: Aggregated cost analysis across all assistants.
Return type:: AnalysisResult

add_run(assistant_run: AssistantRun) → None[source]¶

Record a new AssistantRun under its assistant.

Parameters:: assistant_run (AssistantRun) – The run to add.

generate_cost_summary() → List[FormattedAnalysis][source]¶

Generate a summary table of costs per assistant, plus a total row.

Returns:: Formatted cost entries for each assistant and the grand total.
Return type:: List[FormattedAnalysis]