immortal-wm 2fc9753673 fix: corrected latency accuracy (#189) ...

* fix latency calc

* feat: add timing context management to AIModel for latency tracking

- Introduced a new TimingContextRaceConditionError to handle race conditions in multi-threaded environments.
- Implemented a timing_context method in AIModel to track request timing and prevent concurrent usage.
- Updated various model classes (LargeLanguageModel, ModerationModel, RerankModel, Speech2TextModel, TextEmbeddingModel, TTSModel) to utilize the new timing context for latency calculations.
- Added tests to validate timing context behavior under concurrent and sequential usage scenarios.

* feat: implement ModelFactory for dynamic model instance creation

- Added ModelFactory class to generate stateless model instances based on provider configurations and model classes.
- Updated PluginRegistration to utilize ModelFactory for managing model instances, enhancing code organization and maintainability.

* feat: add unit tests for model registry and mock model provider

- Introduced a new test file for validating model registry functionality.
- Implemented mock classes for model provider and LLM to facilitate testing.
- Added tests to ensure correct model instance retrieval from the registry.

* apply ruff

---------

Co-authored-by: Yeuoly <admin@srmxy.cn>

2025-09-16 16:15:20 +08:00

..

openai_compatible

fix: incorrectly uses new_tool_call.function.name while getting existing tool calls (#104)

2025-04-25 14:00:23 +08:00

__init__.py

fix: incorrectly uses new_tool_call.function.name while getting existing tool calls (#104)

2025-04-25 14:00:23 +08:00

test_ai_model_timing_context.py

fix: corrected latency accuracy (#189)

2025-09-16 16:15:20 +08:00

test_llm_timing_context.py

fix: corrected latency accuracy (#189)

2025-09-16 16:15:20 +08:00