/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/__init__.py:30: UserWarning: Importing Anthropic from langchain root module is no longer supported. Please use langchain_community.llms.Anthropic instead.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/__init__.py:30: UserWarning: Importing HuggingFaceHub from langchain root module is no longer supported. Please use langchain_community.llms.HuggingFaceHub instead.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/__init__.py:30: UserWarning: Importing LLMChain from langchain root module is no longer supported. Please use langchain.chains.LLMChain instead.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/__init__.py:30: UserWarning: Importing OpenAI from langchain root module is no longer supported. Please use langchain_community.llms.OpenAI instead.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/__init__.py:30: UserWarning: Importing PromptTemplate from langchain root module is no longer supported. Please use langchain_core.prompts.PromptTemplate instead.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/__init__.py:30: UserWarning: Importing Anthropic from langchain root module is no longer supported. Please use langchain_community.llms.Anthropic instead.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/__init__.py:30: UserWarning: Importing HuggingFaceHub from langchain root module is no longer supported. Please use langchain_community.llms.HuggingFaceHub instead.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/__init__.py:30: UserWarning: Importing LLMChain from langchain root module is no longer supported. Please use langchain.chains.LLMChain instead.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/chat_models/__init__.py:33: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.chat_models import ChatOpenAI`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/langchain/chat_models/__init__.py:33: LangChainDeprecationWarning: Importing chat models from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.chat_models import ChatOpenAI`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
/Volumes/workplace/fmcore/fmcore/src/fmcore/prompt_tuner/evaluator/base_evaluator.py:13: GenericBeforeBaseModelWarning: Classes should inherit from `BaseModel` before generic classes (e.g. `typing.Generic[T]`) for pydantic generics to work properly.
  class BaseEvaluator(Generic[I, O], MutableTyped, Registry, ABC):
/Volumes/workplace/fmcore/fmcore/src/fmcore/prompt_tuner/evaluator/llm_as_a_judge_boolean/llm_as_a_judge_boolean_evaluator.py:17: GenericBeforeBaseModelWarning: Classes should inherit from `BaseModel` before generic classes (e.g. `typing.Generic[T]`) for pydantic generics to work properly.
  class LLMAsJudgeBooleanEvaluator(BaseEvaluator[Dict, bool]):
/Volumes/workplace/fmcore/fmcore/src/fmcore/prompt_tuner/evaluator/classification/classification_evaluator.py:9: GenericBeforeBaseModelWarning: Classes should inherit from `BaseModel` before generic classes (e.g. `typing.Generic[T]`) for pydantic generics to work properly.
  class ClassificationEvaluator(BaseEvaluator[Dict, bool]):
Running Prompt Tuner with Classification
Running Prompt Tuner with Boolean as judge
{'task_type': 'TEXT_GENERATION', 'dataset_config': {'inputs': {'TRAIN': {'name': 'train', 'path': 's3://dc-test-rajsiba/auto_prompt_tuner/datasets/ads_copilot/', 'storage': S3, 'format': PARQUET, 'contents': None, 'file_glob': None, 'data_schema': None}}, 'output': {'name': 'output', 'path': 's3://dc-test-rajsiba/auto_prompt_tuner/outputs/ads_copilot', 'storage': S3, 'format': PARQUET, 'contents': None, 'file_glob': None, 'data_schema': None}}, 'prompt_config': {'prompt': 'Identify the right plan based on converstaion history and the latest query\n\n', 'input_fields': [{'name': 'conversation_history', 'description': 'This field represents conversation history with the advertiser', 'field_type': 'string'}, {'name': 'latest_query', 'description': 'This field represents latest query from the advertiser', 'field_type': 'string'}], 'output_fields': [{'name': 'label', 'description': "This field represents the selected plan, can be one of 'BID_BUDGET_RECOMMENDATION', 'INAPPROPRIATE_OFF_TOPIC_PLAN',  'EDUCATIONAL' or 'OPEN_DOMAIN_DIALOG_PLAN'", 'field_type': 'string'}]}, 'framework': 'DSPY', 'optimizer_config': {'evaluator_config': {'evaluator_type': 'CLASSIFICATION', 'evaluator_params': {'prediction_field': 'input.label', 'ground_truth_field': 'output.label', 'groundTruth': ''}}, 'teacher_config': {'provider_type': 'BEDROCK', 'model_id': 'anthropic.claude-3-5-sonnet-20240620-v1:0', 'model_params': {'temperature': 1.0, 'max_tokens': 4096, 'top_p': 0.9}, 'provider_params_list': [{'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::863518436859:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::615299746603:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::615299746603:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::710271919393:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::710271919393:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::872515274170:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::872515274170:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::339712890513:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::339712890513:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::760397367430:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::760397367430:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::268097248272:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::268097248272:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::932671304170:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::932671304170:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::957971773207:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::957971773207:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}]}, 'student_config': {'provider_type': 'BEDROCK', 'model_id': 'anthropic.claude-3-haiku-20240307-v1:0', 'model_params': {'temperature': 0.1, 'max_tokens': 4096, 'top_p': 0.9}, 'provider_params_list': [{'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::863518436859:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::615299746603:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::615299746603:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::710271919393:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::710271919393:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::872515274170:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::872515274170:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::339712890513:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::339712890513:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::760397367430:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::760397367430:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::268097248272:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::268097248272:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::932671304170:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::932671304170:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::957971773207:role/ModelFactoryBedrockAccessRole', 'region': 'us-east-1'}, {'retries': {'max_retries': 50, 'backoff_factor': 1.0, 'jitter': 1.0, 'retryable_exceptions': ['InvalidSignatureException', 'ThrottlingException', 'ModelTimeoutException', 'ServiceUnavailableException', 'ModelNotReadyException', 'ServiceQuotaExceededException', 'ModelErrorException', 'EndpointConnectionError']}, 'rate_limit': {'max_rate': 1000, 'time_period': 60}, 'role_arn': 'arn:aws:iam::957971773207:role/ModelFactoryBedrockAccessRole', 'region': 'us-west-2'}]}, 'optimizer_type': 'MIPRO_V2', 'optimizer_params': {'optimizer_metric': 'ACCURACY', 'num_candidates': 10, 'max_errors': 10, 'minibatch': False, 'auto': 'light'}}}

Read 0 row(s):   0%|          | 0/2 [00:00<?, ?file/s]
Read 0 row(s):   0%|          | 0/2 [00:00<?, ?file/s]
Read 0 row(s):  50%|█████     | 1/2 [00:03<00:03,  3.79s/file]
Read 0 row(s):  50%|█████     | 1/2 [00:03<00:03,  3.79s/file]
Read 326 row(s):  50%|█████     | 1/2 [00:03<00:03,  3.79s/file]
Read 326 row(s):  50%|█████     | 1/2 [00:03<00:03,  3.79s/file]
Read 326 row(s): 100%|██████████| 2/2 [00:04<00:00,  1.96s/file]
Read 326 row(s): 100%|██████████| 2/2 [00:04<00:00,  1.96s/file]
Read 652 row(s): 100%|██████████| 2/2 [00:04<00:00,  1.96s/file]
Read 652 row(s): 100%|██████████| 2/2 [00:04<00:00,  1.96s/file]
Read 652 row(s): 100%|██████████| 2/2 [00:04<00:00,  1.96s/file]
Read 652 row(s): 100%|██████████| 2/2 [00:04<00:00,  1.96s/file]
Read 652 row(s): 100%|██████████| 2/2 [00:04<00:00,  2.10s/file]
Bootstrapping set 1/5
2025/04/28 17:09:49 INFO dspy.teleprompt.mipro_optimizer_v2: 
Bootstrapping set 2/5
RUNNING WITH THE FOLLOWING LIGHT AUTO RUN SETTINGS:
Bootstrapping set 3/5
num_trials: 7
minibatch: True
num_candidates: 5
valset size: 100

2025/04/28 17:09:49 INFO dspy.teleprompt.mipro_optimizer_v2: 
==> STEP 1: BOOTSTRAP FEWSHOT EXAMPLES <==
2025/04/28 17:09:49 INFO dspy.teleprompt.mipro_optimizer_v2: These will be used as few-shot example candidates for our program and for creating instructions.

2025/04/28 17:09:49 INFO dspy.teleprompt.mipro_optimizer_v2: Bootstrapping N=5 sets of demonstrations...

  0%|          | 0/391 [00:00<?, ?it/s]
  0%|          | 1/391 [00:02<18:36,  2.86s/it]/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/aiolimiter/leakybucket.py:102: RuntimeWarning: This AsyncLimiter instance is being re-used across loops. Please create a new limiter per event loop as re-use can lead to undefined behaviour.
  _warn_reuse()

  1%|          | 2/391 [00:04<15:26,  2.38s/it]
  1%|          | 3/391 [00:06<14:31,  2.25s/it]
  1%|          | 4/391 [00:09<14:06,  2.19s/it]
  1%|          | 4/391 [00:09<14:39,  2.27s/it]

  0%|          | 0/391 [00:00<?, ?it/s]Bootstrapped 4 full traces after 4 examples for up to 1 rounds, amounting to 4 attempts.
Bootstrapping set 4/5

  0%|          | 1/391 [00:02<18:44,  2.88s/it]
  1%|          | 2/391 [00:04<14:42,  2.27s/it]/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/aiolimiter/leakybucket.py:102: RuntimeWarning: This AsyncLimiter instance is being re-used across loops. Please create a new limiter per event loop as re-use can lead to undefined behaviour.
  _warn_reuse()

  1%|          | 3/391 [00:06<12:26,  1.92s/it]
  1%|          | 3/391 [00:06<13:26,  2.08s/it]

  0%|          | 0/391 [00:00<?, ?it/s]Bootstrapped 3 full traces after 3 examples for up to 1 rounds, amounting to 3 attempts.
Bootstrapping set 5/5

  0%|          | 1/391 [00:02<13:13,  2.03s/it]
  0%|          | 1/391 [00:02<13:13,  2.03s/it]
2025/04/28 17:10:06 INFO dspy.teleprompt.mipro_optimizer_v2: 
Bootstrapped 1 full traces after 1 examples for up to 1 rounds, amounting to 1 attempts.
==> STEP 2: PROPOSE INSTRUCTION CANDIDATES <==
2025/04/28 17:10:06 INFO dspy.teleprompt.mipro_optimizer_v2: We will use the few-shot examples from the previous step, a generated dataset summary, a summary of the program code, and a randomly selected prompting tip to propose instructions.
/Users/rajsiba/miniconda3/envs/fmcore/lib/python3.11/site-packages/aiolimiter/leakybucket.py:102: RuntimeWarning: This AsyncLimiter instance is being re-used across loops. Please create a new limiter per event loop as re-use can lead to undefined behaviour.
  _warn_reuse()
