{% include 'pages/navbar.html' %}
← LLM campaigns

New LLM {% if mode == 'llm_gen' %}generation{% else %}evaluation{% endif %} campaign

Preset
{% if mode == 'llm_eval' %} {% include 'include/annotation_span_categories.html' %}
{% endif %}
Prompting strategy
The prompting strategy controls how the prompt will be used to obtain the output. Use parse_raw when structured output cannot be enforced for the model. Use sentence_split to prompt the model separately for each sentence. Usually, you can keep the default strategy.
{% if mode == 'llm_eval' %}
Annotation JSON
Configure which fields should be included in the JSON schema for annotations. Important: Your prompt template must clearly describe the purpose and format of each enabled field.
{% endif %}
Prompt template
Prompt for the model. See the instructions for configuring the prompt at factgenie wiki.
System message
Role of the model (optional).
{% if mode == 'llm_gen' %}
Start with
The model will need to start the generated text with this string (optional).
{% endif %}
API provider
Select one of the available providers (or add your own class to factgenie/prompting/api.py). Do not forget to add the relevant API keys in the app settings or in the factgenie/config/config.yml file.
Local API URL (base)
For local services, this is the API URL, e.g. http://my-server.com:11434. The parameter is ignored for external APIs.
Model
The identifier of the model you are querying.
Model arguments
API arguments to be passed to the model. Internally, we use ast.literal_eval() for determining the argument type. Use floating point format (e.g. 0.0) for the numbers that should be interpreted as floats.
Extra arguments
Any other parameters used by the metric.
{% if mode == 'llm_eval' %}
Whether the model should annotate at word or character level. Word-level enforces word boundaries during annotation matching.
Whether the model should be allowed to produce overlapping annotations.
{% endif %}
{% include 'include/data_selection.html' %}
{% include 'include/example_annotation_modal.html' %}