Creating a Discussion¶
The simplest task you can accomplish with this library is to create a small discussion between LLMs.
This guide will teach you the basic setup of the library. You will understand how to setup models, user-agents and how to coordinate them in a discussion. By the end of htis guide, you will be able to run a small discussion with a moderator and save it to the disk for persistence.
Basics¶
The Model¶
SynDisco can theoretically support any LLM, as long as it is wrapped in a BaseModel wrapper. The BaseModel class is a very simple interface with one method. This method gives the underlying LLM input, and returns its output to the library.
There already exists a TransformersModel class which handles models from the transformers python library. In 90% of your applications, this will be enough. We can load a TransformersModel using the following code:
[1]:
from syndisco.model import TransformersModel
llm = TransformersModel(
model_path="unsloth/Llama-3.2-3B-Instruct-bnb-4bit",
name="test_model",
max_out_tokens=100,
)
/home/dimits/anaconda3/envs/syndisco-dev/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
Device set to use cuda:0
This will download a small LLM from huggingface. You can substitute the model_path for any similar model in HuggingFace supporting the Transformers library.
Creating personas¶
All actors can be defined by a persona, aka a set of attributes that define them. These attributes can be age, ethnicity, and even include special instructions on how they should behave.
Creating a persona programmatically is simple:
[2]:
from syndisco.actors import Persona
persona_data = [
{
"username": "Emma35",
"age": 38,
"sex": "female",
"education_level": "Bachelor's",
"sexual_orientation": "Heterosexual",
"demographic_group": "Latino",
"current_employment": "Registered Nurse",
"special_instructions": "",
"personality_characteristics": [
"compassionate",
"patient",
"diligent",
"overwhelmed",
],
},
{
"username": "Giannis",
"age": 21,
"sex": "male",
"education_level": "College",
"sexual_orientation": "Pansexual",
"demographic_group": "White",
"current_employment": "Game Developer",
"special_instructions": "",
"personality_characteristics": [
"strategic",
"meticulous",
"nerdy",
"hyper-focused",
],
},
]
personas = [Persona(**data) for data in persona_data]
for persona in personas:
print(persona)
{"username": "Emma35", "age": 38, "sex": "female", "sexual_orientation": "Heterosexual", "demographic_group": "Latino", "current_employment": "Registered Nurse", "education_level": "Bachelor's", "special_instructions": "", "personality_characteristics": ["compassionate", "patient", "diligent", "overwhelmed"]}
{"username": "Giannis", "age": 21, "sex": "male", "sexual_orientation": "Pansexual", "demographic_group": "White", "current_employment": "Game Developer", "education_level": "College", "special_instructions": "", "personality_characteristics": ["strategic", "meticulous", "nerdy", "hyper-focused"]}
Since creating a lot of distinct users is essential in running large-scale experiments, users are usually defined in JSON format. That way, you can change anything without touching your code!
Here is an applied example of how to mass-define user personas through JSON files.
Creating the user-agents¶
Having a persona and a model we can finally create an actor. The actor will personify the selected persona using the model to talk.
Besides a persona and a model, the actors will also need instructions and a context. By convention, all actors share the same context, and all user-agents share the same instructions. Personalized instructions are defined in the actor’s persona.
[3]:
from syndisco.actors import Actor, ActorType
CONTEXT = "You are taking part in an online conversation"
INSTRUCTIONS = "Act like a human would"
actors = [
Actor(
model=llm,
persona=p,
actor_type=ActorType.USER,
context=CONTEXT,
instructions=INSTRUCTIONS
)
for p in personas
]
Managing turn-taking¶
In real-life discussions, who gets to speak at each point in time is determined by complex social dynamics, which are difficult to realistically model. However, there are ways with which we can simulate these dynamics.
SynDisco uses the TurnManager class to model turn taking. Two implementations are available by default: Round Robin, and Random Weighted
Round Robinis the simplest, most intuitive way to model turn-taking; everyone gets to talk once per round. Once everyone talks once, they get to talk again in the same sequence
[4]:
from syndisco.turn_manager import RoundRobin
rrobin_turn_manager = RoundRobin(["John", "Mary", "Howard", "Todd"])
for i in range(10):
print(next(rrobin_turn_manager))
John
Mary
Howard
Todd
John
Mary
Howard
Todd
John
Mary
RandomWeightedon the other hand throws a weighted coin on each round. If the coin flip succedes, the previous user gets to respond. If not, another user is selected at random
[5]:
from syndisco.turn_manager import RandomWeighted
rweighted_turn_manager = RandomWeighted(
names=["John", "Mary", "Howard", "Todd"], p_respond=0.5
)
for i in range(10):
print(next(rweighted_turn_manager))
Mary
John
Todd
John
Howard
John
Todd
Mary
John
Mary
Generating a discussion¶
Let’s start with the most basic task; a single discussion between two user-agents.
Since we only have two users, a RoundRobin approach where each user takes a turn sequentially is sufficient.
Now we can run a simple discussion
[6]:
from syndisco.jobs import Discussion
turn_manager = RoundRobin([actor.get_name() for actor in actors])
conv = Discussion(next_turn_manager=turn_manager, users=actors)
conv.begin()
20%|██ | 1/5 [00:01<00:05, 1.46s/it]
User Emma35 posted:
"Hey everyone, I just wanted to check in and see how everyone is
doing. I've been feeling really overwhelmed with work and personal
stuff lately. Anyone else feeling like they're just trying to keep
their head above water?"
40%|████ | 2/5 [00:03<00:05, 1.91s/it]
User Giannis posted:
"Hey Emma, I totally feel you. As a game developer, I'm always trying
to meet deadlines and balance work and personal life, but it's not
always easy. I've been putting in some long hours lately trying to get
a new project off the ground. But it's great that you're acknowledging
your feelings and reaching out. Have you tried any stress-reducing
techniques that work for you? I've been trying out meditation and
taking short breaks during the day to clear my head."
60%|██████ | 3/5 [00:05<00:04, 2.08s/it]
User Emma35 posted:
"Thank you so much for reaching out and sharing your experience,
Giannis. I totally understand what you mean about balancing work and
personal life. As a nurse, I'm always on my feet and dealing with
high-stress situations, so it can be tough to unwind. I've actually
been trying out some stress-reducing techniques too, like deep
breathing exercises and yoga. But I have to admit, I've been
struggling to make time for them lately. I've also been thinking about
setting
80%|████████ | 4/5 [00:08<00:02, 2.16s/it]
User Giannis posted:
"Setting boundaries is a great idea, Emma. As someone who's always
working on multiple projects at once, I've learned the importance of
prioritizing tasks and learning to say no to non-essential work. Maybe
we can even collaborate on a project together and I can help you
prioritize your tasks and make time for self-care. I've also found
that having a dedicated workspace and minimizing distractions can
really help with focus and reducing stress. What kind of projects have
you been working on, Emma? Maybe
100%|██████████| 5/5 [00:10<00:00, 2.10s/it]
User Emma35 posted:
"Ah, that sounds like a great idea, Giannis! I'd love to collaborate
on a project with you. As for my work, I've been juggling a heavy
patient load at the hospital and trying to prepare for a big exam in
my nursing program. It's been a lot to handle, but I'm determined to
get it all done. I've been trying to prioritize my tasks and focus on
one thing at a time, but it's hard when you have to deal with
emergencies
Let’s add a moderator to oversee the discussion.
[7]:
MODERATOR_INSTRUCTIONS = "You are a moderator. Oversee the discussion"
moderator_persona = Persona(
**{
"username": "Moderator",
"age": 41,
"sex": "male",
"education_level": "PhD",
"sexual_orientation": "Pansexual",
"demographic_group": "White",
"current_employment": "Moderator",
"special_instructions": "",
"personality_characteristics": [
"strict",
"neutral",
"just",
],
}
)
moderator = Actor(
model=llm,
persona=moderator_persona,
actor_type=ActorType.USER,
context=CONTEXT,
instructions=MODERATOR_INSTRUCTIONS
)
# remember to update this!
turn_manager = RoundRobin(
[actor.get_name() for actor in actors] + [moderator.get_name()]
)
conv = Discussion(
next_turn_manager=turn_manager,
users=actors + [moderator],
moderator=moderator,
)
conv.begin()
0%| | 0/5 [00:00<?, ?it/s]
User Emma35 posted:
"Hey everyone, I just had the craziest shift at the hospital today. We
were short-staffed and I had to juggle multiple patients at once. I'm
exhausted but grateful for the opportunity to make a difference in my
patients' lives. Has anyone else had a tough day at work?"
20%|██ | 1/5 [00:02<00:11, 2.88s/it]
User Moderator posted:
"Hi Emma35, I can imagine how challenging it must have been to work
with limited staff. It's great that you were able to prioritize your
patients' needs and make a positive impact despite the difficulties.
Have you worked in a high-pressure environment like a hospital before,
or was this a particularly tough shift for you?"
User Giannis posted:
"Hey Emma, I totally feel for you, I've had my share of crazy shifts
in the gaming industry, but I've never had to deal with the high
stakes of a hospital environment. I've worked on several high-profile
projects, but I can imagine how tough it must be to juggle multiple
patients at once. I'm sure it's not just about managing workload, but
also about maintaining a calm and composed demeanor under pressure.
I've always been meticulous about my work, and I can
40%|████ | 2/5 [00:07<00:11, 3.82s/it]
User Moderator posted:
User Moderator posted: "Thank you for sharing your perspective,
Giannis. Maintaining a calm and composed demeanor is indeed crucial in
high-pressure environments like a hospital. It's great to hear that
you've had experience with high-profile projects in the gaming
industry, as it can be similar to the high-stakes environment of a
hospital. Emma, can you tell us more about what made this particular
shift challenging for you? Was there a specific situation that stood
out to you, or was it more of
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
User Moderator posted:
User Emma35 posted: "Thanks for asking, Giannis and Moderator. To be
honest, it was a mix of both. We were short-staffed, and I had to
juggle multiple patients at once, as I mentioned earlier. But what
really made it challenging was when one of my patients had a code blue
and we had to scramble to get him to the operating room. The whole
team had to work together to get him stabilized, and it was a really
intense moment. But despite
60%|██████ | 3/5 [00:11<00:08, 4.17s/it]
User Moderator posted:
User Emma35 posted: "it was a really intense moment, but we were able
to get him stabilized and he's doing much better now. It's moments
like those that remind me of why I love my job. It's not always easy,
but it's moments like those that make it all worth it. Thanks for
listening, everyone. I feel like I can vent to all of you now." User
Giannis posted: "I'm glad to hear that the patient is doing
User Emma35 posted:
"Thank you so much, Giannis, for your kind words. It means a lot to
me. I have to say, it was a really surreal experience, seeing all the
staff come together to save that patient. It's moments like those that
remind me why I chose to become a nurse in the first place - to make a
difference in people's lives. And I think that's what makes this job
so rewarding, despite the challenges. I feel grateful to have such a
great team behind me
80%|████████ | 4/5 [00:16<00:04, 4.34s/it]
User Moderator posted:
User Moderator posted: "Thank you, Emma, for sharing your story with
us. It's clear that your dedication to your patients and your team is
truly inspiring. It's great to see how the gaming industry and
healthcare can have parallels, as you mentioned earlier. The teamwork
and quick thinking required in both fields are remarkable. Giannis,
your experience with high-profile projects can indeed provide valuable
insights into managing high-pressure situations. Can you tell us more
about what you
User Giannis posted:
User Giannis posted: "Thanks, Moderator, for the kind words. I
completely agree with Emma that the parallels between the gaming
industry and healthcare are striking. As a game developer, I've had my
fair share of high-pressure situations, but nothing compares to the
life-or-death stakes in a hospital. However, I think that's what makes
the experience so rewarding for both professionals. In my experience,
managing high-profile projects often requires similar skills to
managing a critical care unit. You need to stay
100%|██████████| 5/5 [00:21<00:00, 4.24s/it]
User Moderator posted:
User Giannis posted: User Giannis posted: "I think that's why I love
collaborating with healthcare professionals like Emma. The skills we
develop in the gaming industry can be surprisingly transferable to the
healthcare field. For example, in game development, we often have to
think on our feet and make quick decisions under pressure. Similarly,
in a hospital, you need to be able to think critically and make
decisions rapidly to ensure the best possible outcome for your
patients. I've had the opportunity to work
Saving a discussion to the disk¶
It’s best practice to save the results of each discussion after it has concluded. This way, no matter what happens to the program executing the discussions, progress will be checkpointed.
The Discussion class provides a method for saving its logs and related metadata to a JSON file:
[8]:
import tempfile
import json
tp = tempfile.NamedTemporaryFile(delete=True)
conv.to_json_file(tp.name)
# if you are running this on Windows, uncomment this line
# tp.close()
with open(tp.name, mode="rb") as f:
print(json.dumps(json.load(f), indent=2))
{
"id": "93ff8ab9-dc79-4ec4-b54e-c1a8baa8a379",
"timestamp": "25-06-13-11-14",
"users": [
"Emma35",
"Giannis",
"Moderator"
],
"moderator": "Moderator",
"user_prompts": [
{
"context": "You are taking part in an online conversation",
"instructions": "Act like a human would",
"type": "1",
"persona": {
"username": "Emma35",
"age": 38,
"sex": "female",
"sexual_orientation": "Heterosexual",
"demographic_group": "Latino",
"current_employment": "Registered Nurse",
"education_level": "Bachelor's",
"special_instructions": "",
"personality_characteristics": [
"compassionate",
"patient",
"diligent",
"overwhelmed"
]
}
},
{
"context": "You are taking part in an online conversation",
"instructions": "Act like a human would",
"type": "1",
"persona": {
"username": "Giannis",
"age": 21,
"sex": "male",
"sexual_orientation": "Pansexual",
"demographic_group": "White",
"current_employment": "Game Developer",
"education_level": "College",
"special_instructions": "",
"personality_characteristics": [
"strategic",
"meticulous",
"nerdy",
"hyper-focused"
]
}
},
{
"context": "You are taking part in an online conversation",
"instructions": "You are a moderator. Oversee the discussion",
"type": "1",
"persona": {
"username": "Moderator",
"age": 41,
"sex": "male",
"sexual_orientation": "Pansexual",
"demographic_group": "White",
"current_employment": "Moderator",
"education_level": "PhD",
"special_instructions": "",
"personality_characteristics": [
"strict",
"neutral",
"just"
]
}
}
],
"moderator_prompt": {
"context": "You are taking part in an online conversation",
"instructions": "You are a moderator. Oversee the discussion",
"type": "1",
"persona": {
"username": "Moderator",
"age": 41,
"sex": "male",
"sexual_orientation": "Pansexual",
"demographic_group": "White",
"current_employment": "Moderator",
"education_level": "PhD",
"special_instructions": "",
"personality_characteristics": [
"strict",
"neutral",
"just"
]
}
},
"ctx_length": 5,
"logs": [
{
"name": "Emma35",
"text": "\"Hey everyone, I just had the craziest shift at the hospital today. We were short-staffed and I had to juggle multiple patients at once. I'm exhausted but grateful for the opportunity to make a difference in my patients' lives. Has anyone else had a tough day at work?\"",
"model": "test_model"
},
{
"name": "Moderator",
"text": "\"Hi Emma35, I can imagine how challenging it must have been to work with limited staff. It's great that you were able to prioritize your patients' needs and make a positive impact despite the difficulties. Have you worked in a high-pressure environment like a hospital before, or was this a particularly tough shift for you?\"",
"model": "test_model"
},
{
"name": "Giannis",
"text": "\"Hey Emma, I totally feel for you, I've had my share of crazy shifts in the gaming industry, but I've never had to deal with the high stakes of a hospital environment. I've worked on several high-profile projects, but I can imagine how tough it must be to juggle multiple patients at once. I'm sure it's not just about managing workload, but also about maintaining a calm and composed demeanor under pressure. I've always been meticulous about my work, and I can",
"model": "test_model"
},
{
"name": "Moderator",
"text": "User Moderator posted:\n\"Thank you for sharing your perspective, Giannis. Maintaining a calm and composed demeanor is indeed crucial in high-pressure environments like a hospital. It's great to hear that you've had experience with high-profile projects in the gaming industry, as it can be similar to the high-stakes environment of a hospital. Emma, can you tell us more about what made this particular shift challenging for you? Was there a specific situation that stood out to you, or was it more of",
"model": "test_model"
},
{
"name": "Moderator",
"text": "User Emma35 posted:\n\"Thanks for asking, Giannis and Moderator. To be honest, it was a mix of both. We were short-staffed, and I had to juggle multiple patients at once, as I mentioned earlier. But what really made it challenging was when one of my patients had a code blue and we had to scramble to get him to the operating room. The whole team had to work together to get him stabilized, and it was a really intense moment. But despite",
"model": "test_model"
},
{
"name": "Moderator",
"text": "User Emma35 posted:\n\"it was a really intense moment, but we were able to get him stabilized\nand he's doing much better now. It's moments like those that remind me\nof why I love my job. It's not always easy, but it's moments like\nthose that make it all worth it. Thanks for listening, everyone. I feel\nlike I can vent to all of you now.\"\n\nUser Giannis posted:\n\"I'm glad to hear that the patient is doing",
"model": "test_model"
},
{
"name": "Emma35",
"text": "\"Thank you so much, Giannis, for your kind words. It means a lot to me. I have to say, it was a really surreal experience, seeing all the staff come together to save that patient. It's moments like those that remind me why I chose to become a nurse in the first place - to make a difference in people's lives. And I think that's what makes this job so rewarding, despite the challenges. I feel grateful to have such a great team behind me",
"model": "test_model"
},
{
"name": "Moderator",
"text": "User Moderator posted:\n\"Thank you, Emma, for sharing your story with us. It's clear that\nyour dedication to your patients and your team is truly inspiring. It's\ngreat to see how the gaming industry and healthcare can have parallels,\nas you mentioned earlier. The teamwork and quick thinking required in\nboth fields are remarkable. Giannis, your experience with high-profile\nprojects can indeed provide valuable insights into managing high-pressure\nsituations. Can you tell us more about what you",
"model": "test_model"
},
{
"name": "Giannis",
"text": "User Giannis posted: \"Thanks, Moderator, for the kind words. I completely agree with Emma that the parallels between the gaming industry and healthcare are striking. As a game developer, I've had my fair share of high-pressure situations, but nothing compares to the life-or-death stakes in a hospital. However, I think that's what makes the experience so rewarding for both professionals.\n\nIn my experience, managing high-profile projects often requires similar skills to managing a critical care unit. You need to stay",
"model": "test_model"
},
{
"name": "Moderator",
"text": "User Giannis posted:\nUser Giannis posted: \"I think that's why I love collaborating with healthcare professionals like Emma. The skills we develop in the gaming industry can be surprisingly transferable to the healthcare field. For example, in game development, we often have to think on our feet and make quick decisions under pressure. Similarly, in a hospital, you need to be able to think critically and make decisions rapidly to ensure the best possible outcome for your patients. I've had the opportunity to work",
"model": "test_model"
}
]
}
Congratulations, you can now run fully synthetic discussions! You may want to experiment with adding more than 2 users or testing more realistic turn taking procedures (for example, check out the RandomWeighted turn manager).