Quick updates of discussions associated with streamlit and chainlit

To date, streamlit was introduced in 2019 as the Python of Python that enhances the process of AI Prototyping AI requirements in need of the user's meeting (UIS). Data technicians can concentrate on the back parts (e.g. Chainlit, and Python Frameworks. While the brainstorm of the pain. of END-TOD-END DOME CHATBOT, and give effective recommendations.

Note: All statistics from the following sections are made by the author of this article.

Demos of the last Chatbot

Local setting

For convenience, we will create demographic programs to be easily screened in the area of ​​high language systems (llMs) available on Ollama, quality downloads, management, and open communication in the local user machine.

Of course, Demos will be converted later for producing, eg, by installing the latest llms given to Openai or Google, and by sending Chatbot to the hywscaler used as AWs, AWIs, or GCP. All the initiatives below are tested at Cos Sequence 15.6.1, and should be the same as Linux and Windows.

Go here to download and install Ollama. Check that the installation has been successfully conducting this command in the furnace:

ollama --version

We will use the Gemma-Gemma Gemma model with 2B parameters, which can be downloaded by this command:

ollama pull gemma:2b

The model file size around 1.7 GB, so download it may take a few minutes depending on your Internet connection. Make sure the model is downloaded using this command:

ollama list

This will show all the models downloaded with Ollama so far.

Next, we will set up the project guide using uvQuick and functional project management tool. Follow the instructions here to install it uvand ensure the installation uses this command:

uv --version

Start a project guide called chatbot-demos In your area in your area as follows:

uv init --bare chatbot-demos

Without specifying --bare Option, uv Did you create normal art during the start time, such as main.py, README.mdand the PIN file version of the Python Version, but this is not required for our demes. A small process only creates a pyproject.toml File.

In chatbot-demos Project Directory, Create a requirements.txt File by leaning for the following:

chainlit==2.7.2
ollama==0.5.3
streamlit==1.49.1

Now create visualization of the Python 3.12 within the project guide, activate the environment, and enter the reliance:

uv venv --python=3.12 
source .venv/bin/activate
uv add -r requirements.txt

Check that leaning is included:

uv pip list

We will use a class called LLMClient The background of backdender can be interrupted in the operative of the UI-Centric, which is different to classify the structures such as chainlit. For example, LLMClient Being able to care for tasks such as choices between the LLM suppliers, using the LLM calls, communicating with external representation details (RAG), including discussion history to review the conversation. Here is the use of an example of LLMClientstored in a file called llm_client.py:

import logging
import time
from datetime import datetime, timezone
from typing import List, Dict, Optional, Callable, Any, Generator
import os
import ollama

LOG_FILE = os.path.join(os.path.dirname(__file__), "conversation_history.log")

logger = logging.getLogger("conversation_logger")
logger.setLevel(logging.INFO)

if not logger.handlers:
    fh = logging.FileHandler(LOG_FILE, encoding="utf-8")
    fmt = logging.Formatter("%(asctime)s - %(message)s")
    fh.setFormatter(fmt)
    logger.addHandler(fh)

class LLMClient:
    def __init__(
        self,
        provider: str = "ollama",
        model: str = "gemma:2b",
        temperature: float = 0.2,
        retriever: Optional[Callable[[str], List[str]]] = None,
        feedback_handler: Optional[Callable[[Dict[str, Any]], None]] = None,
        logger: Optional[Callable[[Dict[str, Any]], None]] = None
    ):
        self.provider = provider
        self.model = model
        self.temperature = temperature
        self.retriever = retriever
        self.feedback_handler = feedback_handler
        self.logger = logger or self.default_logger

    def default_logger(self, data: Dict[str, Any]):
        logging.info(f"[LLMClient] {data}")

    def _format_messages(self, messages: List[Dict[str, str]]) -> str:
        return "n".join(f"{m['role'].capitalize()}: {m['content']}" for m in messages)

    def _stream_provider(self, prompt: str, temperature: float) -> Generator[str, None, None]:
        if self.provider == "ollama":
            for chunk in ollama.generate(
                model=self.model,
                prompt=prompt,
                stream=True,
                options={"temperature": temperature}
            ):
                yield chunk.get("response", "")
        else:
            raise ValueError(f"Streaming not implemented for provider: {self.provider}")

    def stream_generate(
        self,
        messages: List[Dict[str, str]],
        on_token: Callable[[str], None],
        temperature: Optional[float] = None
    ) -> Dict[str, Any]:
        start_time = time.time()

        if self.retriever:
            query = messages[-1]["content"]
            docs = self.retriever(query)
            if docs:
                context_str = "n".join(docs)
                messages = [{"role": "system", "content": f"Use this context:n{context_str}"}] + messages

        prompt = self._format_messages(messages)
        assembled_text = ""
        temp_to_use = temperature if temperature is not None else self.temperature

        try:
            for token in self._stream_provider(prompt, temp_to_use):
                assembled_text += token
                on_token(token)
        except Exception as e:
            assembled_text = f"Error: {e}"

        latency = time.time() - start_time

        result = {
            "text": assembled_text,
            "timestamp": datetime.now(timezone.utc),
            "latency": latency,
            "provider": self.provider,
            "model": self.model,
            "temperature": temp_to_use,
            "messages": messages
        }

        self.logger({
            "event": "llm_stream_call",
            "provider": self.provider,
            "model": self.model,
            "temperature": temp_to_use,
            "latency": latency,
            "prompt": prompt,
            "response": assembled_text
        })

        return result

    def record_feedback(self, feedback: Dict[str, Any]):
        if self.feedback_handler:
            self.feedback_handler(feedback)
        else:
            self.logger({"event": "feedback", **feedback})
    
    def log_interaction(self, role: str, content: str):
        logger.info(f"{role.upper()}: {content}")

The basic demo demo

Create a file called st_app_basic.py In the project guide and paste the following code:

import streamlit as st
from llm_client import LLMClient

MAX_HISTORY = 5
llm_client = LLMClient(provider="ollama", model="gemma:2b")

st.set_page_config(page_title="Streamlit Basic Chatbot", layout="centered")
st.title("Streamlit Basic Chatbot")

if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat history
for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        st.markdown(msg["content"])

# User input
if prompt := st.chat_input("Type your message..."):
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.session_state.messages = st.session_state.messages[-MAX_HISTORY:]
    llm_client.log_interaction("user", prompt)

    with st.chat_message("assistant"):
        response_container = st.empty()
        state = {"full_response": ""}

        def on_token(token):
            state["full_response"] += token
            response_container.markdown(state["full_response"])

        result = llm_client.stream_generate(st.session_state.messages, on_token)
        st.session_state.messages.append({"role": "assistant", "content": result["text"]})
        llm_client.log_interaction("assistant", result["text"])

Run the app to localhost:8501 Say:

streamlit run st_app_basic.py

If the app can automatically be opened in your default browser, sleeping in manual URL (you should see an empty discussion.

What formula turn Celsius to Fahrenheit?

Figure 1 shows the result:

Now, ask the following question:

Can you use that formula in Python?

As the implementation of our demo keeps the history of the interview last minutes, Chatbot will be able to associate “that form” and that in the past, as shown in Figure 2 below:

Feel free to play around several several. To close the app, extract Control + Celebrate c in the bulk.

The chainlit's basic chainlit

Create a file called cl_app_basic.py In the project guide and paste the following code:

import chainlit as cl
from llm_client import LLMClient

MAX_HISTORY = 5
llm_client = LLMClient(provider="ollama", model="gemma:2b")

@cl.on_chat_start
async def start():
    await cl.Message(content="Welcome! Ask me anything.").send()
    cl.user_session.set("messages", [])

@cl.on_message
async def main(message: cl.Message):
    messages = cl.user_session.get("messages")
    messages.append({"role": "user", "content": message.content})
    messages[:] = messages[-MAX_HISTORY:]
    llm_client.log_interaction("user", message.content)

    state = {"full_response": ""}
    
    def on_token(token):
        state["full_response"] += token

    result = llm_client.stream_generate(messages, on_token)
    messages.append({"role": "assistant", "content": result["text"]})
    llm_client.log_interaction("assistant", result["text"])

    await cl.Message(content=result["text"]).send()

Run the app to localhost:8000 (Notice a separate port) as follows:

chainlit run cl_app_basic.py

Because of the comparison, we will make the same same as before. The results are displayed in Mathematics 3 and 4 below:

As before, after playing around more than several more, shut off the app by doing Control + Celebrate c in the bulk.

Demo Advanced Advanced

Now we will add a basic demo to persisting sideback on the left side of the slider widget to convert the LLM heating. Customizing the operating system structure and adding the global widgets can be easily on streatitlit but may be good in the chainlit – interested students can give you the opportunity to get first difficulties.

Here have an expanded streamlit app, stored in a file called st_app_advanced.py:

import streamlit as st
from llm_client import LLMClient
import json

MAX_HISTORY = 5
llm_client = LLMClient(provider="ollama", model="gemma:2b")

st.set_page_config(page_title="Streamlit Advanced Chatbot", layout="wide")
st.title("Streamlit Advanced Chatbot")

# Sidebar controls
st.sidebar.header("Model Settings")
temperature = st.sidebar.slider("Temperature", 0.0, 1.0, 0.2, 0.1)  # min, max, default, increment size
st.sidebar.download_button(
    "Download Chat History",
    data=json.dumps(st.session_state.get("messages", []), indent=2),
    file_name="chat_history.json",
    mime="application/json"
)

if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat history
for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        st.markdown(msg["content"])

# User input
if prompt := st.chat_input("Type your message..."):
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.session_state.messages = st.session_state.messages[-MAX_HISTORY:]
    llm_client.log_interaction("user", prompt)

    with st.chat_message("assistant"):
        response_container = st.empty()
        state = {"full_response": ""}

        def on_token(token):
            state["full_response"] += token
            response_container.markdown(state["full_response"])

        result = llm_client.stream_generate(
            st.session_state.messages,
            on_token,
            temperature=temperature
        )
        llm_client.log_interaction("assistant", result["text"])
        st.session_state.messages.append({"role": "assistant", "content": result["text"]})

        # Feedback buttons
        col1, col2 = st.columns(2)
        if col1.button("Helpful"):
            llm_client.record_feedback({"rating": "up", "comment": "User liked the answer"})
        if col2.button("Not Helpful"):
            llm_client.record_feedback({"rating": "down", "comment": "User disliked the answer"})

Figure 5 shows the screen for example:

Demo Advanced Chainlit Demo

Next, we will increase the basic chainlit with practical per-message actions and multimodal installation (text and pictures on our pages). Chatese – traditional chainlit frameworks make it easy to use these types of factors more than in distributing. Also, interest students are encouraged to find differences by trying to repeat the work using streaty.

Here is an extended chainlit app, stored in a file called cl_app_advanced.py:

import os
import json
from typing import List, Dict
import chainlit as cl
from llm_client import LLMClient

MAX_HISTORY = 5
DEFAULT_TEMPERATURE = 0.2
SESSIONS_DIR = os.path.join(os.path.dirname(__file__), "sessions")
os.makedirs(SESSIONS_DIR, exist_ok=True)

llm_client = LLMClient(provider="ollama", model="gemma:2b", temperature=DEFAULT_TEMPERATURE)

def _session_file(session_name: str) -> str:
    safe = "".join(c for c in session_name if c.isalnum() or c in ("-", "_"))
    return os.path.join(SESSIONS_DIR, f"{safe or 'default'}.json")

def _save_session(session_name: str, messages: List[Dict]):
    with open(_session_file(session_name), "w", encoding="utf-8") as f:
        json.dump(messages, f, ensure_ascii=False, indent=2)

def _load_session(session_name: str) -> List[Dict]:
    path = _session_file(session_name)
    if os.path.exists(path):
        with open(path, "r", encoding="utf-8") as f:
            return json.load(f)
    return []

@cl.on_chat_start
async def start():
    cl.user_session.set("messages", [])
    cl.user_session.set("session_name", "default")
    cl.user_session.set("last_assistant_idx", None)

    await cl.Message(
        content=(
            "Welcome! Ask me anything."
        ),
        actions=[
            cl.Action(name="set_session_name", label="Set session name", payload={"turn": None}),
            cl.Action(name="save_session", label="Save session", payload={"turn": "save"}),
            cl.Action(name="load_session", label="Load session", payload={"turn": "load"}),
        ],
    ).send()

@cl.action_callback("set_session_name")
async def set_session_name(action):
    await cl.Message(content="Please type: /name YOUR_SESSION_NAME").send()

@cl.action_callback("save_session")
async def save_session(action):
    session_name = cl.user_session.get("session_name")
    _save_session(session_name, cl.user_session.get("messages", []))
    await cl.Message(content=f"Session saved as '{session_name}'.").send()

@cl.action_callback("load_session")
async def load_session(action):
    session_name = cl.user_session.get("session_name")
    loaded = _load_session(session_name)
    cl.user_session.set("messages", loaded[-MAX_HISTORY:])
    await cl.Message(content=f"Loaded session '{session_name}' with {len(loaded)} turn(s).").send()

@cl.on_message
async def main(message: cl.Message):
    if message.content.strip().startswith("/name "):
        new_name = message.content.strip()[6:].strip() or "default"
        cl.user_session.set("session_name", new_name)
        await cl.Message(content=f"Session name set to '{new_name}'.").send()
        return

    messages = cl.user_session.get("messages")

    user_text = message.content or ""
    if message.elements:
        for element in message.elements:
            if getattr(element, "mime", "").startswith("image/"):
                user_text += f" [Image: {element.name}]"

    messages.append({"role": "user", "content": user_text})
    messages[:] = messages[-MAX_HISTORY:]
    llm_client.log_interaction("user", user_text)

    state = {"full_response": ""}
    msg = cl.Message(content="")

    def on_token(token: str):
        state["full_response"] += token
        cl.run_sync(msg.stream_token(token))

    result = llm_client.stream_generate(messages, on_token, temperature=DEFAULT_TEMPERATURE)
    messages.append({"role": "assistant", "content": result["text"]})
    llm_client.log_interaction("assistant", result["text"])

    msg.content = state["full_response"]
    await msg.send()
    
    turn_idx = len(messages) - 1
    cl.user_session.set("last_assistant_idx", turn_idx)

    await cl.Message(
        content="Was this helpful?",
        actions=[
            cl.Action(name="thumbs_up", label="Yes", payload={"turn": turn_idx}),
            cl.Action(name="thumbs_down", label="No", payload={"turn": turn_idx}),
            cl.Action(name="save_session", label="Save session", payload={"turn": "save"}),
        ],
    ).send()

@cl.action_callback("thumbs_up")
async def thumbs_up(action):
    turn = action.payload.get("turn")
    llm_client.record_feedback({"rating": "up", "turn": turn})
    await cl.Message(content="Thanks for your feedback!").send()

@cl.action_callback("thumbs_down")
async def thumbs_down(action):
    turn = action.payload.get("turn")
    llm_client.record_feedback({"rating": "down", "turn": turn})
    await cl.Message(content="Thanks for your feedback.").send()

Figure 6 shows the screening screen:

Practical directory

As the previous class shows, it is possible to be a simple prototype that is easy for simple Chatbot applications with streattit and chainlit. The basic demons we have used, there were fewer Buildings: Calls to Ollama and the login in the dialog is issued using LLMClient Category, State size was determined using a joint change called MAX_HISTORYAnd history was separated from the PicaindExty chat format. Since the demons are developed indicating, however, each frame is different, including some opportunities and deals in accordance with the process of using and related recommendations.

While the broadcasting of the standard Data-Centric, active web apps, the chainlit focuses on the creation and management of AI. Therefore, the chainlit can make it more thought to use when Chatbot is in the prototype; As examples of the above code show, the chainlit cares for several Boilplate data (eg, Chat features built-in-built-in interior of indigenous types, messages, and code / code. But if Chatbot is added to the great AI product, the direction of Stream Stream is best for a larger app (eg, global data recognition, custom widgets).

In addition, things to convert conversations to AI programs may require a good deal of user (Ux). async including await Keywords, confirming that the app can manage the same functions without blocking UI. The framework cares for the lowest information by managing websoccase and customizations, so whenever the event is renewable (SIB Controlls Revictively reset the user's contact;

Finally, across the ends that come with the focus on Chat, the chainlit was released after a few years after mature, so it is currently raised and a small engineer. Eg Although the chainlit appears quickly and the posts are referred to, developers can meet periodic changes between versions, comprehensive comprehensive documents, and a limited directory of integrating certain submission areas. The product groups are still wish to produce Chatbot-Centruc applications for future buildings for the temporary organizational benefits.