Using LLMs to Categorize Your Chats

If you're like me, you probably have 2000 chats. Fifteen <> Stripe, Fifteen <> Cloudflare, Fifteen <> McDonalds. I should be adding my colleagues to these groups, setting up agents to watch them, and generating backups for compliance.

Now I don't want to give access to my private chats to the rest of my team. But I also don't want to go through hundreds of chats manually clicking "Assign to Organization" on each one. That's insane.

Here's the solution: if you've connected Fifteen to Claude Desktop (or any other LLM with tool calling capabilities), the AI can do it for you.

The setup

You need:

1. Fifteen installed and running, ideally with your full chat history indexed (so let it run for a few hours).

2. An LLM connected to Fifteen's MCP server (Claude Desktop, Claude Code, or any LLM that supports MCP and tool calling)

3. An organization created in Fifteen (your company)

That's it. Now the AI has access to your Fifteen app and can categorize chats based on your instructions.

Note: While this guide uses Claude Desktop as the example, any LLM with tool calling capabilities can do this. GPT-4 with function calling, Anthropic's API with tools, local models that support tool use - they all work the same way. The MCP protocol is LLM-agnostic. Connect your preferred LLM to Fifteen's MCP server and follow the same workflow.

The basic approach

We'll use Claude Desktop (or your LLM of choice) to make it categorize for us. Depending on the number of chats, its good to work in batches. Prompt quality matters as well, something like:

I'm working on categorizing Telegram chats, assigning them to the AcmeCorp
organization if they are a business/company chat. Usually they have a title 
like "Acme <> Other Client". If the title doesn't have enough information, we 
can look at the participants of the chats as well. If my coworkers at Acme are 
in the chat, its probably an Acme chat too.

I want you to fetch unassigned chats in batches of 50; check which chats could 
be Acme chats, provide me with a list so I can manually verify, and if I 
approve, I want you to assign those chats. 

You can prompt engineer to see what works best.

• What's your organization name?

• How can it identify work chats vs personal chats?

• Are there specific keywords or patterns in the chat names?

Depending on your exact objective, you'll want to slightly adjust your prompt:

Here you see the AI successfully using Fifteen to categorize 7 different chats, performing the assigning as well.

Using Fifteen, you can interactively go through thousands of chats in a matter of minutes (maybe hours depending on the quality of the LLM). Besides categorizing and analyzing messages, you can generate reports, or even have the LLM send messages to chats using Fifteen.