Practical guide · 27 October 2026

Set up a Slack assistant.Check the evidence yourself.

Install Open Agent Harness and evaluate one cited answer in a small pilot channel.

This guide follows the repository documentation. It has not been run end to end; the example questions are evaluation suggestions, not demonstrated results.

Watch the introduction video and explore the project →

1. Choose a small pilot

Start with a non-sensitive Slack channel and a few teammates. Open Agent Harness is an early-stage, MIT-licensed assistant built on Hermes. Slack and an OpenAI-compatible model endpoint are required; Telegram, Jira, GitHub and Notion are optional.

Allowed assistant users share access to all collected history. There are no per-user source permissions. Relevant context goes to your configured model provider and enabled hosted connectors. Read the security notes before connecting data. Hosting and model usage may cost money.

You need Git, Docker with Compose, permission to install a Slack app, and a model endpoint that supports tool calling.

2. Configure the project

For guided setup, install the setup skill for your coding agent:

npx skills add MUST-Company-AX-Booster/Open-Agent-Harness --skill open-agent-setup

Then ask: “Set up Open Agent for my team.” Review the proposed configuration. Without Node.js, ask your agent to follow the setup skill instructions directly.

For manual setup:

git clone --recurse-submodules https://github.com/MUST-Company-AX-Booster/Open-Agent-Harness.git
cd Open-Agent-Harness
cp .env.example .env

Edit .env locally. Configure LLM_API_KEY, LLM_BASE_URL and AGENT_MODEL. Set three separate database passwords: POSTGRES_PASSWORD, COLLECTOR_DB_PASSWORD and HERMES_DB_PASSWORD. You can generate each with openssl rand -hex 24.

Follow the README’s Slack app setup to generate the manifest, install the app, and obtain SLACK_BOT_TOKEN and SLACK_APP_TOKEN. Compose requires non-empty Slack values even when generating the manifest; the setup skill allows temporary pending values for that step. Replace them before starting services.

Set SLACK_ALLOWED_USERS to the intended Slack member IDs and configure TEAM_TIMEZONE if UTC is unsuitable. SLACK_ALLOW_ALL_USERS=true overrides the allowlist, so keep the pilot narrowly scoped. Keep tokens, .env and data/ private.

Not using Jira? The default Compose stack still includes its service. Follow the README’s Jira removal instructions before startup. Leave other optional integrations disabled for the first evaluation.

3. Start and inspect the services

Create the bind-mount paths before starting Docker:

mkdir -p data/hermes/skills/team-history
touch data/hermes/config.yaml
docker compose up -d --build
docker compose ps

If the Hermes submodule is empty, run git submodule update --init before building. Inspect services that are restarting and check database permissions:

docker compose logs --tail 50 hermes
docker compose logs -f collector
./db/check_permissions.sh

The setup skill says each permission-check line should report ok. Read service logs before changing configuration. Missing environment values, an incorrect model endpoint, or a database initialized with different passwords can prevent startup.

4. Ask one verifiable question

Invite the installed bot into the pilot channel, replacing the name if needed:

/invite @Open-Agent

Allow time for collection: the README describes a five-minute Slack collection interval. Ask about a non-sensitive decision whose original message you can check:

@Open-Agent What did we decide about [project decision]?
Include your sources and distinguish proposals from decisions.

Then inspect the response:

  1. Open or locate each cited source in the original channel.
  2. Check that it supports the answer, including the date, owner and decision status.
  3. Check that proposals have not been presented as agreed decisions.
  4. Ask about a topic absent from the pilot data. Does the assistant acknowledge missing evidence?
  5. Confirm that everyone allowed to use the assistant may access every connected source.

If the answer is missing or stale, check channel membership, allow for the collection interval, and inspect docker compose logs -f hermes collector. A citation is a way to inspect evidence, not a guarantee of correctness.

Help improve the first-run experience

Try a small pilot and share feedback on GitHub: which setup step was unclear, which model you used, and whether the citations supported the answer. Include your commit, environment and sanitized error details. Do not include credentials or private team messages in a public issue.

Sources: the project README and setup skill. Installation, live Slack behavior and answer quality still need validation in your environment.