Kishore Kanthan

Analytics AI agents

Agents that find duplicate metrics, review code changes before people do, and draft missing metric descriptions.

Before someone builds a metric, an agent searches by meaning for one that already exists and flags the likely duplicate.

The problem

The metric library was large, and many metrics were near copies of each other. Much of it had no description, so people could not find the metric they needed and wrote a new one. Code review was also slow, and reviewers spent time on checks a machine could do.

Constraints

  • An agent must never see more data than the person using it.
  • Every agent action that changes something needs a person to approve it.
  • Answers must be checked against a fixed set of questions before release.

What I built

  • Metadata from code. A pipeline that uses AI to draft a plain description of each metric. It fills the gaps in the list of metrics. Summer interns built it, and I mentored them. See KPI metadata drafts.

  • A metric discovery agent. Ask it for a metric and it searches by meaning, not exact words. It finds what already exists and flags likely duplicates.

  • Logic updates by pipeline. An AI tool that updates the code behind a metric, one step at a time.

  • The KPI assistant. My part in the end-user assistant: metadata upkeep, regression tests and the test set. See KPI assistant.

  • A content agent, in pilot. It audits, fixes and migrates dashboard content, and builds visualisations. It works in three roles: analyst, builder and migrator.

  • A pull request review agent. I wrote what it checks. Before a person reviews a change, the agent checks that:

    • the change does what its Jira ticket asks;
    • it agrees with the data product manager on the business need, and with the architect on the design;
    • the KPI inventory is updated;
    • the technical page in Confluence is updated;
    • the release notes are updated.

    Then it posts the result to the team chat.

  • A team coding workflow. One AI agent writes and another reviews. Shared rules and guardrails sit in code: no merge, tag or production change happens without a person.

  • A shared skills repo. I built the whole repo. Agents use its skills for routine team work:

    • create Jira tickets, act on tickets, and look into the team’s Jira projects;
    • create a standard operating procedure (SOP) page, and act on other Confluence pages;
    • make a data pipeline change;
    • check a pull request against its template;
    • get DORA metrics, sprint velocity and sprint status in one step.

    To build the Jira skills, I probed the company’s Jira instance to work out its API. I then ran a demo and rolled the skills out to the teams.

How the agents connect

  • One set of metadata. The drafts pipeline writes the descriptions. The discovery agent and the KPI assistant both search them.
  • One rule on access. Every agent runs as the person using it, never as a shared account. The KPI assistant page shows how access stays with the user.
  • One way to release. An agent ships only after it passes a fixed test set. The KPI assistant page shows how the test set is built.
  • Agents draft, people decide. The agents remove routine work. They do not merge or publish on their own.

What changed

Duplicate metrics are easier to find before someone builds another one. Reviewers start from a checked change, not a raw one. The team spends less time on routine checks.

What I’d do differently

Build the shared parts first: the metadata, the access rule and the test set. Each new agent then starts from them.