observability

LangSmith

Connect your LangSmith workspace to let agents query traces, open failing runs, and compare error rates, latency, tokens, and cost across your tracing projects. Polylane watches each project for changes in errors, latency, and cost, and your LangSmith alert rules reach Polylane the moment they fire.

Connect

Connect LangSmith from the terminal. Pass --help for the flags LangSmith accepts.

Terminal

polylane integration connect --type langsmith

A service key from Settings → API Keys in LangSmith, with access to the workspace you want to connect.

Connect your LangSmith workspace with a service key.

In LangSmith settings, open API Keys and create a service key for the workspace you want to connect.
Paste the key into Polylane. Polylane checks it and finds the region, the organization, and the workspace. If the key can read more than one workspace, pick the one to connect.

Read LangSmith traces, runs, feedback, and datasets through Polylane, and request API changes. Changes require safety review and approval.

Polylane watches each tracing project for changes in errors, latency, tokens, and cost, and adds a webhook action to each of your LangSmith alert rules, so an alert that fires opens an issue in Polylane.

Polylane uses these LangSmith APIs:

LangSmith API	What it unlocks
`GET /api/v1/sessions`	Lists tracing projects with their run counts, error rates, latency, tokens, and cost.
`POST /api/v1/runs/query`, `GET /api/v1/runs/{id}`	Finds failing and slow runs and opens a whole trace.
`POST /api/v1/sessions/{id}/dashboard`	Reads error rate, latency, token, and cost series per project.
`GET /api/v1/feedback`	Reads feedback scores and comments on runs.
`GET /api/v1/datasets`, `GET /api/v1/examples`	Reads datasets and their examples.
`/api/v1/platform/alerts`	Reads your alert rules and adds the Polylane webhook action to each one.
`langsmithApi`	Calls any LangSmith API; calls that change state require safety review and approval. API keys and trace ingestion are not available.

Disconnecting removes the Polylane webhook action from your alert rules and leaves the service key in LangSmith. Remove it under Settings → API Keys in LangSmith.

Disconnect

Disconnecting stops every sync and check and deletes the credentials Polylane held. Polylane removes what LangSmith's API lets it remove. The following stays until you remove it yourself:

LangSmith API key

Terminal

polylane integration disconnect <id>

Tools

Operations agents can perform in threads.

List Projects

List LangSmith tracing projects with run counts, error rates, latency, tokens and cost

Query Runs

Search LangSmith runs and traces by project, time window, status, run type and filter

Get Trace

Read one LangSmith trace as a run tree with latency, tokens, cost and errors

Get Project Metrics

Read LangSmith project metrics over time: traces, errors, latency, LLM calls, tokens and cost

List Feedback

List LangSmith feedback scores and comments on runs

List Datasets

List LangSmith datasets and read dataset examples

List Alert Rules

List LangSmith alert rules and where each rule sends its alerts

Api

Call the connected LangSmith REST API

Back to integrations