API & automation

Middleware package

AI crawlers do not run JavaScript. GPTBot, ClaudeBot, PerplexityBot and friends fetch your HTML and leave, so a browser beacon — which needs a browser — can never see them. @ranksify/middleware is the planned server adapter for seeing them in Express, Fastify or Next.js.

Zero dependencies, Node 18 or newer. It never throws at your application and never awaits in your request path.

Release status

Future package setup

You need two things:

1. Your brand’s id — the UUID in the app URL.
2. An ingest key. Mint one in Settings → your brand → ingest token (the API is POST /api/projects/{id}/ingest-token, admin-only). It looks like rk_… and carries the ingest scope and nothing else.

The brand id must be the same brand the ingest key is limited to.

Express

const express = require("express");
const { ranksifyExpress } = require("@ranksify/middleware");

const app = express();
app.use(
  ranksifyExpress({
    projectId: process.env.RANKSIFY_PROJECT_ID,
    apiKey: process.env.RANKSIFY_API_KEY,
  }),
);

Mount it first, before your routes, so it observes every response. It calls next() synchronously and reports on the response’s finish event, so it adds nothing to your response time.

Fastify

const Fastify = require("fastify");
const { ranksifyFastify } = require("@ranksify/middleware");

const app = Fastify();
app.register(
  ranksifyFastify({
    projectId: process.env.RANKSIFY_PROJECT_ID,
    apiKey: process.env.RANKSIFY_API_KEY,
  }),
);

Next.js

Next middleware is a function you export, not something you use(), so call the reporter from inside your own middleware.ts:

import { NextResponse, type NextFetchEvent, type NextRequest } from "next/server";
import { ranksifyNext } from "@ranksify/middleware";

const report = ranksifyNext({
  projectId: process.env.RANKSIFY_PROJECT_ID!,
  apiKey: process.env.RANKSIFY_API_KEY!,
});

export function middleware(request: NextRequest, event: NextFetchEvent) {
  report(request, event);
  return NextResponse.next();
}

Create the reporter at module scope, as above, not inside middleware(), so its bot registry and bounded retry queue survive warm invocations. Passing Next’s event attaches the immediate delivery promise to event.waitUntil(). The existing one-argument report(request) form still starts an immediate best-effort flush, but a serverless runtime may terminate it.

Options

API

const rk = createRanksify({ projectId, apiKey });

rk.classify(userAgent);                              // → { bot, platform } | null
rk.report({ url, userAgent, referrer, status, ts }); // enqueue, fire-and-forget
await rk.flush();                                    // attempt the queue now — call on shutdown

BUNDLED_BOTS;             // the shipped bot list
formatCombinedLine(hit);  // the exact log line we would send

report() never throws and never awaits. flush() never rejects.

What it reports, and what it does not

Crawlers only. Every adapter classifies the user-agent first and reports nothing unless it matches a known AI crawler. Humans who arrive from an AI assistant are the browser beacon’s job — see the events reference. Install both and nothing is counted twice: two different populations, two different collectors.

No IP addresses. Hits are sent as combined-log-format lines with the IP field set to a literal -. There is nothing to redact later because nothing is collected. The user-agent, path, referrer and, when observable, status are sent; that is all.

Bounded retry delivery. A failed or timed-out batch is reported to onError, returned to the bounded queue and retried once after flushMs. A second failure stays queued until new traffic or an explicit flush; there is no perpetual outage loop. Flushes are serialized, request bodies stay within the ingest limit, and maxQueue remains the hard line-count ceiling.

The bot list keeps itself current. classify() always answers synchronously from the list in hand — the bundled copy at first — and refreshes it in the background from the public registry. A refresh that fails, times out or returns a malformed body leaves the last known-good list in place. New crawlers reach you without a package upgrade, and a Ranksify outage cannot stop you classifying.