createEmendAiHandler

createEmendAiHandler builds your server route. It checks each request from the browser, lets you decide who is allowed, calls your AI model through a generate function you write, and streams the answer back.

It returns a standard (request: Request) => Promise<Response> function, so it works with Next.js route handlers and any framework that uses standard Request and Response objects. It works with any AI provider.

import { createEmendAiHandler } from "@emend/ai/server"

Options

interface EmendAiHandlerOptions {
  readonly generate: (
    request: EmendAiRequest,
    signal: AbortSignal
  ) => AsyncIterable<string>
  readonly authorize?: (
    request: Request,
    payload: EmendAiRequest
  ) => Promise<boolean> | boolean
  readonly manifest?: readonly EmendActionManifestEntry[]
  readonly limits?: Partial<EmendRequestLimits>
}

Start without an AI model

import { createEmendAiHandler, mockGenerate } from "@emend/ai/server"

export const POST = createEmendAiHandler({
  generate: mockGenerate,
})

mockGenerate returns fake answers, so you can test the whole route without an API key.

Connect an AI model

generate only runs for requests that passed every check. Call your model, and yield each piece of Markdown text as it arrives.

import { createEmendAiHandler } from "@emend/ai/server"

export const POST = createEmendAiHandler({
  async *generate(request, signal) {
    const stream = createProviderStream({
      action: request.actionId,
      instruction: request.instruction,
      target: request.targetMarkdown,
      context: request.contextMarkdown,
      messages: request.messages,
      signal,
    })

    for await (const chunk of stream) yield chunk
  },
})

Pass signal to your provider. If the user clicks Stop or closes the page, the model stops too, so you do not pay for an answer nobody sees.

Check who is making the request

export const POST = createEmendAiHandler({
  authorize: async (request, payload) => {
    const user = await getCurrentUser(request)
    return Boolean(user && (await userCanEdit(user, payload)))
  },
  generate,
})

Return false to reject the request. If authorize throws, the browser gets a generic internal_error, so your server's error details stay private.

What happens to each request

1. The request must be a POST with JSON.
2. The JSON is checked: format, action, and size.
3. authorize runs, if you added it.
4. Each piece of text from generate is sent to the browser.
5. The stream ends with done, or with an error if generate failed.

HTTP responses

A successful response streams with text/event-stream; charset=utf-8. Read Production Considerations before you go live.