AI TOOLKIT
Get started
Recipes/API Servers

Node.js HTTP Server

Learn how to use the AI TOOLKIT in a Node.js HTTP server

2 min readapi serversstreamingView source

You can use the AI TOOLKIT in a Node.js HTTP server to generate text and stream it to the client.

Run it locally

$ npm install ai

Examples

The examples start a simple HTTP server that listens on port 8080. You can e.g. test it using curl:

curl -X POST http://localhost:8080

<Note>

The examples use the Vercel AI Gateway. Ensure that your AI Gateway API key is

set in the AI_GATEWAY_API_KEY environment variable.

</Note>

Full example: github.com/khulnasoft/ai-toolkit/examples/node-http-server

### UI Message Stream

You can use the pipeUIMessageStreamToResponse method to pipe the stream data to the server response.

import { streamText } from 'ai-toolkit';

import { createServer } from 'http';

createServer(async (req, res) => {

const result = streamText({

model: 'openai/gpt-4o',

prompt: 'Invent a new holiday and describe its traditions.',

});

result.pipeUIMessageStreamToResponse(res);

}).listen(8080);

### Sending Custom Data

createUIMessageStream and pipeUIMessageStreamToResponse can be used to send custom data to the client.

import {

createUIMessageStream,

pipeUIMessageStreamToResponse,

streamText,

} from 'ai-toolkit';

import { createServer } from 'http';

createServer(async (req, res) => {

switch (req.url) {

case '/stream-data': {

const stream = createUIMessageStream({

execute: ({ writer }) => {

// write some custom data

writer.write({ type: 'start' });

writer.write({

type: 'data-custom',

data: {

custom: 'Hello, world!',

},

});

const result = streamText({

model: 'openai/gpt-4o',

prompt: 'Invent a new holiday and describe its traditions.',

});

writer.merge(

result.toUIMessageStream({

sendStart: false,

onError: error => {

// Error messages are masked by default for security reasons.

// If you want to expose the error message to the client, you can do so here:

return error instanceof Error ? error.message : String(error);

},

}),

);

},

});

pipeUIMessageStreamToResponse({ stream, response: res });

break;

}

}

}).listen(8080);

### Text Stream

You can send a text stream to the client using pipeTextStreamToResponse.

import { streamText } from 'ai-toolkit';

import { createServer } from 'http';

createServer(async (req, res) => {

const result = streamText({

model: 'openai/gpt-4o',

prompt: 'Invent a new holiday and describe its traditions.',

});

result.pipeTextStreamToResponse(res);

}).listen(8080);

bash
  1. 1curl -X POST http://localhost:8080
index.ts
ts
  1. 1import { streamText } from 'ai-toolkit';
  2. 2import { createServer } from 'http';
  3. 3createServer(async (req, res) => {
  4. 4 const result = streamText({
  5. 5 model: 'openai/gpt-4o',
  6. 6 prompt: 'Invent a new holiday and describe its traditions.',
  7. 7 });
  8. 8 result.pipeUIMessageStreamToResponse(res);
  9. 9}).listen(8080);
index.ts
ts
  1. 1import {
  2. 2 createUIMessageStream,
  3. 3 pipeUIMessageStreamToResponse,
  4. 4 streamText,
  5. 5} from 'ai-toolkit';
  6. 6import { createServer } from 'http';
  7. 7createServer(async (req, res) => {
  8. 8 switch (req.url) {
  9. 9 case '/stream-data': {
  10. 10 const stream = createUIMessageStream({
  11. 11 execute: ({ writer }) => {
  12. 12 // write some custom data
  13. 13 writer.write({ type: 'start' });
  14. 14 writer.write({
  15. 15 type: 'data-custom',
  16. 16 data: {
  17. 17 custom: 'Hello, world!',
  18. 18 },
  19. 19 });
  20. 20 const result = streamText({
  21. 21 model: 'openai/gpt-4o',
  22. 22 prompt: 'Invent a new holiday and describe its traditions.',
  23. 23 });
  24. 24 writer.merge(
  25. 25 result.toUIMessageStream({
  26. 26 sendStart: false,
  27. 27 onError: error => {
  28. 28 // Error messages are masked by default for security reasons.
  29. 29 // If you want to expose the error message to the client, you can do so here:
  30. 30 return error instanceof Error ? error.message : String(error);
  31. 31 },
  32. 32 }),
  33. 33 );
  34. 34 },
  35. 35 });
  36. 36 pipeUIMessageStreamToResponse({ stream, response: res });
  37. 37 break;
  38. 38 }
  39. 39 }
  40. 40}).listen(8080);
index.ts
ts
  1. 1import { streamText } from 'ai-toolkit';
  2. 2import { createServer } from 'http';
  3. 3createServer(async (req, res) => {
  4. 4 const result = streamText({
  5. 5 model: 'openai/gpt-4o',
  6. 6 prompt: 'Invent a new holiday and describe its traditions.',
  7. 7 });
  8. 8 result.pipeTextStreamToResponse(res);
  9. 9}).listen(8080);

Troubleshooting

- Streaming not working when proxied

Browse all recipes