OpenAI LLM

OpenAILLM provides the language model integration between OpenAI and the Ragfish Core Framework.

It allows Ragfish applications to use an OpenAI language model for generating AI responses.

The implementation is provided by the @ragfish/openai package and can be configured through Settings.llm.

Installation

Install the OpenAI integration package:

npm install @ragfish/openai

Make sure the Ragfish Core package is also installed:

npm install @ragfish/core

API Key

Store your OpenAI API key in an environment variable.

OPENAI_API_KEY=your-api-key

Do not hard-code API credentials in your application source code.

Basic Configuration

Import Settings from @ragfish/core and OpenAILLM from @ragfish/openai.
import { Settings } from "@ragfish/core";
import { OpenAILLM } from "@ragfish/openai";

Settings.llm = new OpenAILLM({
 apiKey: process.env.OPENAI_API_KEY
});

Once configured, the Ragfish framework can use the OpenAI language model when generating responses.

Using OpenAILLM with Chat

After configuring the LLM, create a Chat instance.

import { Chat, Settings } from "@ragfish/core";
import { OpenAILLM } from "@ragfish/openai";

Settings.llm = new OpenAILLM({
 apiKey: process.env.OPENAI_API_KEY
});

const chat = new Chat({
 retriever
});
You can then send a message:
const response = await chat.message(
 "What is Ragfish?"
);

console.log(response);

The overall flow is:

User Question
     |
     ▼
    Chat
     |
     ▼
 Retriever
     |
     ▼
Relevant Context
     |
     ▼
OpenAILLM
     |
     ▼
AI Response

OpenAILLM in a RAG Application

OpenAILLM is responsible for the generation stage of the RAG pipeline.

The embedding model handles vector generation and retrieval, while OpenAILLM generates the final response.

Knowledge
   |
   ▼
Chunking
   |
   ▼
OpenAI Embedding
   |
   ▼
Vector Store
   |
   ▼
Retriever
   |
   ▼
Chat
   |
   ▼
OpenAILLM
   |
   ▼
Response

This separation allows the language model and embedding model to have independent responsibilities.

Configuration

The OpenAILLM configuration should be based on the options supported by the version of @ragfish/openai installed in your project.

The basic configuration requires an OpenAI API key:

const llm = new OpenAILLM({
 apiKey: process.env.OPENAI_API_KEY
});

If your installed version exposes additional model or generation options, refer to the package's TypeScript definitions and API Reference for the supported configuration.

Environment-Based Configuration

For production applications, keep provider configuration outside your source code.

Example

OPENAI_API_KEY=your-api-key

Then:

const llm = new OpenAILLM({
 apiKey: process.env.OPENAI_API_KEY
});

Settings.llm = llm;

This makes it easier to manage different environments without changing application code.

Complete Example

import { Settings, Chat } from "@ragfish/core";
import { OpenAILLM } from "@ragfish/openai";

Settings.llm = new OpenAILLM({
 apiKey: process.env.OPENAI_API_KEY
});

const chat = new Chat({
 retriever
});

const response = await chat.message(
 "What is Ragfish?"
);

console.log(response);

Error Handling

OpenAI requests depend on an external service, so applications should handle failures appropriately.

For example:

try {
 const response = await chat.message(
   "What is Ragfish?"
 );

 console.log(response);
} catch (error) {
 console.error("AI request failed:", error);
}

In production applications, avoid exposing internal provider errors directly to users. Log the technical details and return an appropriate application-level message.

Best Practices

When using OpenAILLM:

1. Store the API key in environment variables.

2. Configure the LLM during application startup.

3. Keep provider configuration separate from application logic.

4. Use the Settings.llm abstraction rather than coupling your application directly to provider-specific APIs.

5. Refer to the installed package's TypeScript definitions for the exact supported configuration options.

6. Handle provider and network errors at the application boundary.

Next Steps

Now that you understand how OpenAILLM provides language-model capabilities to Ragfish, continue to OpenAI Embeddings to learn how OpenAI embeddings are used to prepare knowledge and queries for semantic retrieval.