Skip to main content

OpenAI Embedder

Generate embeddings using OpenAI's API, enabling semantic analysis and machine learning tasks.

What is OpenAI Embedder Node?​

The image shows a user interface element labeled 'OpenAI Embedder V2 Knowledge' with options for credentials. There are icons for information, settings, copying, and deleting. A red asterisk indicates required credentials. An output labeled 'Embedder OpenAIEmbedder' is connected on the right.

The OpenAI Embedder Node connects to OpenAI's API using a specified API key to generate embeddings. These embeddings are useful for various tasks such as semantic analysis, natural language processing, and machine learning. The node outputs the embedder to be used withing your workflow.

How to use it?​

To use the OpenAI Embedder Node, follow these steps:

  1. Drag the OpenAI Embedder Node into the Workflow:

    • Locate the OpenAI Embedder Node under the 'Knowledge' category.
    • Drag and drop it into your workflow canvas.

  2. Configure the OpenAI Embedder Node:

  • Credentials: Select your OpenAI Credentials
  1. Connect Nodes:
    • Since the OpenAI Embedder Node does not have input anchors, you can directly connect it with nodes that provide the necessary inputs for embedding.
    • Attach the output anchor of the OpenAI Embedder Node to any downstream nodes that will use the generated embeddings. The output type is embedder|openAIEmbedder.

Example of usage​

Here is an example workflow that integrates the OpenAI Embedder Node with an OpenSearch Vector Store for document indexing and retrieval. This setup allows you to store and search various documents like project plans, meeting transcripts, and other data efficiently.

Flowchart with six interconnected modules on a dark background. Modules include 'OpenSearch Client', 'OpenAI Embedder', 'Vector Store', 'API Input', 'Document Splitter', and 'Vector Store Writer'. Lines connect modules, indicating data flow between them. Each module has editable fields.

Step-by-Step Guide:​

  1. API Input

    • Purpose: Capture incoming documents, such as project plans, meeting transcripts, and other data.
    • Configuration:
      • Add a API Input node to your workflow.
      • Configure the output anchors dynamically to match the fields in the API call's POST body.

  2. OpenAI Embedder

    • Purpose: Generate embeddings for the captured text data.
    • Configuration:
      • Add the OpenAI Embedder Node to your workflow.
      • Enter your OpenAI API key in the configuration panel.

  3. OpenSearch Client

    • Purpose: Configure and connect to an OpenSearch cluster.
    • Configuration:
      • Add an OpenSearch Client node to your workflow.
      • Enter the endpoint URL for the OpenSearch cluster.

  4. Vector Store

    • Purpose: Manage document vector embeddings.
    • Configuration:
      • Add a Vector Store node to your workflow.
      • Enter the index name (e.g., "documents").
      • Connect the OpenSearch Client node's output to the Vector Store node.
      • Connect the OpenAI Embedder Node's output to the Vector Store node.

  5. Vector Store Writer

    • Purpose: Write documents to the vector store.
    • Configuration:
      • Add a Vector Store Writer node to your workflow.
      • Connect the Vector Store node's output to the Vector Store Writer node.
      • Connect the API Input node's output to the Vector Store Writer node.

  6. Document Splitter

    • Purpose: Split your incoming documents into smaller chunks so that relevant information can be retreived later.
    • Configuration:
      • Add a Document Splitter node to your workflow.
      • Select the type of the splitter (e.g., "Recursive Character Splitter").
      • Connect the Document Splitters's output to the Vectore Store Writer.

Additional Information​

  • OpenAI API Key: Ensure you have a valid API key from OpenAI to use this node. You can get your API key by signing up on the OpenAI website.
  • OpenSearch Configuration: Ensure your OpenSearch cluster is correctly configured and accessible. You can find more about setting up OpenSearch in the OpenSearch documentation.
  • Document Types: The workflow can handle various document types, such as text, JSON, and others, depending on the configuration of the API Input node.

By following the above steps, you can set up a workflow that uses the OpenAI Embedder Node to generate embeddings for documents, store them in an OpenSearch vector store, and enable efficient document retrieval and search capabilities.

FAQ​

Q: What is an embedding?​

An embedding is a representation of text data in a vector space where similar texts are closer together. This facilitates various downstream tasks like clustering, classification, and semantic search.

Q: How secure is my OpenAI API key?​

Your OpenAI API key is stored securely within the node configuration and is only used to authenticate requests to the OpenAI API. Ensure you do not share your API key publicly.

Q: Can I use this node with other vector databases?​

Yes, while this example uses OpenSearch, the OpenAI Embedder Node can be connected to any vector database that accepts embeddings, provided you have the appropriate nodes to handle the database operations.