Building a Search API with NestJS and OpenSearch

The integration of OpenSearch with NestJS enables developers to easily add search functionality to their applications. Developers can create robust search functionality quickly and easily, without having to worry about the complexities of search engine implementation.

Here’s an example of what the folder structure could look like for a NestJS project with an OpenSearch integration:

src/
├── search/
│   ├── search.module.ts
│   ├── search.service.ts
│   └── search.dto.ts
└── main.ts

The search/ folder contains the module, service, and dto for the OpenSearch integration

This is a sample dataset that is being used for the service.

[
    {
        "id": 1,
        "name": "Rhaenyra Targaryen",
        "quote": "First Queen."
    },
    {
        "id": 2,
        "name": "Daemon Targaryen",
        "quote": "You cannot live your life in fear, or you will forsake the best parts of it."
    },
    {
        "id": 3,
        "name": "Corlys Velaryon",
        "quote": "Our worth is not given. it must be made."
    },
    {
        "id": 4,
        "name": "Jaime Lannister",
        "quote": "The things I do for love."
    },
    {
        "id": 5,
        "name": "Tyrion Lannister",
        "quote": "I drink and I know things."
    },
    {
        "id": 6,
        "name": "Jon Snow",
        "quote": "I don't know anything."
    },
    {
        "id": 7,
        "name": "Daenerys Targaryen",
        "quote": "I am the blood of the dragon."
    },
    {
        "id": 8,
        "name": "Arya Stark",
        "quote": "A girl has no name."
    }
]

First, let’s take a look at search.module.ts

import { Module, DynamicModule } from "@nestjs/common";
import { Client } from "@opensearch-project/opensearch";
import { SearchService } from "./search.service";

@Module({})
export class SearchModule {
  static register(url): DynamicModule {
    return {
      module: SearchModule,
      providers: [
        SearchService,
        {
          provide: "Open_Search_JS_Client",
          useValue: {
            instance: new Client({
              node: "https://localhost:9200",
              ssl: {
                rejectUnauthorized: false,
              },
            }),
          },
        },
      ],
      exports: [SearchService],
    };
  }
}

Explaination:

The SearchModule is a module in NestJS that is responsible for providing the SearchService and the OpenSearch client instance. It uses the @opensearch-project/opensearch package to create the client instance, and exports the SearchService for use in other parts of the application.

Here’s a breakdown of the code:

• The @nestjs/common package provides the Module and DynamicModule decorators that are used in this file.
• The @opensearch-project/opensearch package provides the Client class that is used to create the OpenSearch client instance.
• The static register() method is a static method that returns a DynamicModule object. It takes a url parameter that is used to connect to the OpenSearch server.
• The providers property is an array of providers that are used by the module. In this case, it provides the SearchService and an instance of the OpenSearch client.
• The provide property is used to specify the token that will be used to inject the OpenSearch client instance into other parts of the application. In this case, it is set to "Open_Search_JS_Client".
• The useValue property is used to provide a value for the provider. In this case, it creates a new instance of the Client class using the provided url parameter.

Let’s focus on the search.service.ts now :

Here is a search.dto.ts class that is being imported in a service.

export class DataSet {
  indexName: string;
  characters: Charaters[];
}

export class Charaters {
  id: string;
  name: string;
  quote: string;
}

export class DeleteInput {
  indexName: string;
  id?: string;
}

export class searchCharacterByKeyword {
  indexName: string;
  keyword: string;
}

Overall, NestJS SearchService is designed to provide easy integration with OpenSearch and implement various search functionalities. The service consists of five main methods, namely, bulkDataIngestion(), singleDataIngestion(), searchCharacterByKeyword(), purgeIndex(), and purgeDocumentById().

import { Injectable, Logger } from "@nestjs/common";
import { DataSet, DeleteInput, searchCharacterByKeyword } from "./series.dto";

@Injectable()
export class SearchService {
  private openSearchClient: any;
  logger: Logger;

  constructor(@Inject("Open_Search_JS_Client") openSearchClient) {
    this.openSearchClient = openSearchClient.instance;
    this.logger = new Logger();
  }

  async bulkDataIngestion(input: DataSet): Promise<any> {
    
  }

  async singleDataIngestion(input: DataSet): Promise<any> {
    
  }

  async searchCharacterByKeyword(input: searchCharacterByKeyword): Promise<any> {
 
  }

  async purgeIndex(input: DeleteInput): Promise<any> {
   
  }

  async purgeDocumentById(input: DeleteInput): Promise<any> {
   
  }
}

It uses dependency injection to inject an instance of the OpenSearch JavaScript client that is registered as a provider in the module.ts file of the application. The private openSearchClient: any; property holds the instance of the OpenSearch client, which is used by the service to perform search operations on the OpenSearch cluster.

Method 1: bulkDataIngestion()

This method is responsible for ingesting bulk data into an Opensearch index. It takes a DataSet object as input, which contains an array of characters and the name of the index. It then creates a body object using the flatMap() method, which returns an array of objects that contain the index and doc properties. The index property contains the index name and the id property of the character, while the doc property contains the character object itself.

async bulkDataIngestion(input: DataSet): Promise<any> {
  this.logger.log(
    `Inside bulkUpload() Method | Ingesting Bulk data of length ${input.characters.length} having index ${input.indexName}`
  );

  const body = input.characters.flatMap((doc) => {
    return [{ index: { _index: input.indexName, _id: doc.id } }, doc];
  });

  try {
    let res = await this.openSearchClient.bulk({ body });
    return res.body;
  } catch (err) {
    this.logger.error(`Exception occurred : ${err})`);
    return {
      httpCode: 500,
      error: err,
    };
  }
}

Method 2: singleDataIngestion()

This method is responsible for ingesting a single data point into an OpenSearch index. It takes a DataSet object as input, which contains a single character object and the name of the index. It then creates a body object using the character object's id, name, and quote properties.

async singleDataIngestion(input: DataSet): Promise<any> {
  this.logger.log(
    `Inside singleUpload() Method | Ingesting single data with index ${input.indexName} `
  );

  let character = input.characters[0];

  try {
    let res = await this.openSearchClient.index({
      id: character.id,
      index: input.indexName,
      body: {
        id: character.id,
        name: character.name,
        quote: character.quote,
      },
    });
    return res.body;
  } catch (err) {
    this.logger.error(`Exception occurred : ${err})`);
    return {
      httpCode: 500,
      error: err,
    };
  }
}

Method 3: searchCharacterByKeyword()

This method is responsible for searching for a character by keyword in an OpenSearch index. It takes a searchCharacterByKeyword object as input, which contains the keyword to search for and the name of the index. It then creates a body object using the multi_match query type, which searches for the keyword in multiple fields.

async searchCharaterByKeyword(input: searchCharacterByKeyword): Promise<any> {
  this.logger.log(`Inside searchByKeyword() Method`);
  let body: any;

  this.logger.log(
    `Searching for Keyword: ${input.keyword} in the index : ${input.indexName} `
  );
  body = {
    query: {
      multi_match: {
        query: input.keyword,
      },
    },
  };

  try {
    let res = await this.openSearchClient.search({
      index: input.indexName,
      body,
    });
    if (res.body.hits.total.value == 0) {
      return {
        httpCode: 200,
        data: [],
        message: `No Data found based based on Keyword: ${input.keyword}`,
      };
    }
    let result = res.body.hits.hits.map((item) => {
      return {
        _id: item._id,
        data: item._source,
      };
    });

    return {
      httpCode: 200,
      data: result,
      message: `Data fetched successfully based on Keyword: ${input.keyword}`,
    };
  } catch (error) {
    this.logger.error(`Exception occurred while doing : ${error})`);
    return {
      httpCode: 500,
      data: [],
      error: error,
    };
  }
}

Method 4: purgeIndex()

This method deletes all records from the specified index. The DeleteInput parameter contains the name of the index to be deleted. The method uses the indices.delete() function of the OpenSearch client to delete the entire index.

async purgeIndex(input: DeleteInput): Promise<any> {
    this.logger.log(`Inside purgeIndex() Method`);
    try {
      this.logger.log(`Deleting all records having index: ${input.indexName}`);

      await this.openSearchClient.indices.delete({
        index: input.indexName,
      });

      return {
        httpCode: 200,
        message: `Record deleted having index: ${input.indexName}, characterId: ${input.id}`,
      };
    } catch (error) {
      this.logger.error(`Exception occurred while doing : ${error})`);
      return {
        httpCode: 500,
        error: error,
      };
    }
  }

Method 5: purgeDocumentById()

This method deletes all records from the specified index. The DeleteInput parameter contains the name of the index to be deleted. The method uses the indices.delete() function of the OpenSearch client to delete the entire index.

async purgeDocumentById(input: DeleteInput): Promise<any> {
    this.logger.log(`Inside purgeDocumentById() Method : ${input}`);
    try {
      if (input.id != null && input.indexName != null) {
        this.logger.log(
          `Deleting record having index: ${input.indexName}, id: ${input.id}`
        );
        await this.openSearchClient.delete({
          index: input.indexName,
          id: input.id,
        });
      } else {
        this.logger.log(`indexName or document id is missing`);
        return {
          httpCode: 200,
          message: `indexName or document id is missing`,
        };
      }

      return {
        httpCode: 200,
        message: `Record deleted having index: ${input.indexName}, id: ${input.id}`,
      };
    } catch (error) {
      this.logger.error(
        `Exception occurred while doing purgeDocumentById : ${error})`
      );
      return {
        httpCode: 500,
        message: error,
      };
    }
}

This method deletes a specific record from the specified index. The DeleteInput parameter contains the name of the index and the ID of the document to be deleted. The method uses the delete() function of the OpenSearch client to delete the specified document.

With the help of the provided code examples and explanations, you should now have a better understanding of how to get started with this integration. Happy searching!