Using OpenSearch Java Client and Spring Boot for Powerful Search Integration

In today’s data-driven world, the ability to quickly search, analyze, and visualize vast amounts of information is crucial. Enter OpenSearch, a powerful and versatile open-source suite designed to do just that. But what exactly is OpenSearch, and why should you consider it? Let’s break it down.

What is OpenSearch and How Does the OpenSearch Java Client Work?

At its core, OpenSearch is an open-source search and analytics suite. Think of it as a toolbox packed with everything you need to:

• Index and store massive datasets: From application logs and website traffic to e-commerce transactions and sensor data, OpenSearch can handle it all.
• Search and explore your data in near real-time: Need to find specific information quickly? OpenSearch offers powerful full-text search, structured search, and even geospatial search capabilities.
• Visualize and analyze your data: Go beyond simple search with built-in tools for creating dashboards, visualizations, and reports to gain deeper insights from your data.

OpenSearch is built upon the foundation of Apache 2.0 licensed software, ensuring it’s truly open and free to use, modify, and distribute. It’s designed to be highly scalable and distributed, meaning it can grow with your data needs, whether you’re a small startup or a large enterprise.

The Backstory: Why OpenSearch Exists and How It Compares to Elasticsearch

To understand OpenSearch fully, it’s important to know its origins. OpenSearch is a fork of Elasticsearch and Kibana. This happened when Elasticsearch, originally open source, transitioned to a dual license model that restricted certain features. A community-driven effort emerged to maintain a truly open-source alternative, leading to the birth of OpenSearch. 

This backstory is important because it highlights the commitment to open source principles that are deeply embedded in OpenSearch. It’s not just about free software; it’s about community collaboration, transparency, and ensuring powerful search and analytics tools remain accessible to everyone.

In this context, the OpenSearch vs Elasticsearch comparison becomes important for developers when considering which search solution best fits their application needs.

Key Benefits of OpenSearch: Why Choose OpenSearch Java SDK for Your Application?

Now, let’s get to the juicy part: the benefits of using OpenSearch. Here are some compelling reasons why you should consider it for your search and analytics needs:

• Truly Open Source: This is perhaps the biggest differentiator. OpenSearch is licensed under Apache 2.0, guaranteeing freedom from vendor lock-in and licensing fees. You have full control and transparency over the software.
• Cost-Effective: Beyond the lack of licensing fees, OpenSearch can lead to significant cost savings compared to proprietary solutions. You control your infrastructure and can optimize resource usage.
• Highly Scalable and Performant: Designed for massive datasets and high query volumes, OpenSearch can scale horizontally to handle growing data needs. It’s built for speed and efficiency. To maximize the effectiveness of OpenSearch, performance tuning is essential, especially for high-query-volume applications.
• Feature-Rich: Don’t think open source means lacking features. OpenSearch is packed with powerful capabilities:
  • Full-Text Search: Find relevant information within documents using advanced text analysis.
  • Structured Search: Query data based on specific fields and values.
  • Geospatial Search: Analyze location-based data.
  • Analytics and Aggregations: Perform complex data aggregations and calculations.
  • Dashboards and Visualizations: Create interactive dashboards to monitor key metrics and gain insights.
  • Observability Features: Analyze logs, metrics, and traces for system monitoring and troubleshooting.
  • Security Features: Role-based access control, encryption, and auditing to protect your data.
• Extensible and Customizable: OpenSearch is designed to be extended with plugins and integrations. You can tailor it to your specific needs and connect it with other tools in your ecosystem.
• Active Community and Support: OpenSearch boasts a vibrant and growing community of developers, users, and contributors. You’ll find ample documentation, forums, and community support resources to help you along the way.
• Flexibility and Deployment Options: You can deploy OpenSearch on-premises, in the cloud (self-managed or through managed services offered by various cloud providers), or in hybrid environments.

Step-by-Step Implementation of OpenSearch in Spring Boot

Let’s break down the key parts of our Spring Boot application. By using the OpenSearch Java SDK, integrating OpenSearch becomes easier than ever.

Download the source code and necessary Docker files from our GitHub repository

Prerequisites:

• Java and Maven: Ensure you have Java and a build tool like Maven or Gradle installed.
• Your Favorite IDE: IntelliJ IDEA, Eclipse, or VS Code will work great.
• Running OpenSearch Instance:  Make sure you have an OpenSearch instance running locally or accessible. You can use Docker for a quick setup. I have included a docker-compose file in the source code. You can run the below command to start the OpenSearch with 2 nodes and a dashboard.

docker compose -f docker/opensearch-compose.yaml -p opensearch up 

You can open this URL and see if OpenSearch node is up and running https://localhost:9200/_cluster/health
It will prompt for username and password. The username is admin by default and the password we have provided in the compose file environment section. You can change the password there

OPENSEARCH_INITIAL_ADMIN_PASSWORD

1. Project Setup (using Spring Initializr):

Go to https://start.spring.io/ and create a new Spring Boot project with the following dependencies:

• Spring Web: For building our REST API.
• H2 database: Provides a fast in-memory database that supports JDBC.
• Lombok: Java annotation library which helps to reduce boilerplate code.
• Spring Data JPA: Persist data in SQL stores with Java Persistence API using Spring Data and Hibernate.

Choose your preferred build tool (Maven or Gradle), language (Java), and Spring Boot version. Generate and download the project.

2. Add OpenSearch Dependacy

<dependency>

  <groupId>org.opensearch.client</groupId>

  <artifactId>opensearch-rest-client</artifactId>

  <version>2.19.0</version>

</dependency>
<dependency>

  <groupId>org.opensearch.client</groupId>

  <artifactId>opensearch-java</artifactId>

  <version>2.6.0</version>

</dependency>Code language: HTML, XML (xml)

3. Configure Database and OpenSearch Connection (application.yml):

spring:

 application:

   name: opensearchdemo

 datasource:

   url: jdbc:h2:mem:testdb

   driverClassName: org.h2.Driver

   username: sa

   password: password

 jpa:

   database-platform: org.hibernate.dialect.H2Dialect

 h2:

   console:

     enabled: true

opensearch:

 host: localhost

 port: 9200

 scheme: https

 username: admin

 password: Random!Password1

 truststore:

   path: opensearch-truststore.jks

   password: changeitCode language: CSS (css)

4. Create an OpenSearch Configuration Class

@Configuration

public class OpenSearchConfig {

   @Value("${opensearch.host}")

   private String host;

   @Value("${opensearch.port}")

   private int port;

   @Value("${opensearch.username}")

   private String username;

   @Value("${opensearch.password}")

   private String password;

   @Value("${opensearch.scheme}")

   private String scheme;

   @Value("${opensearch.ssl.truststore.path}")

   private String truststorePath;

   @Bean

   public OpenSearchClient openSearchClient() throws Exception {

       final HttpHost host = new HttpHost(this.host, this.port, this.scheme);

       KeyStore truststore = KeyStore.getInstance("JKS");

       try (InputStream is = new ClassPathResource("opensearch-truststore.jks").getInputStream()) {

           truststore.load(is, "changeit".toCharArray());

       }

       // Create SSL context

       TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());

       tmf.init(truststore);

       SSLContext sslContext = SSLContext.getInstance("TLS");

       sslContext.init(null, tmf.getTrustManagers(), null);

       // Set up credentials

       final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();

       credentialsProvider.setCredentials(new AuthScope(host),

               new UsernamePasswordCredentials(this.username, this.password));

       // Initialize the client with SSL and credentials

       final RestClient restClient = RestClient.builder(host)

               .setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder

                       .setDefaultCredentialsProvider(credentialsProvider)

                       .setSSLContext(sslContext))

               .build();

       final OpenSearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());

       return new OpenSearchClient(transport);

   }

}Code language: PHP (php)

5. Create a Service & Repository Class to Save and retrieve data to and from OpenSearch

@Service

public class StudentService {

   private final StudentRepository studentRepository;

   private final OpenSearchClient openSearchClient;

   private static final String STUDENT_INDEX = "students";

   @Autowired

   public StudentService(StudentRepository studentRepository, OpenSearchClient openSearchClient) {

       this.studentRepository = studentRepository;

       this.openSearchClient = openSearchClient;

   }

   public Student createStudent(Student student) throws IOException {

       // Save to Database

       Student savedStudent = studentRepository.save(student);

       // Save to OpenSearch

       openSearchClient.index(i -> i.index(STUDENT_INDEX).id(savedStudent.getId().toString()).document(savedStudent));

       return savedStudent;

   }

   public Student updateStudent(Long id, Student student) throws IOException {

       Student existingStudent = getStudent(id);

       existingStudent.setName(student.getName());

       existingStudent.setEmail(student.getEmail());

       // Update in Database

       Student updatedStudent = studentRepository.save(existingStudent);

       // Update in OpenSearch

       openSearchClient.index(i -> i.index(STUDENT_INDEX).id(String.valueOf(id)).document(updatedStudent));

       return updatedStudent;

   }

   public Student getStudent(Long id) {

       // Get from Database

       return studentRepository.findById(id)

               .orElseThrow(() -> new RuntimeException("Student not found"));

   }

   public void deleteStudent(Long id) throws IOException {

       // Delete from OpenSearch

       openSearchClient.delete(d -> d.index(STUDENT_INDEX).id(String.valueOf(id)));

       // Delete from Database

       studentRepository.deleteById(id);

   }

   public List<Student> searchStudents(StudentSearchRequest searchRequest) throws IOException {

       // Build the search query

       BoolQuery.Builder boolQueryBuilder = new BoolQuery.Builder();

       // Add search conditions for each field

       if (searchRequest.getSearchFields() != null) {

           for (Map.Entry<String, String> field : searchRequest.getSearchFields().entrySet()) {

               boolQueryBuilder.must(Query.of(q -> q

                   .match(m -> m

                       .field(field.getKey())

                       .query(FieldValue.of(field.getValue()))

                   )

               ));

           }

       }

       // Build the search request with pagination

       SearchRequest request = new SearchRequest.Builder()

           .index(STUDENT_INDEX)

           .query(boolQueryBuilder.build()._toQuery())

           .from(searchRequest.getOffset())

           .size(searchRequest.getLimit())

           .build();

       // Execute search

       SearchResponse<Student> searchResponse = openSearchClient.search(request, Student.class);

       // Convert response to list

       List<Student> students = new ArrayList<>();

       searchResponse.hits().hits().forEach(hit -> students.add(hit.source()));

       return students;

   }

}Code language: JavaScript (javascript)

6. Create a REST Controller (StudentController.java):

@RestController

@RequestMapping("/api/v1/students")

public class StudentsController {

   private static final Logger LOGGER = LoggerFactory.getLogger(StudentsController.class);

   @Autowired

   private StudentService studentService;

   @PostMapping

   public ResponseEntity<Student> createStudent(@RequestBody Student student) {

       try {

           Student createdStudent = studentService.createStudent(student);

           return ResponseEntity.ok(createdStudent);

       } catch (IOException e) {

           LOGGER.error("Error creating student", e);

           return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();

       }

   }

   @PutMapping("/{id}")

   public ResponseEntity<Student> updateStudent(@PathVariable Long id, @RequestBody Student student) {

       Student existingStudent = studentService.getStudent(id);

       if (existingStudent == null) {

           return ResponseEntity.status(HttpStatus.NOT_FOUND).build();

       }  

       try {

           Student updatedStudent = studentService.updateStudent(id, student);

           return ResponseEntity.ok(updatedStudent);

       } catch (IOException e) {

           LOGGER.error("Error updating student", e);

           return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();

       }

   }

   @PostMapping("/search")

   public ResponseEntity<List<Student>> searchStudents(@RequestBody StudentSearchRequest searchRequest) {

       try {

           List<Student> students = studentService.searchStudents(searchRequest);

           return ResponseEntity.ok(students);

       } catch (IOException e) {

           LOGGER.error("Error searching students", e);

           return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();

       }

   }

}Code language: JavaScript (javascript)

Run the application and test with Swagger

http://localhost:8080/swagger-ui/index.html#

Conclusion

From powering lightning-fast search experiences to enabling insightful data analysis, OpenSearch, especially when used as a search cache, transforms how applications interact with data. By offloading search workloads and providing specialized search capabilities, it frees up your primary systems and unlocks new levels of performance and user satisfaction. As data volumes continue to explode, tools like OpenSearch, with their open, scalable, and adaptable nature, will become increasingly indispensable for building responsive and intelligent applications. The future of data interaction is open and searchable, and OpenSearch is at the forefront.

Author's Bio:

Pratik Kale