[Feature][Rest api] Use jetty to replace Hazelcast native rest api

[liugddx]

Search before asking

• I had searched in the feature and found no similar feature requirement.

Description

Backgroud

The current rest api implementation is to use the native Hazelcast rest api, which is simple but there are very many limitations, such as the inability to customize the url, the performance is not as good as jetty, and scalability is not enough.

Motivation

The Hazelcast Native REST API presents significant limitations in functionality, scalability, performance, security, maintenance, and integration, which hinder its effectiveness in complex front-end and back-end unified projects. On the other hand, Jetty stands out as a high-performance, feature-rich, and highly customizable web server capable of addressing the demands of modern Java applications.

Therefore, it is recommended to phase out the Hazelcast Native REST API and adopt Jetty as the unified front-end and back-end server solution. This transition will enhance system performance, strengthen security, improve maintainability, and provide a robust foundation for future feature expansions.

Architecture design

The current rest api design looks like this,it is implemented through hazelcast native rest api.

 @Override
    public TextCommandService createTextCommandService() {
        return new TextCommandServiceImpl(node) {
            {
                register(HTTP_GET, new Log4j2HttpGetCommandProcessor(this));
                register(HTTP_POST, new Log4j2HttpPostCommandProcessor(this));
                register(HTTP_GET, new RestHttpGetCommandProcessor(this));
                register(HTTP_POST, new RestHttpPostCommandProcessor(this));
            }
        };
    }

• RestHttpGetCommandProcessor provides processing of get requests.
  • /hazelcast/rest/maps/overview
  • /hazelcast/rest/maps/running-jobs
  • /hazelcast/rest/maps/job-info
  • /hazelcast/rest/maps/finished-jobs
  • /hazelcast/rest/maps/encrypt-config
  • /hazelcast/rest/maps/thread-dump
  • /hazelcast/rest/maps/system-monitoring-information
• RestHttpPostCommandProcessor provides processing of post requests.
  • /hazelcast/rest/maps/submit-job
  • /hazelcast/rest/maps/submit-jobs
  • /hazelcast/rest/maps/stop-job
  • /hazelcast/rest/maps/stop-jobs
  • /hazelcast/rest/maps/update-tags

You can see that the current URL cannot be modified, which limits our use.

SeaTunnelServer binding jetty service

Since SeaTunnelServer is a resident process, we can bind a new port specifically for jetty use.We can add two configuration items

seatunnel:
  engine:
  	jetty-port: 8080
    context-path: /hazelcast/rest/maps

This is part of the code of SeaTunnelServerStarter

 private static HazelcastInstanceImpl initializeHazelcastInstance(
            @NonNull SeaTunnelConfig seaTunnelConfig, String customInstanceName) {
        ...

        // create jetty server
        createJettyServer(original);

        return original;
    }
 
 public static void createJettyServer(HazelcastInstanceImpl hazelcastInstance) {
        SeaTunnelConfig seaTunnelConfig = ConfigProvider.locateAndGetSeaTunnelConfig();
        Server server = new Server(seaTunnelConfig.getEngineConfig().getJettyPort());

        ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
        context.setContextPath("/");

        context.setResourceBase(
                SeaTunnelServerStarter.class.getClassLoader().getResource("").toExternalForm());
        context.addServlet(
                new org.eclipse.jetty.servlet.ServletHolder(
                        "default", new org.eclipse.jetty.servlet.DefaultServlet()),
                "/");

        ServletHolder overviewHolder = new ServletHolder(new OverviewServlet(hazelcastInstance));

        context.addServlet(overviewHolder, convertUrlToPath(seaTunnelConfig, OVERVIEW));

        server.setHandler(context);

        try {
            try {
                server.start();
                server.join();
            } catch (Exception e) {
                log.error("Jetty server start failed", e);
                throw new RuntimeException(e);
            }
        } finally {
            server.destroy();
        }
    }

Existing API interfaces can be quickly migrated to Jetty,The following is an example of getting a running jobs

public class RunningJobsServlet extends BaseServlet {

    public RunningJobsServlet(HazelcastInstanceImpl hazelcastInstance) {
        super(hazelcastInstance);
    }

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp)
            throws ServletException, IOException {
        IMap<Long, JobInfo> values = hazelcastInstance.getMap(Constant.IMAP_RUNNING_JOB_INFO);
        JsonArray jobs =
                values.entrySet().stream()
                        .map(
                                jobInfoEntry ->
                                        convertToJson(
                                                jobInfoEntry.getValue(), jobInfoEntry.getKey()))
                        .collect(JsonArray::new, JsonArray::add, JsonArray::add);
        writeJson(resp, jobs);
    }
}

The new rest api can coexist with the old one and can completely replace it in future releases.

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[liugddx]

@hailin0 @Hisoka-X PTAL

[liugddx]

#7647

[hailin0]

suggestion default config

seatunnel:
  engine:
  	http
            port: 5800
            context-path: /api

[hailin0]

How to coexist /api and /hazelcast/rest/maps? This is the premise for smooth migration

[liugddx]

How to coexist /api and /hazelcast/rest/maps? This is the premise for smooth migration

Yes, I will keep the original url path unchanged, and the new url will be bound to the new port

[Hisoka-X]

Closed by #7647