How to do sequence of operations and ensure one operation is complete before next one in Spring Reactor web app?

Question

I have Spring Boot 2 web app in which I need to identify site visitor by cookie and gather page view stats. So I need to intercept every web request. The code I had to write is

Answer 1

Another variant which creates pageview and updates the user in the webfilter in a non-blocking way, before passing a request to the controller:

@Bean
public WebFilter filter() {
    return (exchange, chain) -> {
        ServerHttpRequest req = exchange.getRequest();
        String uri = req.getURI().toString();
        log.info("[i] Web Filter: received the request: {}", uri);

        var headers = req.getHeaders();
        List<String> tokenList = headers.get("token");

        if (tokenList != null && tokenList.get(0) != null) {
            String token = tokenList.get(0);
            Mono<User> foundUser = userRepo
                    .findByToken(token)
                    .doOnNext(user -> log.info("[i] Web Filter: {} has been found", user));
            return updateUserStat(foundUser, exchange, chain, uri);
        } else {
            String token = UUID.randomUUID().toString();
            Mono<User> createdUser = userRepo
                    .save(new User(token))
                    .doOnNext(user -> log.info("[i] Web Filter: a new {} has been created", user));
            return updateUserStat(createdUser, exchange, chain, uri);
        }
    };
}

private Mono<Void> updateUserStat(Mono<User> userMono, ServerWebExchange exchange, WebFilterChain chain, String uri) {
    return userMono
            .doOnNext(user -> exchange.getAttributes().put("_token", user.getToken()))
            .doOnNext(u -> {
                String token = exchange.getAttribute("_token");
                log.info("[i] Web Filter: token attribute has been set to '{}'", token);
            })
            .flatMap(user -> pageViewRepo.save(new PageView(uri)).flatMap(user::addPageView).flatMap(userRepo::save))
            .doOnNext(user -> {
                int numberOfPages = 0;
                List<PageView> pageViews = user.getPageViews();
                if (pageViews != null) {
                    numberOfPages = pageViews.size();
                }
                log.info("[i] Web Filter: {} has been updated. Number of pages: {}", user, numberOfPages);
            })
            .then(chain.filter(exchange));
}

This code produces the following results:

1) Token is not present: create a new user, create a page view, update the new user, pass a request to controller

2019-01-20 14:39:10.033 [ctor-http-nio-3] : [i] Web Filter: received the request: http://localhost:8080/users?test=784
2019-01-20 14:39:10.110 [ctor-http-nio-3] : Inserting Document containing fields: [token, _class] in collection: user
2019-01-20 14:39:10.206 [ntLoopGroup-2-2] : [i] Web Filter: a new User(id=5c446bee24c86426ac6c0ae5, token=fba944cd-decb-4923-9757-724da5a60061) has been created
2019-01-20 14:39:10.212 [ntLoopGroup-2-2] : [i] Web Filter: token attribute has been set to 'fba944cd-decb-4923-9757-724da5a60061'
2019-01-20 14:39:11.227 [     parallel-1] : Inserting Document containing fields: [URL, createdDate, _class] in collection: pageView
2019-01-20 14:39:11.242 [ntLoopGroup-2-2] : Saving Document containing fields: [_id, token, pageViews, _class]
2019-01-20 14:39:11.256 [ntLoopGroup-2-2] : [i] Web Filter: User(id=5c446bee24c86426ac6c0ae5, token=fba944cd-decb-4923-9757-724da5a60061) has been updated. Number of pages: 1
2019-01-20 14:39:11.289 [ntLoopGroup-2-2] : [i] Controller: handling 'get all users' request. Token attribute is 'fba944cd-decb-4923-9757-724da5a60061'
2019-01-20 14:39:11.369 [ntLoopGroup-2-2] : find using query: { } fields: Document{{}} for class: class io.github.cepr0.demo.User in collection: user

2) Token is present: find an existing user, create a page view, update the user, pass a request to controller

2019-01-20 14:51:21.983 [ctor-http-nio-3] : [i] Web Filter: received the request: http://localhost:8080/users?test=538
2019-01-20 14:51:22.074 [ctor-http-nio-3] : Created query Query: { "token" : "b613b810-cc36-4961-ad2e-db44f52cd2dd" }, Fields: { }, Sort: { }
2019-01-20 14:51:22.092 [ctor-http-nio-3] : find using query: { "token" : "b613b810-cc36-4961-ad2e-db44f52cd2dd" } fields: Document{{}} for class: class User in collection: user
2019-01-20 14:51:22.102 [ntLoopGroup-2-2] : [i] Web Filter: User(id=5c434c2eb338ac3530cbd56d, token=b613b810-cc36-4961-ad2e-db44f52cd2dd) has been found
2019-01-20 14:51:22.102 [ntLoopGroup-2-2] : [i] Web Filter: token attribute has been set to 'b613b810-cc36-4961-ad2e-db44f52cd2dd'
2019-01-20 14:51:23.103 [     parallel-2] : Inserting Document containing fields: [URL, createdDate, _class] in collection: pageView
2019-01-20 14:51:23.115 [ntLoopGroup-2-2] : Saving Document containing fields: [_id, token, pageViews, _class]
2019-01-20 14:51:23.117 [ntLoopGroup-2-2] : [i] Web Filter: User(id=5c434c2eb338ac3530cbd56d, token=b613b810-cc36-4961-ad2e-db44f52cd2dd) has been updated. Number of pages: 13
2019-01-20 14:51:23.118 [ntLoopGroup-2-2] : [i] Controller: handling 'get all users' request. Token attribute is 'b613b810-cc36-4961-ad2e-db44f52cd2dd'
2019-01-20 14:51:23.119 [ntLoopGroup-2-2] : find using query: { } fields: Document{{}} for class: class User in collection: user

3) Token is present but user is not found: pass a request to controller

2019-01-20 14:52:41.842 [ctor-http-nio-3] : [i] Web Filter: received the request: http://localhost:8080/users?test=513
2019-01-20 14:52:41.844 [ctor-http-nio-3] : Created query Query: { "token" : "-b613b810-cc36-4961-ad2e-db44f52cd2dd" }, Fields: { }, Sort: { }
2019-01-20 14:52:41.845 [ctor-http-nio-3] : find using query: { "token" : "-b613b810-cc36-4961-ad2e-db44f52cd2dd" } fields: Document{{}} for class: class User in collection: user
2019-01-20 14:52:41.850 [ntLoopGroup-2-2] : [i] Controller: handling 'get all users' request. Token attribute is 'null'
2019-01-20 14:52:41.850 [ntLoopGroup-2-2] : find using query: { } fields: Document{{}} for class: class User in collection: user

Demo: sb-reactive-filter-demo(branch: update-user-in-web-filter)

Answer 2

For gathering page view stats I'll suggest to change the strategy and use Actuator and Micrometer instead:

1. Add the actuator dependency to your project
2. Expose the relevant endpoints (here, metrics)
3. Go to /actuator/metrics and select the metric for server HTTP requests (see the reference documentation).

Micrometer offers way more and helps you to get your metrics right, like: taking into account GC pauses when measuring time, providing histograms/percentiles/..., and more.

Answer 3

If I understand you correctly, you need to perform long operations with database asynchronously to prevent the filter (and the request itself) from blocking?

In this case, I would recommend the following solution that works for me:

@Bean
public WebFilter filter() {
    return (exchange, chain) -> {
        ServerHttpRequest req = exchange.getRequest();
        String uri = req.getURI().toString();
        log.info("[i] Got request: {}", uri);

        var headers = req.getHeaders();
        List<String> tokenList = headers.get("token");

        if (tokenList != null && tokenList.get(0) != null) {
            String token = tokenList.get(0);
            log.info("[i] Find a user by token {}", token);
            return userRepo.findByToken(token)
                    .map(user -> process(exchange, uri, token, user))
                    .then(chain.filter(exchange));
        } else {
            String token = UUID.randomUUID().toString();
            log.info("[i] Create a new user with token {}", token);
            return userRepo.save(new User(token))
                    .map(user -> process(exchange, uri, token, user))
                    .then(chain.filter(exchange));
        }
    };
}

Here I slightly change your logic and take the token value from the appropriate header (not from cookies) to simplify my implementation.

So if the token is present then we try to find its user. If the token isn't present then we create a new user. If the user is found or created successfully, then the process method is calling. After that, regardless of the result, we return chain.filter(exchange).

The method process puts a token value to the appropriate attribute of the request and call asynchronously the method updateUserStat of the userService:

private User process(ServerWebExchange exchange, String uri, String token, User user) {
    exchange.getAttributes().put("_token", token);
    userService.updateUserStat(uri, user); // async call
    return user;
}

User service:

@Slf4j
@Service
public class UserService {

    private final UserRepo userRepo;
    private final PageViewRepo pageViewRepo;

    public UserService(UserRepo userRepo, PageViewRepo pageViewRepo) {
        this.userRepo = userRepo;
        this.pageViewRepo = pageViewRepo;
    }

    @SneakyThrows
    @Async
    public void updateUserStat(String uri, User user) {
        log.info("[i] Start updating...");
        Thread.sleep(1000);
        pageViewRepo.save(new PageView(uri))
                .flatMap(user::addPageView)
                .blockOptional()
                .ifPresent(u -> userRepo.save(u).block());
        log.info("[i] User updated.");
    }
}

I've added here a small delay for test purposes to make sure that requests work without any delay, regardless of the duration of this method.

A case when the user is found by the token:

2019-01-06 18:25:15.442  INFO 4992 --- [ctor-http-nio-3] : [i] Got request: http://localhost:8080/users?test=1000
2019-01-06 18:25:15.443  INFO 4992 --- [ctor-http-nio-3] : [i] Find a user by token 84b0f7ec-670c-4c04-8a7c-b692752d7cfa
2019-01-06 18:25:15.444 DEBUG 4992 --- [ctor-http-nio-3] : Created query Query: { "token" : "84b0f7ec-670c-4c04-8a7c-b692752d7cfa" }, Fields: { }, Sort: { }
2019-01-06 18:25:15.445 DEBUG 4992 --- [ctor-http-nio-3] : find using query: { "token" : "84b0f7ec-670c-4c04-8a7c-b692752d7cfa" } fields: Document{{}} for class: class User in collection: user
2019-01-06 18:25:15.457  INFO 4992 --- [ntLoopGroup-2-2] : [i] Get all users...
2019-01-06 18:25:15.457  INFO 4992 --- [         task-3] : [i] Start updating...
2019-01-06 18:25:15.458 DEBUG 4992 --- [ntLoopGroup-2-2] : find using query: { } fields: Document{{}} for class: class User in collection: user
2019-01-06 18:25:16.459 DEBUG 4992 --- [         task-3] : Inserting Document containing fields: [URL, createdDate, _class] in collection: pageView
2019-01-06 18:25:16.476 DEBUG 4992 --- [         task-3] : Saving Document containing fields: [_id, token, pageViews, _class]
2019-01-06 18:25:16.479  INFO 4992 --- [         task-3] : [i] User updated.

Here we can see that updating the user is performed in the independent task-3 thread after the user already has a result of 'get all users' request.

A case when the token is not present and the user is created:

2019-01-06 18:33:54.764  INFO 4992 --- [ctor-http-nio-3] : [i] Got request: http://localhost:8080/users?test=763
2019-01-06 18:33:54.764  INFO 4992 --- [ctor-http-nio-3] : [i] Create a new user with token d9bd40ea-b869-49c2-940e-83f1bf79e922
2019-01-06 18:33:54.765 DEBUG 4992 --- [ctor-http-nio-3] : Inserting Document containing fields: [token, _class] in collection: user
2019-01-06 18:33:54.776  INFO 4992 --- [ntLoopGroup-2-2] : [i] Get all users...
2019-01-06 18:33:54.777  INFO 4992 --- [         task-4] : [i] Start updating...
2019-01-06 18:33:54.777 DEBUG 4992 --- [ntLoopGroup-2-2] : find using query: { } fields: Document{{}} for class: class User in collection: user
2019-01-06 18:33:55.778 DEBUG 4992 --- [         task-4] : Inserting Document containing fields: [URL, createdDate, _class] in collection: pageView
2019-01-06 18:33:55.792 DEBUG 4992 --- [         task-4] : Saving Document containing fields: [_id, token, pageViews, _class]
2019-01-06 18:33:55.795  INFO 4992 --- [         task-4] : [i] User updated.

A case when the token is present but user is not found:

2019-01-06 18:35:40.970  INFO 4992 --- [ctor-http-nio-3] : [i] Got request: http://localhost:8080/users?test=150
2019-01-06 18:35:40.970  INFO 4992 --- [ctor-http-nio-3] : [i] Find a user by token 184b0f7ec-670c-4c04-8a7c-b692752d7cfa
2019-01-06 18:35:40.972 DEBUG 4992 --- [ctor-http-nio-3] : Created query Query: { "token" : "184b0f7ec-670c-4c04-8a7c-b692752d7cfa" }, Fields: { }, Sort: { }
2019-01-06 18:35:40.972 DEBUG 4992 --- [ctor-http-nio-3] : find using query: { "token" : "184b0f7ec-670c-4c04-8a7c-b692752d7cfa" } fields: Document{{}} for class: class User in collection: user
2019-01-06 18:35:40.977  INFO 4992 --- [ntLoopGroup-2-2] : [i] Get all users...
2019-01-06 18:35:40.978 DEBUG 4992 --- [ntLoopGroup-2-2] : find using query: { } fields: Document{{}} for class: class User in collection: user

My demo project: sb-reactive-filter-demo