DataBufferLimitException: Exceeded limit on max bytes to buffer webflux error

Question

While sending a file I receive an array of bytes. I always have a problem with webflux to receive an array. the error thrown as below :

org.springframework.co         


        

Answer 1

This workerd for me

1. Create a bean in your one of the configuration class or the main Springbootapplication class

   @Bean
   public WebClient getWebClientBuilder(){
       return   WebClient.builder().exchangeStrategies(ExchangeStrategies.builder()
               .codecs(configurer -> configurer
                         .defaultCodecs()
                         .maxInMemorySize(16 * 1024 * 1024))
                       .build())
                     .build();
   }
   

2. Next go to your desired class where you want to use the webclient

     @RestController / @Bean/ @Service
      public class PaySharpGatewayController {
           @Autowired
           WebClient webClient;
   
           public void test(){
            String out = webClient
                         .get()
                         .uri("end point of an API")
                         .retrieve()
                         .bodyToMono(String.class)
                        .block();
   
            sysout(out)
           }
   

Answer 2

Set the maximum bytes (in megabytes) in you spring boot configuration file like below:

spring.codec.max-in-memory-size=20MB

Answer 3

Instead of retrieving data at once, you can stream:

Mono<String> string = webClient.get()
    .uri("end point of an API")
    .retrieve()
    .bodyToFlux(DataBuffer.class)
    .map(buffer -> {
        String string = buffer.toString(Charset.forName("UTF-8"));
        DataBufferUtils.release(buffer);
        return string;
    });

Alternatively convert to stream:

    .map(b -> b.asInputStream(true))
    .reduce(SequenceInputStream::new)
    .map(stream -> {
        // consume stream
        stream.close();
        return string;
    });

In most cases you don't want to really aggregate the stream, rather than processing it directly. The need to load huge amount of data in memory is mostly a sign to change the approach to more reactive one. JSON- and XML-Parsers have streaming interfaces.

Answer 4

I suppose this issue is about adding a new spring.codec.max-in-memory-size configuration property in Spring Boot. Add it to the properties like:

spring:
  codec:
    max-in-memory-size: 10MB

Answer 5

i was getting this error for a simple RestController (i post a large json string).

here is how i successfully changed the maxInMemorySize

import org.springframework.context.annotation.Configuration;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.web.reactive.config.ResourceHandlerRegistry;
import org.springframework.web.reactive.config.WebFluxConfigurer;

@Configuration
public class WebfluxConfig implements WebFluxConfigurer {

    @Override
    public void addResourceHandlers(ResourceHandlerRegistry registry) {

        registry.addResourceHandler("/swagger-ui.html**")
            .addResourceLocations("classpath:/META-INF/resources/");

        registry.addResourceHandler("/webjars/**")
            .addResourceLocations("classpath:/META-INF/resources/webjars/");
    }

    @Override
    public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
        configurer.defaultCodecs().maxInMemorySize(16 * 1024 * 1024);
    }
}

this was surprisingly hard to find